We present in this paper a new method for the design of customizable self-evolving fuzzy rule-based classifiers. The presented approach is based on a first-order Takagi-Sugeno fuzzy inference system. This approach involves first an incremental clustering and adaptation of the premise part of the system, and secondly, an incremental learning of the linear consequents parameters of the system using a modified version of the Recursive Least Square method. We use this method to build an evolving handwritten gesture recognition system. The self-adaptive nature of this system allows starting the learning process by few learning data, to continuously adapt and evolve according to any new data, and to keep robust when introducing a new unseen class at any moment in the life-long learning process.