This paper is basically concerned with a practical problem: the on-the-fly quick learning of handwritten character recognition systems. More generally, it explores the problem of generating new learning examples, especially from very scarce (2 to 5 per class) original learning data. It presents two different methods. The first one is based on applying distortions on original characters using knowledge on handwriting properties like speed, curvature etc. The second one consists in generation based on the notion of analogical dissimilarity which quantifies the analogical relation “A is to B almost as C is to D”. We give an algorithm to compute the k-least dissimilar objects D, hence generating k new objects from three examples A, B and C. Finally, we experimentally prove on 12 writers the efficiency of both methods, especially when used in conjunction.