Learning from Positive and Unlabeled Examples - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Theoretical Computer Science Year : 2005

Learning from Positive and Unlabeled Examples

Abstract

In many machine learning settings, labeled examples are difficult to collect while unlabeled data are abundant. Also, for some binary classification problems, positive examples, that is examples of the target class, are available. Can these additional data be used to improve accuracy of supervised learning algorithms? We investigate in this paper the design of learning algorithms from positive and unlabeled data only. Many machine learning and data mining algorithms, such as decision tree induction algorithms and naive Bayes algorithms, only use examples in order to evaluate statistical queries (SQ-like algorithms). Kearns designed the Statistical Query learning model in order to describe these algorithms. Here, we design an algorithm scheme which transforms any SQ-like algorithm into an algorithm based on positive statistical queries (estimates for probabilities over the set of positive instances) and instance statistical queries (estimates for probabilities over the instance space). We prove that any class learnable in the Statistical Query learning model is learnable from positive statistical queries and instance statistical queries only if a lower bound on the weight of any target concept $f$ can be estimated in polynomial time. Then, we design a decision tree induction algorithm POSC4.5, based on C4.5, that uses only positive and unlabeled examples and we give experimental results for this algorithm. The case of imbalanced classes in the sense that one of the two classes (say the positive class) is heavily underrepresented compared to the other class remains open. This problem is challenging because it is encountered in many real-world applications.
Fichier principal
Vignette du fichier
onlypos02.pdf (305.35 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

inria-00536692 , version 1 (16-11-2010)

Identifiers

Cite

François Denis, Rémi Gilleron, Fabien Letouzey. Learning from Positive and Unlabeled Examples. Theoretical Computer Science, 2005, 348 (1), pp.70-83. ⟨10.1016/j.tcs.2005.09.007⟩. ⟨inria-00536692⟩
266 View
428 Download

Altmetric

Share

Gmail Facebook X LinkedIn More