Introduction to multivariate discrimination - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2013

Introduction to multivariate discrimination

Balázs Kégl
  • Function : Author
  • PersonId : 842855


Multivariate discrimination or classification is one of the best-studied problem in machine learning, with a plethora of well-tested and well-performing algorithms. There are also several good general textbooks [1-9] on the subject written to an average engineering, computer science, or statistics graduate student; most of them are also accessible for an average physics student with some background on computer science and statistics. Hence, instead of writing a generic introduction, we concentrate here on relating the subject to a practitioner experimental physicist. After a short introduction on the basic setup (Section 1) we delve into the practical issues of complexity regularization, model selection, and hyperparameter optimization (Section 2), since it is this step that makes high-complexity non-parametric fitting so different from low-dimensional parametric fitting. To emphasize that this issue is not restricted to classification, we illustrate the concept on a low-dimensional but non-parametric regression example (Section 2.1). Section 3 describes the common algorithmic-statistical formal framework that unifies the main families of multivariate classification algorithms. We explain here the large-margin principle that partly explains why these algorithms work. Section 4 is devoted to the description of the three main (families of) classification algorithms, neural networks, the support vector machine, and AdaBoost. We do not go into the algorithmic details; the goal is to give an overview on the form of the functions these methods learn and on the objective functions they optimize. Besides their technical description, we also make an attempt to put these algorithm into a socio-historical context. We then briefly describe some rather heterogeneous applications to illustrate the pattern recognition pipeline and to show how widespread the use of these methods is (Section 5). We conclude the chapter with three essentially open research problems that are either relevant to or even motivated by certain unorthodox applications of multivariate discrimination in experimental physics.
Fichier principal
Vignette du fichier
epjconf_sos2012_02001.pdf (297.97 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

in2p3-00846125 , version 1 (18-07-2013)



Balázs Kégl. Introduction to multivariate discrimination. IN2P3 School of Statistics (SOS2012), May 2012, Autrans, France. pp.022001, ⟨10.1051/epjconf/20135502001⟩. ⟨in2p3-00846125⟩
92 View
138 Download



Gmail Mastodon Facebook X LinkedIn More