KANDINSKY Patterns as IQ-Test for Machine Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2019

KANDINSKY Patterns as IQ-Test for Machine Learning

Andreas Holzinger
  • Function : Author
  • PersonId : 1017033
Michael Kickmeier-Rust
  • Function : Author
  • PersonId : 1067025
Heimo Müller
  • Function : Author
  • PersonId : 1067027


AI follows the notion of human intelligence which is unfortunately not a clearly defined term. The most common definition given by cognitive science as mental capability, includes, among others, the ability to think abstract, to reason, and to solve problems from the real world. A hot topic in current AI/machine learning research is to find out whether and to what extent algorithms are able to learn abstract thinking and reasoning similarly as humans can do – or whether the learning outcome remains on purely statistical correlation. In this paper we provide some background on testing intelligence, report some preliminary results from 271 participants of our online study on explainability, and propose to use our Kandinsky Patterns as an IQ-Test for machines. Kandinsky Patterns are mathematically describable, simple, self-contained hence controllable test data sets for the development, validation and training of explainability in AI. Kandinsky Patterns are at the same time easily distinguishable from human observers. Consequently, controlled patterns can be described by both humans and computers. The results of our study show that the majority of human explanations was made based on the properties of individual elements in an image (i.e., shape, color, size) and the appearance of individual objects (number). Comparisons of elements (e.g., more, less, bigger, smaller, etc.) were significantly less likely and the location of objects, interestingly, played almost no role in the explanation of the images. The next step is to compare these explanations with machine explanations.
Fichier principal
Vignette du fichier
485369_1_En_1_Chapter.pdf (856.94 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02520058 , version 1 (26-03-2020)





Andreas Holzinger, Michael Kickmeier-Rust, Heimo Müller. KANDINSKY Patterns as IQ-Test for Machine Learning. 3rd International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Aug 2019, Canterbury, United Kingdom. pp.1-14, ⟨10.1007/978-3-030-29726-8_1⟩. ⟨hal-02520058⟩
115 View
224 Download



Gmail Facebook X LinkedIn More