Evaluation Methods of Cause-Effect Pairs - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Book Sections Year : 2019

Evaluation Methods of Cause-Effect Pairs


This chapter addresses the problem of benchmarking causal models or validating particular putative causal relationships, in the limited setting of cause-effect pairs, when empirical “observational” data are available. We do not address experimental validations e.g. via randomized controlled trials. Our goal is to compare methods, which provide a score C(X, Y ), called causation coefficient, rating a pair of variable (X, Y ) for being in a potential causal relationship X → Y . Causation coefficients may be used for various purposes, including to prioritize experiments, which may be costly or risky, or guiding decision makers in domains in which experiments are infeasible or unethical. We provide a methodology to evaluate their reliability. We take three points of views: (1) that of algorithm developers who must justify the soundness of their method, particularly with respect to identifiability and consistency, (2) that of practitioners who seek to understand on what basis algorithms make their decisions and evaluate their statistical significance, and (3) that of benchmark organizers who desire to make fair evaluations to compare methods. We adopt the framework of pattern recognition in which pairs of variable (X, Y ) and their ground truth causal graph are drawn i.i.d. from a “mother distribution”. This leads us to define new notions of probabilistic identifiability, Bayes optimal causation coefficients, and multi-part statistical tests. These new notions are evaluated on the data of the first cause-effect pair challenge. We also compile a list of resources, including datasets of real or synthetic pairs, and data generative models.
No file

Dates and versions

hal-02433198 , version 1 (09-01-2020)



Isabelle Guyon, Olivier Goudet, Diviyan Kalainathan. Evaluation Methods of Cause-Effect Pairs. Guyon, Isabelle; Statnikov, Alexander; Batu, Berna Bakir. Cause Effect Pairs in Machine Learning, Springer Verlag, pp.27-99, 2019, The Springer Series on Challenges in Machine Learning, 978-3-030-21809-6. ⟨10.1007/978-3-030-21810-2_2⟩. ⟨hal-02433198⟩
101 View
0 Download



Gmail Facebook X LinkedIn More