On agnostic post hoc approaches to false positive control - Inria - Institut national de recherche en sciences et technologies du numérique
Book Sections Year : 2021

On agnostic post hoc approaches to false positive control

Abstract

Classical approaches to multiple testing grant control over the amount of false positives for a specific method prescribing the set of rejected hypotheses. On the other hand, in practice, many users tend to deviate from a strictly prescribed multiple testing method and follow ad-hoc rejection rules, tune some parameters by hand, compare several methods, or pick from their results the one that suits them best. This will invalidate standard statistical guarantees because of the selection effect. To compensate for any form of such """data snooping""", an approach that has garnered interest recently is to derive """user-agnostic""", or post hoc, bounds on the false positives valid uniformly over all possible rejection sets; this allows arbitrary data snooping from the user. In this chapter, we present a general approach to this problem using a family of candidate rejection subsets (call this a reference family) together with associated bounds on the number of false positives they contain, the latter holding uniformly over the family. We illustrate the interest of our approach in different contexts, for different choices of the reference family, and apply it to a genomic example (differential gene expression) and a neuroimaging example (functional Magnetic Resonance Imaging).
Fichier principal
Vignette du fichier
BNR-chap_HAL-revised.pdf (1.4 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-02320543 , version 1 (21-10-2019)
hal-02320543 , version 2 (27-10-2020)

Identifiers

Cite

Gilles Blanchard, Pierre Neuvial, Etienne Roquain. On agnostic post hoc approaches to false positive control. Xinping Cui, Thorsten Dickhaus, Ying Ding, Jason C. Hsu. Handbook of Multiple Comparisons, 1, Chapman and Hall/CRC, pp.211-232, 2021, Handbooks of Modern Statistical Methods, 9780367140670. ⟨10.1201/9780429030888-9⟩. ⟨hal-02320543v2⟩
466 View
218 Download

Altmetric

Share

More