Communication Dans Un Congrès Année : 2025

In the Search for Truth: Navigating Variability in Neuroimaging Software Pipelines

Résumé

Neuroimaging pipelines -software-driven analysis workflows of brain images -are characterized by a wide range of tools, parameters, and configuration choices. Such flexibility, while enabling diverse scientific inquiries, gives rise to analytical variability: different pipeline variants can lead to different outcomes. In practice, each neuroimaging pipeline variant produces a statistic map -a complex, structured 3D output whose relevance can only be assessed with specific domain expertise, unlike simple metrics such as execution time. And, in most cases, there is no ground truth against which to judge these outputs, making it unclear which variant yields the best result.

In this paper, we introduce a "sampling, variant scoring, learning" methodology to study variability in the absence of a quantitative target -i.e. ground truth. We report our experience in developing an Universal Variability Language (UVL) feature model of 90 features representing the configuration space of a well-established open source neuroimaging analysis software (SPM). We sample and run 1000 valid configurations generating 1000 statistic maps as outputs.

We studied various candidate proxy ground truths and computed Spearman correlations as a quantitative metric of the performance of each pipeline. We tested the following 12 proxy ground truths: the average statistic map (across variants), the output of an expertderived configuration, and a set of randomly selected outputs (as baseline). Then, we used a decision tree learning approach to inspect variability. We evaluated the sensitivity of our method to the choice of (proxy) ground truth, both in terms of predictive accuracy and in the identification of important features. These first results outline the challenge of choosing and validating a referential to assess our understanding of variability in the absence of ground truth.

Fichier principal
Vignette du fichier
neurovariants-sigconf.pdf (2.35 Mo) Télécharger le fichier
SPLC_25_short_paper.pdf (999.37 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05158426 , version 1 (11-07-2025)

Licence

Identifiants

Citer

Youenn Merel Jourdan, Mathieu Acher, Camille Maumet. In the Search for Truth: Navigating Variability in Neuroimaging Software Pipelines. SPLC 2025 - 29th ACM International Systems and Software Product Line Conference, Association for Computing Machinery (ACM), Sep 2025, Coruna, Spain, Spain. pp.129-135, ⟨10.1145/3744915.3748470⟩. ⟨hal-05158426⟩
2789 Consultations
536 Téléchargements

Altmetric

Partager

  • More