Perturbed Model Validation: A New Framework to Validate Model Relevance - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year :

Perturbed Model Validation: A New Framework to Validate Model Relevance

Abstract

This paper introduces Perturbed Model Validation (PMV), a new technique to validate model relevance and detect overfitting or underfitting. PMV operates by injecting noise to the training data, re-training the model against the perturbed data, then using the training accuracy decrease rate to assess model relevance. A larger decrease rate indicates better concept-hypothesis fit. We realise PMV by perturbing labels to inject noise, and evaluate PMV on four real-world datasets (breast cancer, adult, connect-4, and MNIST) and nine synthetic datasets in the classification setting. The results reveal that PMV selects models more precisely and in a more stable way than cross-validation, and effectively detects both overfitting and underfitting.
Fichier principal
Vignette du fichier
main.pdf (1.06 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02139208 , version 1 (24-05-2019)
hal-02139208 , version 2 (27-05-2019)

Identifiers

  • HAL Id : hal-02139208 , version 2

Cite

Jie Zhang, Earl T Barr, Benjamin Guedj, Mark Harman, John Shawe-Taylor. Perturbed Model Validation: A New Framework to Validate Model Relevance. 2019. ⟨hal-02139208v2⟩
273 View
641 Download

Share

Gmail Facebook Twitter LinkedIn More