Analyzing Statistical Tests of Search Engine Bias - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

Analyzing Statistical Tests of Search Engine Bias


Search engines play a significant role in shaping the Internet, but there are concerns about the motivation behind their rankings: some content and service providers have complained about being ranked abusively low to favor engines' commercially-close content, initiating the so-called search neutrality debate. There are similar worries about possible orientations towards some ideologies or political points of view. Whatever the opinion of the reader on this sensitive issue of the definition and need for search engine neutrality, it is important to have at our disposal tools to monitor search engines behavior and understand deviations from "expected" results. The goal of this paper is to review existing statistical tests of potential bias by search engines, compare and combine them both formally and numerically. We end up with a battery of ANOVA and Dixon Q tests for which we characterize the bias detection probability and false positive probability, the latter requiring to be minimal if tests have to be used officially and publicly. We also run the tests on a campaign of searches on real-life search engines and discuss the outcome.
Fichier principal
Vignette du fichier
comparison.pdf (712.7 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : CC BY - Attribution

Dates and versions

hal-04360118 , version 1 (21-12-2023)




  • HAL Id : hal-04360118 , version 1


Patrick Maillé, Nils Peyrouset, Bruno Tuffin. Analyzing Statistical Tests of Search Engine Bias. 2023. ⟨hal-04360118⟩
25 View
17 Download


Gmail Facebook X LinkedIn More