Using Global Statistics to Rank Retrieval Systems without Relevance Judgments - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2010

Using Global Statistics to Rank Retrieval Systems without Relevance Judgments

Résumé

How to reduce the amount of relevance judgments is an important issue in retrieval evaluation. In this paper, we propose a novel method using global statistics to rank retrieval systems without relevance judgments. In our method, a series of global statistics of a system, which indicate the percentage of its documents found by k out of all the N systems (k = 1, 2, ..., N), are selected, then a linear combination of the series of global statistics is utilized to fit the mean average precision (MAP) of the retrieval system. Optimal coefficients are obtained by linear regression. No human relevance judgments are required in the entire process. Compared with existing evaluation methods without relevance judgments, our method has two advantages. Firstly, it outperforms all early attempts. Secondly, it is adjustable for different effectiveness measurements, e.g. MAP, precision at n, and so forth.
Fichier principal
Vignette du fichier
Using_Global_Statistics_to_Rank_Retrieval_Systems_without_Relevance_Judgments.pdf (274.38 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01055060 , version 1 (11-08-2014)

Licence

Paternité

Identifiants

Citer

Zhiwei Shi, Bin Wang, Peng Li, Zhongzhi Shi. Using Global Statistics to Rank Retrieval Systems without Relevance Judgments. 6th IFIP TC 12 International Conference on Intelligent Information Processing (IIP), Oct 2010, Manchester, United Kingdom. pp.183-192, ⟨10.1007/978-3-642-16327-2_24⟩. ⟨hal-01055060⟩
117 Consultations
102 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More