Ordinal Regression for Interaction Quality Prediction
Résumé
The automatic prediction of the quality of a dialogue is useful to
keep track of a spoken dialogue system’s performance and, if necessary,
adapt its behaviour. Classifiers and regression models have
been suggested to make this prediction. The parameters of these
models are learnt from a corpus of dialogues evaluated by users or
experts. In this paper, we propose to model this task as an ordinal regression
problem. We apply support vector machines for ordinal regression
on a corpus of dialogues where each system-user exchange
was given a rate on a scale of 1 to 5 by experts. Compared to previous
models proposed in the literature, the ordinal regression predictor
has significantly better results according to the following evaluation
metrics: Cohen’s agreement rate with experts ratings, Spearman’s
rank correlation coefficient, and Euclidean and Manhattan errors.