eOSCE stations live versus remote evaluation and scores variability - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles BMC Medical Education Year : 2022

eOSCE stations live versus remote evaluation and scores variability

Nathan Peiffer Smadja
  • Function : Author
Léonore Muller
  • Function : Author
Laure Falque Pierrotin
  • Function : Author
Michael Thy
  • Function : Author
Maksud Assadi
  • Function : Author
Sonia Yung
  • Function : Author
Christian de Tymowski
  • Function : Author
Quentin Le Hingrat
  • Function : Author
Xavier Eyer
  • Function : Author
Paul Henri Wicky
  • Function : Author
Mehdi Oualha
  • Function : Author
Véronique Houdouin
  • Function : Author
Patricia Jabre
  • Function : Author
Dominique Vodovar
  • Function : Author
Marco Dioguardi Burgio
  • Function : Author
Noémie Zucman
  • Function : Author
Asmaa Tazi
  • Function : Author
Quentin Ressaire
  • Function : Author
Yann Nguyen
  • Function : Author
Muriel Girard
  • Function : Author
Adèle Frachon
  • Function : Author
François Depret
  • Function : Author
Anna Pellat
  • Function : Author
Adèle de Masson
  • Function : Author
Henri Azais
  • Function : Author
Nathalie de Castro
  • Function : Author
Caroline Jeantrelle
  • Function : Author
Nicolas Javaud
  • Function : Author
Alexandre Malmartel
  • Function : Author
Constance Jacquin de Margerie
  • Function : Author
Benjamin Chousterman
Ludovic Fournel
  • Function : Author
Mathilde Holleville
  • Function : Author
Stéphane Blanche
  • Function : Author

Abstract

Abstract Background Objective structured clinical examinations (OSCEs) are known to be a fair evaluation method. These recent years, the use of online OSCEs (eOSCEs) has spread. This study aimed to compare remote versus live evaluation and assess the factors associated with score variability during eOSCEs. Methods We conducted large-scale eOSCEs at the medical school of the Université de Paris Cité in June 2021 and recorded all the students’ performances, allowing a second evaluation. To assess the agreement in our context of multiple raters and students, we fitted a linear mixed model with student and rater as random effects and the score as an explained variable. Results One hundred seventy observations were analyzed for the first station after quality control. We retained 192 and 110 observations for the statistical analysis of the two other stations. The median score and interquartile range were 60 out of 100 (IQR 50–70), 60 out of 100 (IQR 54–70), and 53 out of 100 (IQR 45–62) for the three stations. The score variance proportions explained by the rater (ICC rater) were 23.0, 16.8, and 32.8%, respectively. Of the 31 raters, 18 (58%) were male. Scores did not differ significantly according to the gender of the rater ( p = 0.96, 0.10, and 0.26, respectively). The two evaluations showed no systematic difference in scores ( p = 0.92, 0.053, and 0.38, respectively). Conclusion Our study suggests that remote evaluation is as reliable as live evaluation for eOSCEs.

Domains

Education

Dates and versions

hal-03910361 , version 1 (22-12-2022)

Identifiers

Cite

Donia Bouzid, Jimmy Mullaert, Aiham Ghazali, Valentine Marie Ferré, France Mentré, et al.. eOSCE stations live versus remote evaluation and scores variability. BMC Medical Education, 2022, 22 (1), pp.861. ⟨10.1186/s12909-022-03919-1⟩. ⟨hal-03910361⟩
48 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More