Supervised Learning of Procedures from Tutorial Videos - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2022

Supervised Learning of Procedures from Tutorial Videos

Abstract

Online educational platforms and MOOCs (Massive Open Online Courses) have made it so that learning can happen from anywhere across the globe. While this is extremely beneficial, videos are not accessible by everyone, due to time constraints and lower bandwidths. Textual step-by-step instructions will serve as a good alternative, being less time-consuming to follow than videos, and also requiring lesser bandwidth. In this work, the authors aim to create a system that extracts and presents a step-by-step tutorial from a tutorial video, which makes it much easier to follow as per the user’s convenience. This can be mainly accomplished by using Text Recognition for academic videos and Action Recognition for exercise videos. Text Recognition is accomplished using the Pytesseract package from Python which performs OCR. Action Recognition is performed with the help of a pre-trained OpenPose model. Additional information is extracted from both exercise and academic videos with the help of speech recognition. The information extracted from the videos are then segmented into procedural instructions which is easy to comprehend. For exercise videos, a frame-wise accuracy of 79.27% is obtained. Additionally, for academic videos, an accuracy of 78.47% is obtained.
Fichier principal
Vignette du fichier
526570_1_En_28_Chapter.pdf (533.46 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04381287 , version 1 (09-01-2024)

Licence

Identifiers

Cite

S. Arunima, Amlan Sengupta, Aparna Krishnan, D. Venkata Vara Prasad, Lokeswari Y. Venkataramana. Supervised Learning of Procedures from Tutorial Videos. 5th International Conference on Computational Intelligence in Data Science (ICCIDS), Mar 2022, Virtual, India. pp.356-366, ⟨10.1007/978-3-031-16364-7_28⟩. ⟨hal-04381287⟩
23 View
5 Download

Altmetric

Share

More