Evaluating LIS Instructional Videos: A Comparative Study of AI and Human Assessments

Vijayakumar S; Harinarayana N. S.

Vijayakumar S Research Student, DoS in Library and Information Science, University of Mysore, Mysuru, India.
Harinarayana N. S. Senior Professor, DoS in Library and Information Science, University of Mysore, Mysuru, India.

Abstract

The integration of artificial intelligence (AI) into educational content evaluation offers a promising avenue for enhancing assessment quality and scalability. This study compares AI-generated evaluations, using ChatGPT, with human assessments for 10 Library and Information Science (LIS) instructional videos from the National Institute of Open Schooling (NIOS). Key evaluation features include video resolution, content relevance, engagement, and clarity. Results show that while AI aligned well with human evaluators for objective features such as resolution and duration, discrepancies were significant in subjective areas like engagement and clarity, with error rates exceeding 75% for some videos. Correlation patterns revealed that content complexity and design flaws influenced evaluation alignment. The study underscores the need for hybrid frameworks combining AI’s efficiency with human expertise to improve instructional video evaluations. Recommendations include refining AI for qualitative assessments and optimising video content design, contributing to more reliable evaluation methods in educational contexts.

References

1. Mayer, R. E. (2021). The Cambridge handbook of multimedia learning. In Multimedia Learning. Cambridge University Press. https://doi. org/10.1017/9781316941355
2. Hodges CB, Moore S, Lockee BB, Trust T, Bond MA. The difference between emergency remote teaching and online learning.
3. The National Institute of Open Schooling. (2002). NIOS. https://www.nios.ac.in/
4. Sharma, P., & Gupta, R. (2019). Assessing the quality of online educational resources: A framework for
instructional design. Journal of Educational Technology, 16(3), 45–56.
5. Wong, G., Apthorpe, H. C., Ruiz, K., & Nanayakkara, S. (2018). An innovative educational approach in using
instructional videos to teach dental local anaesthetic skills. European Journal of Dental Education, 23(1), 28–34. https://doi.org/1