Analysis of the Multiple Choice Questions exam in the course of health and disease of newborns to adolescents 1 for 4th year medical students, faculty of Medicine, Naresuan University

Abstract

Introduction: Multiple Choice Questions (MCQs) are widely used in knowledge assessments because they can evaluate multiple objectives through numerous items. Following each exam, a test analysis is conducted to determine the validity and reliability of these questions. This study assessed the quality of MCQs used in the "Health and Diseases in Newborns to Adolescents I" course for fourth-year medical students of the Faculty of Medicine, Naresuan University. The evaluation included analyzing individual question quality and overall test set effectiveness and examining the relationship between question difficulty levels before and after the exam.

Methodology: The study was retrospective data from medical student exams in the 2020 academic year. There were four sets of 100 questions. Sets 1-4 were tested with 65, 50, 49, and 13 students. All MCQ test sets have been reviewed and assessed for quality by an examination validity assessment committee to ensure it according to the purpose of the course. The 400 answers to questions were analyzed as a difficulty (p) and distributing (r) of the exam. Statistical tools were applied to examine relationships, including mean, standard deviation, KR-20 reliability coefficient, and Pearson correlation.

Results: The results showed that the item difficulty for all four test sets ranged from 0.20 to 0.80, with 65.75% of items falling within this range, and item discrimination values above 0.35 for 22.75% of items, indicating good discrimination power. The validity of the test sets was confirmed, as they aligned with the course objectives, content, and structure. Reliability coefficients for all four sets were found to be acceptable (≥0.70). The correlation between the difficulty levels of questions before and after the exam ranged from 0.31 to 0.50, suggesting consistency between pre- and post-exam difficulty levels.

Conclusion: This study analyzed the quality of multiple-choice questions (MCQs) in the Health and Diseases in Newborns to Adolescents I for 4^th year medical students. The findings indicate that most questions had appropriate difficulty and discrimination indices, with validity and reliability at acceptable levels. There was a moderate correlation between pre- and post-exam difficulty levels.