Comparison of The Reliability of The Written Mathematical Communication Skills Test for Students in Grade 6: Application of Generalizability Theory - Sakkamart, Monchaya; Srihaset, Kamontip; Tippayakulpairoj, Darunee (Interdisciplinary Academic and Research Journal, 2025)
Background and Aims: Mathematical communication skills are essential for effective mathematics instruction. Therefore, the assessment of written mathematical communication skills requires high-quality instruments. This research aimed to: 1) develop an assessment tool for evaluating written mathematical communication skills of Grade 6 students and 2) compare the reliability of the assessment tool for written mathematical communication skills of Grade 6 students with the application of generalizability theory using a fully crossed design. In which all raters score all items from all examinees. Methodology: The sample used in this research consisted of 80 Grade 6 students from the Ramkhamhaeng University Demonstration School (Bangna Campus) in the academic year 2024. The sample size was determined using the G*Power 3.1.9.7 program, and cluster sampling was employed. The research instrument was an assessment tool for written mathematical communication skills. The statistical methods used for data analysis included basic descriptive statistics, the kappa coefficient to assess inter-rater reliability, and generalizability coefficients. Results: The research findings revealed that: 1) the developed assessment tool for written mathematical communication skills on the topic of fractions had a content validity index ranging from 0.67 to 1.00. The overall reliability of the test was 0.762. The kappa coefficients for inter-rater reliability ranged from 0.210 to 0.645. The difficulty index ranged from 0.393 to 0.668, and the discrimination index ranged from 0.379 to 0.882. 2) The assessment tool for written mathematical communication skills with a reliability coefficient of no less than 0.80 can be selected as follows: the generalizability coefficients for both relative and absolute decisions were no less than 0.80 when 2 or 3 raters scored at least 10 items. Conclusion: The study of the Generalizability Coefficient (G-Coefficient) aimed to identify an assessment tool that is practically applicable and has a reliability of 0.80. The research findings indicated that the written mathematical communication skills assessment tool suitable for use in developing and improving the quality of instruction on fractions should consist of at least 10 items and be scored by 2 raters.