EXPLORING THE VALIDITY OF MULTIMEDIA WRITTEN ASSESSMENT IN SAUDI ARABIA
MetadataShow full item record
Introduction: Multiple choice questions are widely used in high-stakes written examinations and are continuously being challenged for testing recall facts rather than higher cognition. Innovative MCQ formats (Multimedia-enhanced) can test such skills. To evaluate this format a recent validity framework “The Cambridge Framework” was used to assess and explore this intervention.
Aim: Explore the validity of multimedia MCQs for testing higher cognition in Emergency Medicine (EM) in Saudi Arabia using the Cambridge framework, and evaluate the use of this framework.
Methods: A total of 164 EM residents (seniors and juniors) from three regions of Saudi Arabia took a total of 80 multimedia-text matched items in an end of year exam. A mixed-methods approach triangulating quantitative (pilot test, parallel forms of multimedia and text items, item psychometrics and characteristics, questionnaires) and qualitative methods (semi-systematic literature review, focus-group discussions, Cambridge validity framework implementation and research legitimation), were applied using systematic guidelines for each method to explore multimedia items.
Results: Discrimination was significantly higher for multimedia than text items (DI= 0.19 +18, 0.14 +17, p= .03), and took significantly longer to answer (p=.01). Both formats had a moderate difficulty level (Diff = 0.75,0.74). Multimedia-items had a higher reliability and G-coefficient than text items. Focus group results revealed seven main themes of multimedia that effect item characteristics. Review of the Cambridge Framework demonstrated areas of gaps in sources of validity evidence and external-related factors not covered in other frameworks that have implications on validity.
Conclusion: Multimedia questions were more discriminating and took longer time to answer than the text questions. They test higher cognition and have certain characteristics that effect difficulty level and how they are perceived by the examinees altering their thinking process towards the answer. Gap areas identified in the Cambridge framework and external factors that were contextual, may give room to explore the issue of international validity in assessment.
Showing items related by title, author, creator and subject.
Salmons, Nichola Jane (University of Plymouth, 2000)The thesis considers the issue of multimedia data utilisation within modem health care delivery and the consequent need for an appropriate patient records system. The discussions centre upon the deployment and utilisation ...
A QoE-driven Vertical Handover Management Framework for Multimedia Services over Wireless Networks Liu, Li (University of Plymouth, 2017)With advances in wireless technology and mobile devices, the number of mobile users using multimedia services has increased significantly in recent years. Mobile devices can be connected and roam on heterogeneous wireless ...
Rogers, Thomas John (University of Plymouth, 2000)Interactive multimedia offers a degree of richness that lies outside the scope of conventional design methods for computer based learning. This research seeks to develop an interdisciplinary approach to design, that ...