EXPLORING THE VALIDITY OF MULTIMEDIA WRITTEN ASSESSMENT IN SAUDI ARABIA
Date
2019Author
Subject
Metadata
Show full item recordAbstract
Introduction: Multiple choice questions are widely used in high-stakes written examinations and are continuously being challenged for testing recall facts rather than higher cognition. Innovative MCQ formats (Multimedia-enhanced) can test such skills. To evaluate this format a recent validity framework “The Cambridge Framework” was used to assess and explore this intervention.
Aim: Explore the validity of multimedia MCQs for testing higher cognition in Emergency Medicine (EM) in Saudi Arabia using the Cambridge framework, and evaluate the use of this framework.
Methods: A total of 164 EM residents (seniors and juniors) from three regions of Saudi Arabia took a total of 80 multimedia-text matched items in an end of year exam. A mixed-methods approach triangulating quantitative (pilot test, parallel forms of multimedia and text items, item psychometrics and characteristics, questionnaires) and qualitative methods (semi-systematic literature review, focus-group discussions, Cambridge validity framework implementation and research legitimation), were applied using systematic guidelines for each method to explore multimedia items.
Results: Discrimination was significantly higher for multimedia than text items (DI= 0.19 +18, 0.14 +17, p= .03), and took significantly longer to answer (p=.01). Both formats had a moderate difficulty level (Diff = 0.75,0.74). Multimedia-items had a higher reliability and G-coefficient than text items. Focus group results revealed seven main themes of multimedia that effect item characteristics. Review of the Cambridge Framework demonstrated areas of gaps in sources of validity evidence and external-related factors not covered in other frameworks that have implications on validity.
Conclusion: Multimedia questions were more discriminating and took longer time to answer than the text questions. They test higher cognition and have certain characteristics that effect difficulty level and how they are perceived by the examinees altering their thinking process towards the answer. Gap areas identified in the Cambridge framework and external factors that were contextual, may give room to explore the issue of international validity in assessment.
Collections
Publisher
Commissioning body
The following license files are associated with this item:
Related items
Showing items related by title, author, creator and subject.
-
Producing multimedia resources for teaching medical students about the social context of medicine: a partnership approach
Collett, Tracey Jane; Schoenborn, P; Campbell-Barr, V; Dalton., F; Huggins, V; Poverjuc, O; Wyness, L (University of Plymouth, 2012)The UK General Medical Council (GMC) requires medical school graduates to be able to apply social science principles to medical practice (1993, 2009). At the policy level it is recognised that understanding of the social ... -
Producing multimedia resources for teaching medical students about the social context of medicine: a partnership approach
Collett, T; Schoenborn, P (University of Plymouth, 2012)The UK General Medical Council (GMC) requires medical school graduates to be able to apply social science principles to medical practice (1993, 2009). At the policy level it is recognised that understanding of the social ... -
How will BT meet the challenge to provide multimedia services over its local loop?
Aloufi, Ahmed Salim (University of Plymouth, 2005)This thesis looks at some of the technologies that may feature the future networks. Some of them are very well known and been used very widely such as ADSL and CATV and the others can be classified as the future technologies ...