ORCID
- Daniel Zahra: 0000-0001-6534-6916
Abstract
When setting standards for high-stakes assessments, it is necessary to be clear whether the primary criterion should be importance or difficulty. While standard setting practices frequently rely on the difficulty to determine passing or failing thresholds, this can undermine the intended inferences and uses of assessments. Standard setting based on difficulty conflates importance with subjective perceptions of difficulty, misguiding decisions about what candidates need to demonstrate for safe and effective practice. We explore several reasons why difficulty does not align with the aims of assessment and how prioritising importance can better reflect the intended uses and inferences of a test. Our analysis incorporates: (1) the relevance of validity to setting standards, (2) potential misunderstandings of norm- and criterion referencing, (3) critical differentiation between importance and difficulty, (4) reasons for prioritising validity, (5) the variability of candidates’ difficulty perceptions, (6) challenges of assessor judgments, (7) the need for clear definitions of competence, and (8) the appropriate use of ‘would’ vs. ‘should’ in establishing standards. Ultimately, a valid standard reflects the ability to demonstrate performance of what is important, not difficult, in the workplace.
DOI Link
Publication Date
2026-03-21
Publication Title
Education Sciences
Volume
16
Issue
3
Acceptance Date
2026-03-19
Deposit Date
2026-03-24
Funding
This research received no external funding.
Keywords
Assessment, standard setting, importance, difficulty, fairness
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
First Page
1
Last Page
11
Recommended Citation
Burr, S., Zahra, D., & Robinson, I. (2026) 'The Difficulty with Setting a Standard Based on Difficulty: The Role of Validity in Determining Assessment Standards.', Education Sciences, 16(3), pp. 1-11. Available at: 10.3390/educsci16030488
