I do a lot of exam validation and one of the questions I am frequently asked is: Do we need to validate ALL of our questions and exams? The answer is: It depends on what you are using the questions and exams for.
Based on many years of experience here are my best practice guidelines:
To begin with, there are five steps in a complete exam validation process:
- Validate the questions structurally.
- Align questions and learning objectives.
- Set a valid passing score.
- Perform an item analysis on the exam results.
- Revise and/or replace any questions, if necessary (if you’ve done steps 1 and 2 properly this should be rare).
I’ve written previously about how to correctly perform each of these steps, so I won’t repeat that guidance here. (If you don’t know how to do this or don’t feel comfortable doing it yourselves, hire an assessment consultant.)
But, the question I want to answer today is: Do we always need to do each of these steps? The answer is no. It depends on how the questions and exams are to be used. The chart below will help guide you on which steps are necessary, depending on the assessment purpose.
These are guidelines of course and not absolutes, except for the last row: You must validate high stakes assessments (assessments with job consequences) or you place your company in legal jeopardy, especially if you work in a regulated industry. For example, if you are a life science company (where I do a lot of exam validation) you must fully validate your PI exams.
intelalearning.wordpress.com 