It’s Now More Important Than Ever To Validate Assessments

Now that all learning has moved online (traditional eLearning, microlearning. virtual coaching, virtual classrooms, etc.) it’s important to look at the flip side of the learning coin: Assessments. As we’ve discussed many times in this blog there are two types of assessments:

  • Assessments for learning — sometimes called formative assessments
  • Assessments of learning – sometimes called summative assessments

We have written about both, though as a microlearning vendor we have more often written about assessments for learning. To be truthful, some form of assessment for learning, either in the form of adaptive questioning or gamification, is a core feature of almost all microlearning systems, not just ours.

So, today we will turn our attention to assessments of learning. Why now? Because when all learning is virtual, we need to be able to:

  • Evaluate our learners’ progress during their online courses
  • Evaluate their knowledge at the completion of their courses
  • Evaluate their knowledge retention at a delay, some time after the courses are completed

There are three keys to successful summative assessments. Assessments must be:

  • Fair
  • Reliable
  • Valid

Fair means that the assessment appears fair to the learner. It doesn’t mean that the assessment is easy, just fair.

Reliable means that the assessments consistently give the same results over time, across forms and within the exam itself. There are standard statistics that will tell you this.

There are several forms of validity but for our purposes we are most concerned with content validity: Are the questions properly structured? Do the questions properly cover the learning objectives? Is this knowledge that our employee should reasonably know? Have we set a defensible passing score? (Too often I see companies using an arbitrary passing score that would not stand up to legal scrutiny.)

And finally, are you using an assessment system that provides you with item level data that allows you to measure exam reliability, validate questions and revise exams as necessary?

Remember, validity is not a number, it is a process. Creating valid exams requires both a robust modern assessment system and expertise from a qualified psychometrician.

2 thoughts on “It’s Now More Important Than Ever To Validate Assessments

  1. Steven – can you expand on an example of a passing score situation that would not hold up to legal scrutiny?


    1. I have seen three non-valid methods for setting a passing score:

      1. The Committee Method: “What do you think passing should be? I don’t know 90 seems about right. Sounds good to me.”
      2. The Higher Authority Method: “Our Vice President said it has to be 90.”
      3. The Received Wisdom Method: ” I don’t know, it was 90 when I got here, so I guess it’s 90.”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s