Pointless Quizzes

This morning I completed the University's on-line Diversity Training. In principle a good idea, as it raises awareness about disadvantaging students (or members of staff) where you didn't think you were, but in practice just another thing to do during an already full schedule. And much of it was not relevant for me anyway, as I am not in a position to determine the level of pay of my fellow members of staff, either male or female.

What struck me, however, was a particularly bad quiz. I've been thinking about this in the context of the Stanford AI-Class (on which I will post soon - it'll be finished in two weeks' time), and here it came up again: there were two short on-line multiple-choice quizzes embedded in the course.

The first one was so simple that I could just guess the right answers without having needed to read the previous text. If they are so glaringly obvious that anybody can get them right just with a bit of common sense then it does not really contribute to a good view on the course as a whole - it just comes across as—literally!—a box-ticking exercise.

But the second quiz was even worse, and not only because I got some of the answers wrong. There were various scenarios given, and the four choices you had to choose from were: was this case a) victimisation b) direct discrimination c) indirect discrimination or d) nothing illegal.

Why on Earth do I need to know the difference between 'direct' and 'indirect' discrimination? As far as I am concerned, I need to know what is legal, and what is not, in other words there are only two relevant categories for me: 'discrimination' or 'no discrimination'. So I got several answers wrong because I chose 'direct' when the answer was clearly 'indirect' or the other way round. This was just plain annoying. I can see that I need this when wanting to work in the legal field of employment tribunals, but as a simple bod delivering seminars and lectures about language to students I couldn't care less—as long as I know that I'm not doing anything that would count as illegal discrimination.

But this seems to be a frequent pattern with many on-line tests. Because the people creating such tests have not thought about them properly they come up with spurious details that they ask for, just so that they can have four possible choices, when ideally you should start at the learning outcomes—what do the people taking the quiz need to have learned, and how can we test this?

It is perfectly possible to do really good and useful on-line multiple-choice quizzes, but it requires work and thinking about the purpose of it. Otherwise it just annoys people and makes them want to do things to you that I could not possibly mention on this blog.

More on this topic in a few weeks when I will be discussing my experiences with the on-line AI course...