Bad Idea #1—Emphasizing test format

In this era of high-stakes testing there are mounting pressures—some real, many imagined—on teachers to ensure that their students do well on high-stakes tests. Here in Alberta, Grade 12 students write Provincial Diploma Examinations that are worth 50% of their final grade in exit-level academic subjects. Success in Diploma Examinations leads to acceptance in the better post-secondary institutions. It matters quite a lot to students.

Machine scorable multiple choice

Pressure on the school comes from the fact that aggregate scores on Diploma Exams are publicly available. Notably, the Fraser Institute puts together a methodologically disastrous ranking of high schools based on these aggregate results; worse yet people pay attention to these rankings. Teachers put a lot of pressure on themselves to help students maximize their exam results. One of the ways that teachers attempt to do this is by making every in-class examination a structural copy of the high-stakes exam that the student will ultimately face. Often that means using a list of multiple choice items, perhaps followed by one or two numeric-response items, and then one or two long answer items. The idea is that this is what the “big game” looks like, so students should be as comfortable as possible with the structure, format and content of similar tests to avoid bad surprises.

So far, who could object? Presumably the exams are valid measures of student learning, so working to maximize exam scores should simultaneously maximize student learning. Right? I can immediately think of three strong responses to this line of reasoning.

First, we can acknowledge the validity of the external exams while still noting their inadequacies. Exams of any format will lend themselves to measuring some aspects of student performance, while missing others. For example, if you want your students to have strong laboratory practices, the paper-and-pencil exam simply cannot measure this well. Further, if you want student to demonstrate diligence in solving complex problems, the fixed-format exam described above cannot measure this at all.

Second, the whole idea commits the fallacy of conflating ends and means. If the goal is to write a particularly formatted exam well, it does not follow that writing other exams of that format is the best way to train. You don’t train for marathons by running marathons. You work on various aspects of your running, and piece them together as you build toward a marathon. If you want a child to learn, say, chemistry, then get involved in understanding what is involved, what the student knows know, and develop a strategy to get from here to there. You don’t just assume that writing appropriately formatted examinations will somehow enhance the student’s ultimate success on the high-stakes exam.

Third, and this is my main idea today, you deny the student of meaningful feedback as she/he learns the subject matter. As I have argued in previous posts, the most powerful tool in education is meaningful feedback from student to teacher and from teacher to student. Multiple choice items are opaque to thinking. You know that the student got the item right or wrong, but you have no idea what thoughts or procedures went into the response. What’s worse is that the students themselves are unlikely to remember when they get their exam papers back. Did they make an error and get lucky? Was it a guess? What went wrong? The same goes for numerical response items. The student writes a number in a box, but that’s about all the communication that goes on. Valid for large-population tests? Sure. Meaningful for an individual student? Nope. The long-answer questions should provide better feedback than the others.

What is my point? Assessment should not only provide a number for the calculation of a grade, it should also provide meaningful data for teacher and student to take full account of the learning that has taken place. Further, it should provide meaningful data for student and teacher to plan for what still needs to be learned.

Let’s be honest. Nobody wants students to be baffled by the test format, or made uncomfortable, or nervous. But how many times do they need to see the format not to be confused. Once or twice, maybe? So why do so many teachers use a demonstrably inferior format for so many more occasions than is necessary?

Don’t cave to the pressure, man. Remember what your task is. Find out what the student knows, and help make a plan to go from there to the outcomes of the approved curriculum. Keep the goal in sight.



2 thoughts on “Bad Idea #1—Emphasizing test format

  1. Hard to stay focused and honest in the face of crappy high-stakes tests. Thanks for keeping us honest, John! But I think maybe you’re underestimating how may times a test format needs to be seen and understood before the format becomes transparent.

    Liked by 1 person

    • Good point. This is part of the art. It is incumbent on the teacher to pay attention to student comfort with format, and work it until the test structure really is transparent. And then drop it and move toward richer communication of understanding.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s