Crossing the Rubric Con

Following up on the idea of clear criteria of success, let’s take a moment and look at the now very common idea of using a scoring rubric for assessment of student writing. “Rubric” is just fancy edu-speak for a scoring guide. The word means “red” in Latin, and used to refer to the big, fancy red-letters that monks wrote at the top of a page of text.

Book of Margery Kempe, British Library.

But educational rubrics are not mere ornaments on hand-written pages. The rubric is a series of headings and descriptors that are designed to meaningfully guide both student work and teacher assessment of that work. (Book of Margery Kempe, British Library.) The pretty letter A acts as an anchor to the whole page. And to run off on a tangent, this picture comes from what may be the first autobiography  in English, by Margery Kempe, a 15th century mystic. Here’s a sample rubric from the Utah Education Network, providing guidance for assessing student problem solving in elementary school mathematics.

Distinguished – 4 Proficient – 3 Apprentice – 2 Novice – 1
Understands the Problem Identifies special factors that influences the approach before starting the problem Understands the problem Understands enough to solve part of the problem or to get part of the solution Doesn’t understand enough to get started or make progress
Uses Information Appropriately Explains why certain information is essential to the solution Uses all appropriate information correctly Uses some appropriate information correctly Uses inappropriate information
Applies Appropriate Procedures Explains why procedures are appropriate for the problem Applies completely appropriate procedures Applies some appropriate procedures Applies inappropriate procedures
Uses Representations Uses a representation that is unusual in its mathematical precision Uses a representation that clearly depicts the problem Uses a representation that gives some important information about the problem Uses a representation that gives little or no significant information about the problem
Answers the Problem Correct solution of problem and made a general rule about the solution or extended the solution to a more complicated solution Correct solution Copying error, computational error, partial answer for problem with multiple answers, no answer statement, answer labeled incorrectly No answer or wrong answer based upon an inappropriate plan

Before using the rubric, let’s see what it has to recommend itself. The top row provides score values ranging from “Distinguished” to “Novice”. I don’t think I’d use those names to describe my students on the basis of their problem-solving; it’s not at all clear what good comes from those words. But, some may disagree, and that’s fine. The main thing is that every aspect of the problem is graded on a scale of 1-4, with 1 being the lowest and 4 being the highest. Curious that there’s no zero, but that’s not a big deal. The left column provides descriptions for what is being measured using the 4-point scale. Students are assessed on the degree to which they

  • Understand the problem
  • Use Information appropriately
  • Apply appropriate procedures
  • Use representations
  • Answer the problem.

On the positive side, we see that the great bulk of the marks awarded are for process, not for finding the solution. A student could correctly answer the problem and still receive a low score for failure to indicate understanding, or to effectively communicate mathematics. Note also that a student could get the wrong answer and still get a high mark. Does all of this line up with your intuitions? Another strong feature of this rubric is that students should see precisely how their grade is assigned. They should be able to compare their work to the subscores and know exactly what is right and what is wrong with their work. Well, maybe not exactly, but they’ll certainly have a better idea of their work from this than they would from just seeing the final number written on the margin of their papers. Perhaps the strongest feature of this approach is that the rubric can (and should) be given to the students in advance of the test or assignment, and it can be used throughout the duration of the course. In this way, students have a clear idea of what their target is, and they have a clear set of expectations for their work. Every time. This is very good. Notice, though, that this particular rubric scores all 5 areas equally. Are they equally important to problem solving? Why are representations so important? More concerning are the descriptions. What’s up with “uses a representation that is unusual in its mathematical precision”? I’m not even sure I know what is being rewarded here. The required precision should either be implicit in the problem and procedures, or be stated outright. This is a very odd descriptor. To get full marks the student has to “explain why procedures are appropriate for the problem”. Seriously? You have to explain why addition applies? Or a geometrical construction? Sometimes, sure. But it’s easy to think of dozens of problems where the appropriateness of the procedures is obvious to anyone who can make progress. But these are details: some rubrics will better than others, and teachers need not use each other’s rubrics. The biggest challenge to effective use of rubrics is their insistence on a “one size fits all” methodology. Not all problems have the same criteria for success. Neither are all solutions comparable on the same rubric. For many years I marked high-stakes diploma examinations for Alberta grade 12 students. The rubric did a terrific job of guiding markers to reward similar work similarly, and of separating successful from unsuccessful attempts at solution to problems. For the high-stakes exam, it is crucial that marking be fair, and a team of people working off the same rubric is a good way to ensure that if two students give similar solutions they get the same grade. But invariably there were two types of solutions that crept in. Sometimes work would be clearly clueless, but somehow manage to hit enough key points to get a pretty good grade. Other times, a clearly superior student would make a fundamental error that threw the entire rubric off course and had to be given a much lower grade than your teacher intuitions would support. In both cases, you have no choice but to hold your nose and award a grade that you do not believe the student deserves. It’s an awful feeling. In many cases, especially when assessing student writing, the rubric is only part of the story. Substantive feedback on the details of the student work should supplement the grading tool. Yes the mark is important, and yes, it is desirable that marking be consistent. But student work is about more than just getting check marks for doing the assigned task. Every time a student creates a response, it provides the means for the dialogue that shapes growth and future achievement. The rubric doesn’t push very far into the dialogue. On balance, I support rubrics in spite of their obvious shortcomings. The biggest benefit is in their instructional potential: the rubric should be used to tell the student in advance what the grading scheme will be, and it should be used for targeted personal feedback to help the student to improve. Perhaps the best that can be said about the use of rubrics is a paraphrase of a political quip from Pierre Trudeau: compare them not to the almighty, but to the alternative.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s