Final Exam

The Goal

Near the end of last year, the science teachers decided to start using the SAT II subject tests as the final exams in high school science classes. The SAT II curricula are much more challenging than the MCAS curricula we use now; we wanted to challenge ourselves and our students to try to meet this higher standard. There are three main reasons for this:We knew that it would be hard to get our curricula up to the level of rigor the SAT II's demand, but we decided this was a challenge we had to take, because the SAT II's are the widely accepted measure of what colleges expect someone to have mastered who has taken a high school science course. If we can get you up to SAT II level in the sciences, that will mean that you are prepared to go on studying science in college.

The Problem

The problem, in essence, was that the SAT II in physics is so far beyond the level of what we are doing in your class that it isn't really valid as a final exam, as a way to evaluate how well you learned the material in this class. There are two big reasons that the SAT is so difficult:So, the basic situation is that out of 75 questions on the final, there were only about 45 that you might conceivably have been able to answer based on what you learned in this class, and most of those required a level of understanding of math that BLA simply will not provide you with. The result was that the average score, when taking into account the quarter point penalty for guessing, was something like a 10 out of 75.

Now, it might not be immediately obvious why that is a problem. I can just scale it, can't I?

Actually, I can't just scale it, because, whatever I do, the grades on the SAT will not be an accurate representation of how well you understand physics. The problem here is something that statisticians call signal-to-noise ratio. That sounds complex and mathematical, but actually it's something you deal with every time you try to hear what your friend is saying in a noisy room. If the amount of noise (in the case of the final, the questions you just guessed on) is significantly higher than the amount of solid data (the questions that you actually knew), then the result is not a very trustworthy measurement.

Let me put this another way. If you had taken several different SAT II finals, you would have gotten a wise range of scores. If you knew most of the material, the number of questions you guessed on would be small, and so you would tend to get scores clustered around the same value (56, 59, 55...) with the guessing not helping or hurting you much each time. But if you are guessing on almost every problem, there is a much wider range in your possible scores: you might get a 2, then a 15. Since one of those scores would scale to an F and the other to an A, there is an unacceptable amount of uncertainty in your grade. I, as a teacher, can't with any confidence give you a B+ or a D- or whatever on the final, because I just don't know, based on your grade on the SAT II, what grade you actually merit.

The graph below shows, on the x axis, your grades in this class: grades that I can have a high level of confidence in, because they are based on your performance on many different assessments over the course of a year. The y axis shows the grades you got on the final. I have drawn vertical lines up and down from each data point to show the uncertainty in each final grade: if that person took the final again, they could be expected to get a score anywhere in the range shown by those lines. You have seen something like this in your MCAS score reports.

If the level of uncertainty weren't enough to convince me that the scores were invalid, there is something else that would - the lack of a good correlation between the final exam grades and the course grades. You can see that there is a generally upward trend - good grades in my class tended to get you good grades on the SAT II - but the data points don't lie neatly on the best-fit line; they are scattered all over the place. The lowest grade on the final was someone who had a B+ in the class; someone else who had an F+ in the class scored higher than 90% of the class on the final. In statistical terms, we look at the correlation coefficient between the two, what you know as the r² of the fit; it says that 16% of your grade on the final was due to understanding physics, and the other 84% was due to guesswork. Clearly not a good way to measure your physics performance.

What Did I Do About It?

So, I've got these scores that have some relevance to how well you know physics, but not very much. I have to put something on your report card in the "Final Exam" slot. I could just make the final not count at all - I would do this by using your term 1-5 average as the final exam score. But I know that a lot of people studied hard for the final in the hopes that it would help your year average, and I would like your hard work to help you out somehow.

So, what I did instead is to just apply a ridiculously generous scale to the final. Specifically, I took your raw score on the final, and added 90 to it. That way, everyone gets an A on the final, guaranteeing that at least no one is hurt by it, and those who had low grades in the class are substantially helped by it. This scale meant that some people actually got grades higher than 100%, which may seem like a waste of effort - after all, you can't get a grade higher than an A+. But fear not - since I plugged in the actual grade in my gradebook, not capping it at 100%, the higher grade still helps your final class grade, even if it doesn't show up as an A+++ on the final exam.

Incidentally, this means that, if you really wanted to, you could use your final exam grade to find out what you would have gotten on the SAT II's, had you taken them. Just subtract 90 from your final exam grade to get your raw score, then look it up in this chart. Unfortunately, no one got anywhere near an average grade on the SAT's - 640 is the average for people taking the actual test, and no BLA student got higher than a 560 on the final - so I doubt any of you will be rushing out to take the SAT II physics.