The Question on Higher Grades and Teaching Evaluations

In a recent open comment, Anonymous had some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. In my last post, I tried to make the case that I don’t always give out “higher grades,” although there is a tendency for higher grades to appear in some of my classes. This post addresses the second of several claims/questions/concerns.

Question: “Do you think your preoccupation about evaluations are one reason why you are willing to give out relatively higher grades?”

You know what people’s greatest fear is? Public speaking. So how would you like to teach a course? That’s what my graduate supervisor asked me one day. Now, it wasn’t a question. No, it was more of a prediction about the future: You’re going to teach a course. Gulp.

Back in my day, there weren’t any how-to seminars for graduate students to learn about teaching like they have now. The preferred method was to throw you in the deep end and walk away, leaving you to thrash around, coughing and sputtering, waving your arms frantically. This is, of course, terrifying. How are you supposed to improve? How do you know what you are doing wrong, or maybe even, doing right? The answer came a month after my first course ended: teaching evaluations.

The students I taught were very understanding, and gave me some really good constructive feedback on improving my teaching. Incredibly, the students who I had been trying to teach had ended up teaching me some valuable lessons of my own. (I know, this sounds like every Hollywood movie set in a classroom that’s ever been filmed. If anyone is interested in buying my script, please get in touch with my agent.)

So, obviously, reading evals is like eating chips--bet you can’t eat just one! Am I addicted? Am I so desperate to hear nice things about myself that I will pander to students by pumping up their grades?

Answers: 1) I dunno. 2) Geez, I sure hope not.

As it turns out, the AASUA’s Teaching and Learning Committee has been looking at the issue of the validity of teaching evaluations for the past couple of years. As it also turns out, I know this because I’m on this committee. So I have some actual (but general) answers--not just facetious ones.

Is there a statistical link between higher expected grades and evaluations of teaching? Yes. “Class-average grades are correlated with class-average student’s evaluations of teaching, but the interpretation depends on whether grades represent grading leniency, superior learning, or pre-existing differences” (Marsh & Roche, 1997, p. 1194). So what is the correlation? Unsurprisingly, there is a range, which usually goes from 0.10 up to 0.30; the “best estimate” is taken to be probably about 0.20 (Marsh & Roche, 1997). That’s a correlation, but it’s a pretty weak one. In terms of the overall variance in evaluation scores, grade expectations account for less than 10%. If an instructor decides to pump up his or her ratings by inflating grades (and risking his or her career), the payoff isn’t there--there are too many other things that influence the ratings.

Coming back to me, though (because it is all about me). Am I inflating grades to get good evaluations?

Answers: 1) I dunno. 2) Geez, I sure hope not.

I don’t want to know. I do a lot of statistics on the performance of my classes. (I’ll eventually post about how I use point-biserial correlations to analyze exam performance. Your eyes are guaranteed to glaze over! Woot!) I know, for instance, that student evaluations of me are positively correlated with their evaluations of the textbook (r = 0.498 last time I calculated). This is useful information: it’s really important to find a textbook students like. But, duh, I know that anyway. Why is this related to me, is it just spurious?

But I have not, do not, and will not calculate or correlate my evaluations with student performance. I’m not going to take any student rating of my teaching and compare it with the median, mean, or mode of the marks in any of my classes. Why? I don’t want to know. If I’m unconsciously, unknowingly giving out higher marks to get better evals, that’s one thing. But if I’m doing that willfully, consciously, that’s unethical--it’s just wrong.

It may look like I’m preoccupied with evaluations. I do mention them in class, and even put the evaluation date on the syllabus (we are supposed to tell students when evals are going to take place, ya know--I read the fine print). But telling you about how good my evals are, I think, makes my job harder: your expectations increase. In contrast, if I tell you that I really suck and then I kinda don’t, maybe you’ll be happy and give me a good rating.

So maybe it works against me. You think you’ve got this instructor who thinks he’s hot as snot, but turns out to be awful. So you burn me on the evaluations. I welcome that. As long as you tell me what I did wrong, I can still learn and improve. I can try harder next time. Maybe someday I can be as good as the instructors who inspired me to go into this psychology business in the first place, like my graduate supervisor.

I’ll address the remaining concerns expressed by Anonymous in my next post.

Why aren’t you studying?

References:
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187-1197.