In a recent open comment, Anonymous had some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. In my last post, I tried to make the case that I don’t always give out “higher grades,” although there is a tendency for higher grades to appear in some of my classes. This post addresses the second of several claims/questions/concerns.
Question: “Do you think your preoccupation about evaluations are one reason why you are willing to give out relatively higher grades?”
You know what people’s greatest fear is? Public speaking. So how would you like to teach a course? That’s what my graduate supervisor asked me one day. Now, it wasn’t a question. No, it was more of a prediction about the future: You’re going to teach a course. Gulp.
Back in my day, there weren’t any how-to seminars for graduate students to learn about teaching like they have now. The preferred method was to throw you in the deep end and walk away, leaving you to thrash around, coughing and sputtering, waving your arms frantically. This is, of course, terrifying. How are you supposed to improve? How do you know what you are doing wrong, or maybe even, doing right? The answer came a month after my first course ended: teaching evaluations.
The students I taught were very understanding, and gave me some really good constructive feedback on improving my teaching. Incredibly, the students who I had been trying to teach had ended up teaching me some valuable lessons of my own. (I know, this sounds like every Hollywood movie set in a classroom that’s ever been filmed. If anyone is interested in buying my script, please get in touch with my agent.)
So, obviously, reading evals is like eating chips--bet you can’t eat just one! Am I addicted? Am I so desperate to hear nice things about myself that I will pander to students by pumping up their grades?
Answers: 1) I dunno. 2) Geez, I sure hope not.
As it turns out, the AASUA’s Teaching and Learning Committee has been looking at the issue of the validity of teaching evaluations for the past couple of years. As it also turns out, I know this because I’m on this committee. So I have some actual (but general) answers--not just facetious ones.
Is there a statistical link between higher expected grades and evaluations of teaching? Yes. “Class-average grades are correlated with class-average student’s evaluations of teaching, but the interpretation depends on whether grades represent grading leniency, superior learning, or pre-existing differences” (Marsh & Roche, 1997, p. 1194). So what is the correlation? Unsurprisingly, there is a range, which usually goes from 0.10 up to 0.30; the “best estimate” is taken to be probably about 0.20 (Marsh & Roche, 1997). That’s a correlation, but it’s a pretty weak one. In terms of the overall variance in evaluation scores, grade expectations account for less than 10%. If an instructor decides to pump up his or her ratings by inflating grades (and risking his or her career), the payoff isn’t there--there are too many other things that influence the ratings.
Coming back to me, though (because it is all about me). Am I inflating grades to get good evaluations?
Answers: 1) I dunno. 2) Geez, I sure hope not.
I don’t want to know. I do a lot of statistics on the performance of my classes. (I’ll eventually post about how I use point-biserial correlations to analyze exam performance. Your eyes are guaranteed to glaze over! Woot!) I know, for instance, that student evaluations of me are positively correlated with their evaluations of the textbook (r = 0.498 last time I calculated). This is useful information: it’s really important to find a textbook students like. But, duh, I know that anyway. Why is this related to me, is it just spurious?
But I have not, do not, and will not calculate or correlate my evaluations with student performance. I’m not going to take any student rating of my teaching and compare it with the median, mean, or mode of the marks in any of my classes. Why? I don’t want to know. If I’m unconsciously, unknowingly giving out higher marks to get better evals, that’s one thing. But if I’m doing that willfully, consciously, that’s unethical--it’s just wrong.
It may look like I’m preoccupied with evaluations. I do mention them in class, and even put the evaluation date on the syllabus (we are supposed to tell students when evals are going to take place, ya know--I read the fine print). But telling you about how good my evals are, I think, makes my job harder: your expectations increase. In contrast, if I tell you that I really suck and then I kinda don’t, maybe you’ll be happy and give me a good rating.
So maybe it works against me. You think you’ve got this instructor who thinks he’s hot as snot, but turns out to be awful. So you burn me on the evaluations. I welcome that. As long as you tell me what I did wrong, I can still learn and improve. I can try harder next time. Maybe someday I can be as good as the instructors who inspired me to go into this psychology business in the first place, like my graduate supervisor.
I’ll address the remaining concerns expressed by Anonymous in my next post.
Why aren’t you studying?
References:
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187-1197.
The Question on Higher Grades and Teaching Evaluations
The Question on Higher Grades
In a recent open comment, Anonymous made some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. This post addresses the first of several claims/questions/concerns.
Claim: “...you also give relatively higher grades.”
So, relative to other instructors, I’m taking it? The University, specifically GFC, has approved grade distributions for different undergraduate courses. The fine print says this: “These distributions are provided for guidance in your grading. It is not necessary for the grades in a particular class to follow any of the distributions exactly.” (Unless an instructor is grading on the curve with the help of a spreadsheet, it’s impossible to get these exactly anyway. And I don’t grade on a curve.)
Instead, I focus on the expected medians for each course level:
1st year = B-If my classes don’t match these, well, I don’t know what happens. So far, nothing yet.
2nd year = B
3rd year = B
4th year = B+
Anyway, here are the actual medians for the last 12 courses I’ve taught:
1st year: B, BAny patterns? Am I consistently giving higher grades? It looks like the 100-level courses are a bit higher than expected. Why? Major components of that course (20% of the overall mark) consist of easy marks (Information Literacy, Research Participation) that boost students’ grades. These components are out of my hands; I don’t do any marking, I just accept the results as they are. So should I make my exams harder to compensate for these “free” marks? Of course not. Class means on my exams in that course are around 65%, and I don’t want them any lower than that.
2nd year: B+, B+, B, B, B-, B+, B+
3rd year: B
4th year: A-, B
Let’s skip to my 400-level course. Yup, I recently had a class earn a median of A-. They all deserved it. It was the best bunch of students I’ve ever had in that course, and I was really happy to give the marks I did. Their term papers were great, and their exams were outstanding. Didn’t even know the median was so high until it popped out of my spreadsheet when I was filling in the final grade forms. (And look, another 400-level class only got a median of B.)
It looks like there’s something funny going on in my 200-level courses. Yup, the grades are a bit high, tending to a median of B+, whereas GFC expects a B. That’s not a huge difference--in terms of the percent cutoffs I use, 72% is right in the middle of my “B”, whereas 76.5% is the middle of my “B+”. That’s a difference of 4.5%. Still, for a class of over 200 to have a grade that’s almost 5% “too high” is significant.
So, why the high marks? In my 200-level perception course, the textbook I used was extensively revised a few years ago, and the testbank of multiple choice questions that comes with it was really--how should I put it?--simplified. Because these questions make up about half of the exam, the marks went up by a few points. In my 200-level cognition course, the textbook I use is written by the same person who wrote my perception textbook. This textbook was also recently revised. Guess what the testbank is like?
It’s tough to rewrite dozens of exam questions, but I’m slowly working on it. Realizing that the marks have been increasing, I’ve also been slowly changing the percentage cutoffs for each letter grade. I don’t want to make huge changes all in one term--that’s not fair to those students. But it’s also not fair to give them inflated grades compared to other terms.
I’d like to think that my teaching improves over time--but is this reflected in students’ grades? If that were universally the case, wouldn’t instructors near retirement have sky-high marks in their classes, and wouldn’t graduate students teaching their first class have rock-bottom marks? Hmm, unless those sneaky novice instructors are inflating their students’ marks.
But that’s the topic of my next post.
Why aren’t you studying?
The Spacing Effect
There is a finding in the scientific literature that's been known for over 100 years. Recent research has continued to support its existence, and it turns out to be one of the most robust effects ever discovered about memory and learning. The problem is, few people know about it. It's called: The Spacing Effect.
This effect was first described by Hermann Ebbinghaus, who discovered it while doing a series of experiments on his own memory. In short, he found that, although repetition helped him remember things, spacing the repetitions over time led to a big improvement in his ability to remember.
When you're studying, the spacing effects says that it's counterproductive to repeat what you've studied immediately after you've studied it once. Instead, you should actually let some time go by before you refresh your memories. How much time? That depends--when do you need to know the information? Here's the deal, taken from a study published in 2008:
When the gap between initial learning and test date was a week, the optimum review took place a day after initial learning... With a month gap, the ideal review occurred after about a week; with a year, the prime review came three weeks after learning.So, midterms are roughly a month apart, right? That means you should be reviewing and repeating the material about every week.
The research shows that not only is spaced repetition a benefit to your remembering, but also that cramming is bad. Bad, bad, bad. Really bad. Did I mention it's bad? Optimally spaced repetition beat cramming by 77% to 111%. Note that a 100% improvement would mean a doubling of performance. Yeah, it's that good.
The bottom line: everyone should space out.
(For more, read the article, "Will that be on the test?" from the APS Observer.)
Why aren't you studying?
The High Cost of Textbooks
I've recently been exploring the cost of the textbook for one of my courses. By itself, it goes for $152.35 at the bookstore. However, I require students to do online labs, so they have to buy an access code. Previously, bundling the code with the (new) textbook added exactly $2.95 to the cost. The cost of the textbook + code bundle this term? A whopping $187.40. This means the code adds $35.05 to the price of the book. Yikes!
Just wait--it gets worse. If you buy this code online, it will cost you $33.26. Whuh? The reason why I've gone with the bundle is that it has saved students money--not cost them more. What's the advantage of bundling?
And it's even worse. According to the bookstore, the bundle increased in price since last semester by $17.75. Whuh? Worse, compared to a year ago, it now costs $41.19 more. Double-whuh? This is for the same textbook, mind you; not a new edition or anything.
Back in 2005, an error was made (either by the bookstore or by the publisher) and the price of the textbook + code bundle in another course of mine was too high. In that case, students were able to get a refund of the difference, as long as they still had their receipt from the bookstore. That time, the error was caught pretty early on, when the term was less than one month over. Now I don't know if students will be getting a refund this time, but even if that is the case, who hangs on to their receipts for everything for months and months? (And how come no mistake ever happens where the cost of something is accidentally too low?)
It may come as a surprise, but instructors don't know the price of textbooks beforehand. I kind of have the implicit assumption that prices will be stable--at least from one term to another within the same academic year. I see now that's not the case, which is pretty sneaky.
Asking students to spend almost $190 is just too much. I hate to dump a (very good) textbook just because of its price. I would also hate to get rid of a highly regarded and useful online lab because of price. But too much is too much.
What price do you think is reasonable for a textbook? (No, $0 is not considered reasonable...)
Why aren't you studying?
- - - - - - - - - - - -
Update: The publishing company rep got back to me with this on 3/20/2009:Last year we had to bring our prices more in line with their prices to prevent cross border sales of lower priced product into their market (it is illegal but happens nonetheless). That is largely why the prices went up last year. We are under an agreement to maintain a certain level of compliance with this. We realize the price for the 7th edition is high but it isn’t a mistake.
In a wacky twist, the new edition of the textbook (the 8th edition) is going to be cheaper than the 7th edition.
- - - - - - - - - - - -
Update on 9/3/2009: The new edition of the textbook is in--but the news is not good. The publisher told me the net price of the package is $121; the list price was supposed to be $151.95. The list price is the publisher's suggested retail price. So why is the package selling for $166.95 in the bookstore? According to the bookstore manager, that price reflects the bookstore's actual markup, which is 5% above list price. Argh!
The Screwup
Hi, and welcome to screwup week! Well, that's what it seems like.
First, I was late to a class this week. That's bad enough. But now I've discovered I've screwed something else up: the marking of some labs.
My teaching assistant and an assistant marker accurately followed my marking guide for a recent lab--but it was, er, wrong. So students were penalized for giving the completely correct answer. (I suppose students with the completely wrong answer received a mark that they didn't deserve, although looking at the marks, the average for this lab was almost one full point lower than usual. Duh!)
Worse, it took a group of very sharp students to notice this error and let me know about it. (To be fair, at least my marker noticed the problem, but forgot to tell me.) But wait--it gets worse.
I didn't write this particular question; it (and the standard answer in the marking guide) were provided to me by the creator of the online lab system. I accepted the answer without question, without thinking. I blab on and on about the importance of critical thinking--and then I go and make a screwup like this. Argh.
As penance, I re-marked the few dozen labs that hadn't been picked up by students yet. But I asked my marker to do the rest. (I'm not lazy, I'm up to my ears in marking essay exams which take first priority..and take a lot of time.)
So, what have I learned from all this? 1) Students are smarter than a lot of people give them credit for. I think they've learned the lessons of critical thinking better than I have...which brings me to: 2) I'm not as smart as I like to think I am--but I'm learning...
Why aren't you studying?
The Lateness
I was late for class yesterday. I was late by 15 minutes, and I apologize to everyone in that class.
I could come up with a whole bunch of excuses. I only left an hour before my class was due to start, and it took me three times longer than normal to get to campus. I should have left even earlier. The weather was pretty bad, and the roads were in poor shape. There was also a collision on the way (not in the direction I was going--but everyone else still had to slow down and rubberneck to get a good look at the crash). Surprisingly, the baby was not a factor in my being late--she was still sleeping when I left home.
None of this makes any difference. I was late by 15 minutes, wasting the time of about 200 students. Multiply 200 people by 0.25 hours, and you get 50 people-hours of wasted time. Yikes. Who am I to be wasting so much time--the government?
I try my best to get to all of my classes on time. I do leave home with enough buffer time to (usually) prevent being late. When I'm in my office, I actually have a little clock program that sounds an alarm when it's time for my next class. I try to be very time-aware, but sometimes, well...I'm late.
I'm going to try extra hard for the rest of the term to not be late to that class again. And I'm still going to try not to be late to anything else--classes, meetings, appointments...oops! Look at the time. Better get going; don't want to be late.
Why aren't you studying?
The Office Hour
I love my office hour--I get a lot of work done. Sometimes, I read a magazine or scientific journal. Oh, and sometimes I see a student.
Years ago, I used to have a lot of office hours--more than 10 a week. That number has dwindled over the years. Now, I have only one office hour this term. That's partly due to the fact that I have to help my wife look after our kids; she's got her hands full with the baby--not to mention our other daughter who is feeling sadly neglected. But a big reason is that there's not much point to having a lot of office hours.
Even with the single solitary hour I've set aside, I hardly see anyone. Oh sure, I get a lot of emails. But not many people actually trek through all the construction to get to my palatial office.
So occasionally, I agree to meet with someone outside of my regularly scheduled office hours. This is not easy to do, with my other commitments like committee meetings. When my wife goes back to work after her maternity leave, we'll "trade off." This means that as soon as I'm done work, I'll race home to look after the kids so that she can go to work (and vice-versa). In a situation like this, it's really hard to set up a meeting with a student outside of office hours: it requires that I ask my wife to change her work schedule, which is not easy to do and requires several days advance notice.
We did the "trade-off" thing with our first child, and I tried hard to meet with students. And this is what happened, more often than not. My wife changed her work schedule so that I could meet with a student...and the student never showed up. No message, no apology, no nothing.
So if I'm not available much outside of office hours, the reason is that previous students have wrecked it. Oh well, there's always email.
Why aren't you studying?
The Cold
The good news is that the baby finally had a good night over the weekend, sleeping for 5 hours in a row (!) at night (!!). The bad news is that I'm sick with a cold, so I wasn't able to sleep.
Instead of snuggling in my nice warm bed and trying to take a nap and eat chicken soup, I dragged my sorry self to work today. The Public Health Agency of Canada recommends this: "If you're sick, stay home!" What about the Centers for Disease Prevention and Control? "Stay home when you are sick..." And even the U of A says, "Stay home when you are sick."
Yeah, right.
If I had some kind of substitute or something, suuure I'd stay home. But I don't, and midterms are coming up. What, should I delay the midterm? How will I fit in all my lectures in the classes that remain? Should I just skip that lecture content? But I've already made up the exams and had them printed.
What do you think? Should a sick instructor still come in to teach? Do you appreciate the effort that's being made? Or would you prefer that the sicko stay home so that you don't get what's going around, too? (Yes, I figure you'd prefer a day off from my long-winded, boring, blah-blah-blah. But try to put that aside for a moment.)
Now I'm going to drag my sorry self back home and change some diapers...
Apologies in advance if I cough into the microphone in class.
Why aren't you studying?
The Better Grades
Flipped open the most recent issue of Perspectives on Psychological Science today. The first study grabbed my attention: it's about factors related to students getting higher grades. Does it have your attention now, too?
This study was a meta-analysis, which is an analysis of previously published studies. This means it's probably going to give a pretty accurate big picture on academic performance. It's obvious that a good predictor of how you're going to do in a given course is how well you've done in previous courses. Unfortunately, but it's a bit of a circular argument. (Why did you perform at the level you did in the previous course? Because of how you did in the previous course, etc.) So this study helps break out of that circle.
This study found three factors that are really important contributing factors. First, study motivation was found to be a reliable predictor of GPA. Translation: the more enthused you are about a course, the better you're likely to do. OK, so how do you do that? Choose your courses carefully, and take courses that you want to take, as much as possible. If you're in law school and hating every second of it, how much fun is it going to be when you're a lawyer? (Granted, there are also external or "extrinsic" motivators. Getting filthy rich is an extrinsic motivator for some people. Hmph.)
Study skills include your time management skills, your ability to handle demands, and your capability to organize, summarize, and integrate material. If all you do is memorize stuff, your study skills could use an upgrade. Can you see the big-picture, or patterns in the things you're trying to learn? Study skills also had strong relationships with GPA and individual class grades. (Didja know you can get help with this?)
Next, study habits were also found to be predictive of performance. This includes having discipline in how you study: how often, whether your review, and even your choice of an environment that's conducive to studying (i.e., iPod turned off, no TV or computer, no other distractions).
Lastly, I wanted to mention that having study anxiety was a negative predictor of performance; that is, if studying or writing an exam freaks you out, you're less likely to do well. (Didja know you can get help with this, too?)
Here's the reference for this study:
Crede, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3, 425-453.
Why aren't you studying?
The Lecture Notes
I post my (incomplete) lecture notes online. (Yes, incomplete. Why? More on that in a future post.) I post the notes in HTML format, the "language of the web." I could post them in other formats, but I choose not to.
PDF files are really inflexible. You can look at them, and print them out...and that's about all. What if you want to make the typeface bigger or smaller? It's almost impossible. So you can't edit them or (easily) load them into other software. Why would anyone want to edit them? Quite a few students type in the fill-in words, which are so inconveniently missing in the lecture notes.
OK, so how about Word files? They are totally flexible, and you can even edit them with Google Docs or the free Open Office. I've considered this, but it would be a lot of work to convert what I've got to DOC files.
The ultimate has to be: PowerPoint. If I were to post my PPT slides, all students could have the exact duplicate of what I show on the big screen in class. That's one reason why I don't post them. If I did, attendance in class would drop. Why bother coming to class, right? Maybe just the most motivated students would come. Look, we're all adults here: you don't have to come to class...ever. But you don't have to pass, either. And isn't it my job to help you learn (and pass the course)?
I like to have neat little demos and things in class. If everyone had those beforehand, it would spoil the surprise. I could edit out interactive components before I post the slides, but that would mean a lot of extra work. (I'm all about not doing extra work. I like doing other things, like eating, sleeping, and holding the baby until she screams in my ear for her mama.) Printing out PowerPoint slides is cumbersome, too.
I've used HTML format because it's super easy to adjust the size of the typeface. (Don't know how? Just ask me.) You can also load a page of lecture notes into Word (simply paste the URL of the lecture you want into the open files dialog box and you're good to go). When you print them out, there's a lot of room in the margins for you to take extra notes, make annotations, and other things that help you to learn.
Putting aside the fact that the notes are incomplete, what format do you prefer? Do you want to download a bunch of 50 MB PowerPoint slides? Is Word your favourite? Or are they okay as is?
Why aren't you studying?
The Open Comments: 1
This is an open comments post. So I'm not going to say anything, but let you say whatever you want in the comments below. It can be about this blog, about your in-class experience, questions for me, whatever.
I know I should do this more often--I just forgot! (No comments/suggestions/feedback? You can tell me if you want an open comments posting once a month, once a week...or never.)
The Importance of Sleep
OK, so this is on my mind all the time: sleep. I want it. I need it. The more the better. Cuz babies are sleep-stealers, you know. Waa, waa, waa, all night. Then they close their li'l eyes and drift off to sleep the moment your alarm goes off and you have to go to work and they end up sleeping all day. *sigh*
If you've gone through intro psych, you know a bit about sleep: circadian rhythms, stages of sleep, that kind of stuff. I hope your prof also talked about the psychological effects of sleep as well. Research since the mid-1990s has shown the critical importance of sleep to learning and remembering.
One study looked at neural activity as rats learned to go through a maze. Based on the pattern of neural activity alone, the researchers could tell where the rats were in the actual maze, tracing their route from start to finish. The electrodes were still in the rats as they slept, and the researchers continued to examine neural activity. The pattern of brain activity in sleeping rats was the same as when they were actually running through the maze. The brain seems to be replaying experiences at night when you sleep.
Studies on people show improvements in all kinds of tasks with sleep, including visual and language tasks, logic puzzles, and generally improving memories. When cut back on sleep (e.g., by staying up late or pulling an all-nighter), you're not helping your brain do it's best to form, consolidate, and strengthen the things you're trying to learn. If you're a sleepless, tired student trying to pack as much as you can into your cranium, you're missing out on a great, free, useful tool to getting better grades: blissful sleep.
Want to read more? Scientific American Mind has a great article (I love the title): Sleep on It: How Snoozing Makes You Smarter.
Why aren't you studying? (Or sleeping?)
The New Beginning
Sometimes things come together in strange ways. Of course, it's a new year (happy new year, by the way). So you put up your new Star Wars calendar. Ah, that looks good.
Of course, it's a new semester (happy, er, new semester, by the way). Ah, crap. Life changes completely every 4 months. Just when I adjusted to last semester, too.
Of course, I've got a new baby. Yeah, yeah--my wife did. But I helped. That's an awesome new beginning.
And, of course, I've got a birthday coming up. So that's another new beginning--another trip around the sun. I don't have a problem being with that. But having a baby makes me think about what the world was like when I was born: no one had walked on the moon, the Beatles were still together, and "dude" meant an ill-bred, ignorant, and ostentatious man from the city.
I found an interesting collection of celebrities who share my birth year:
- Cuba Gooding Jr
- LL Cool J
- Edward Burns
- Molly Ringwald
- Daniel Craig
- Kenny Chesney
- Celine Dion
- Patricia Arquette
- Tony Hawk
- Kylie Minogue
- Gillian Anderson
- Rachel Ray
- Will Smith
- Hugh Jackman
- Owen Wilson
- Jill Hennessy
- Lucy Liu
These were born on same day, but in a different year:
- Matt Frewer
- Michael Stipe.
UPDATE: I changed the title of this post to reflect the fact that it's a beginning, not the beginning, if that makes sense.
Find It
