The Marking

Just a short post to interrupt my marking. (Hmm, procrastination--another good topic a future blog post...)

Marking is not the most enjoyable thing. This fact has nothing to do with the quality of the answers. It's just awfully repetitive. I get to read answers to the same essay question over and over and over... This is by choice, mind you. When it comes to written/short-answer/long-answer I try to mark as many of the same question as I can, so that I can be as consistent in marking as possible.

So I get a fresh cup of coffee (hmm, coffee--another good topic for a future blog post...), put on some quiet music and concentrate on marking. This is difficult, as interruptions and distractions are many. Luckily, it's just me and the baby this afternoon. Awesomely, I've discovered the perfect music solution for both of us: Rockabye Baby!

These are baby lullabies of rawk songs, and improbably include songs from:

• The Beatles
• U2
• Bob Marley
• Coldplay
• No Doubt
• Metallica
  
• Led Zeppelin
• Radiohead
• Green Day
• Nine Inch Nails
• AC/DC
• and more.

You've never lived (as a parent) until you've heard American Idiot, Enter Sandman, or Highway to Hell played as a gentle, soothing lullaby. Insidious, bizarre, and just about the only way to simultaneously satisfy both a cranky 3-month-old and a cranky psychology instructor. (Country music fans, don't feel left out: check out Hushabye Baby.)

Why aren't you studying?

The Links

Now that lectures are over, it's time for learning to: continue. By now, being a student of psychology, you've seen just how broad this field is. Well, I'm a student of psychology, too. (OK, so I'm not officially a "student," but I love continuing to learn about psychology--which, really, is learning about ourselves.) I try to stay up-to-date on interesting and important research. A good way to get a general view of things is to follow a number of blogs.

Here are some of the blogs I read for education, for interest, and for entertainment. You might even see some of the things that are written about in these blogs appear in lectures.

• Mind Hacks. Although this Mind Hacks is a really good book on psychology, the blog goes way beyond that. It covers topics from art and aesthetics to hard-core neuroscience and bogus science. Vaughn is the main poster; I don't know how he keeps up the pace. Consistently excellent.

• Scientific American Mind Matters. Postings often include full-text articles from Scientific American Mind magazine. Eclectic and interesting, but a bit too much brain-focused. Still, there are many high-quality postings.

• Discover Mind & Brain. A companion to the print magazine Discover. Not quite as good overall as the preceding two blogs. Postings do not occur as frequency either. Also very brain-centric, which probably helps to attract readers.
  

• jnd.org. Usability "guru" Don Norman writes articles about design, science, and human factors/ergonomics. You can read his monthly column for Interactions magazine here, but there are lots of other neat things. One post made me run out and get a new stapler. Woot!

You can subscribe to all of the above blogs via RSS.

Why aren't you studying?

The Professor of the Month

The Undergraduate Psychology Association has very kindly featured me on their website as "Professor of the Month" for March. (Yeah, so there's only one more day of March. That's OK; the UPA is a volunteer organization--what have you volunteered for lately?) It's not an award or anything; it's an in-depth all-you-ever-wanted-to-know interview. Or at least as much as I could spew out in 18 minutes. (Thanks to Dan L. for interviewing me--and sorry I was in such a rush!)

I'm honoured that they'd want to interview me, even though I'm not technically a professor. Plus, I'm sharing billing this month with Dr. Peter Lee, whose answers to the interview questions are way better than mine. (Wow, one of his hobbies is brewing his own beer! Well, I make jam from the cherries that grow on the tree in my backyard...and I bake pretzels from scratch. Still not as cool as brewing beer, I know.)

This post is not about me promoting myself. Really, I'd like to plug the UPA. If you are serious about a future in psychology--which usually means getting an honours degree and/or graduate school--you should look into becoming a UPA member. Every year, they put on some really good forums on things like graduate studies, careers and volunteering, individual studies, and more.

Their website is full of great information, with a page of really useful info on graduate studies, plus professor and course of the month. The latter is great way to learn more about courses you might be interested in, and about some of your instructors. (And I can learn some secrets about my colleagues. Brewing beer? Hmph. I get my beer the same way my granddad did: I, er, buy it.)

Coming back to me, since this blog is all about me (what, isn't it?), I have to point out that a few years ago, my PSYCO 403 (now PSYCO 494) Human Factors & Ergonomics course was a Course of the Month. (Want to enrol? Sorry, for this fall, Bear Tracks says it's full. Already.)

Why aren't you studying?

The Textbook Change

I don't like changing textbooks. It's a huge hassle for a number of reasons. I like my lecture notes to be organized around the textbook I'm using. (Not to say that I repeat what's in the textbook, but minimally, I try to lecture on things in the same order as they are presented in the book.) So changing books means rearranging all my lectures. Plus, I'll have to rewrite the exams to reflect the material presented in the new text and take out questions based on the old one.

I'm also sensitive to students' concerns. If I change to a new book, students taking the class will not be able to find used textbooks to buy. Likewise, students who have taken the class will not find a market for their unwanted books. And, serving on the AASUA Teaching and Learning Committee, I've talked to Students' Union VPs about the issue of expensive textbooks--so I am trying to Be Booksmart!

But here's the problem: I want to use the best possible textbooks in my courses. Part of the usefulness of a book is its recency. But even a brand-new textbook is: obsolete. Why? After a research study has been completed, it takes up to a year for it to see print in a scientific journal. Next, it takes a few months--or even a year--for an author to write a textbook. By the time the book gets published, a few more months have passed. This means that research being done and theories being developed today are years ahead of the information contained on dead trees. And if I choose to keep using an old textbook, students can be 5 years behind the curve.

Publishing companies have caught on to students' tricks--they know about the used textbook market. So they regularly update their textbooks, ceasing publication of old editions. This means that I'm forced to switch to the new editions, and the process starts all over again. Which brings me to the decisions I'm going to have to make soon. I'm not going to switch to a different textbook, but I'm going to have to make a decision about adopting updated editions of a couple of textbooks I use in my courses.

The textbook I use in my PSYCO 104 course has been updated. It was a good book in the first place, and new edition improves upon it. Even better, it's cheaper than the competing textbook many of my colleagues have chosen--more than $20 cheaper. And there's more good news: I'm working with the company to have a custom-published version of the textbook. This will consist only of the chapters I use in my course (instead of all 18 chapters), which will save students even more money.

The notoriously expensive textbook in my PSYCO 267 course has also been updated. In working with the publisher, we've been able to get the price of the new edition to be $35 less than the old edition. I should switch as soon as possible, right? Choose the new textbook for my Spring course, right? Wait a second. What about all those students this term who want to unload their textbooks? If I switch right away, they won't be able to sell their books. And there won't be any used books available to students in spring term, so they'll have to shell out full price for the new edition.

I have a tricky balancing act to manage. If I stick with the old edition, some students--the ones who want to buy a brand new, unused textbook--will have to pay more. But, on the other hand, the students who buy (and sell) used textbooks will be able to save (and recoup) some money. Obviously, I can't please everyone. So I'll have to do the next best thing, and try to do what's best for the majority. But how many students buy new, and how many buy used? I don't know. So help me out--post a message in the comments telling me what you do.

Why aren't you studying?

The Question on Grade Inflation

In a recent open comment, Anonymous (A studious student) had some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. In my last post, I discussed the possible link between grades and teaching evaluations. This post addresses the third of several claims/questions/concerns.

Question: “What is your thought on grade inflation?”

To the extent it exists, it sucks. But I don’t know how prevalent it is. In talking to my colleagues, I’ve found sentiment is universally against it. But then, like I wrote in my last post, maybe we’re all doing it subconsciously anyway.

Claim: “it is getting more and more difficult for me to set myself apart from other students...By 4th year, ~20% of the class is expected to receive an A/A+”

Going to GFC policy on approved grade distributions, it is expected that in 4th year courses, 37% of students are expected to obtain a letter grade of A- or higher, and 20% are indeed expected to receive either an A or A+.

What is this, officially prescribed grade inflation? I can’t speak for GFC, but I’ll give you my view. By the 4th year, there has been some weeding out. Students who have not been able to handle the material have changed majors, or maybe have even left university. So the students who are left are, in general, more capable than those in, say, first year. Also, class sizes at the 400 level are smaller, giving you more access to the instructor, which (I would hope) impacts grades.

So yes, it is literally harder for you to set yourself apart from other students--in terms of grades. But there are other things you can do to differentiate yourself. Talk to your instructors; show interest in what they’re teaching. I’ve formed great relationships with students over the years in part because they did more than just show up to class. In fact, I’ve been privileged to be able to help some of them advance their academic careers, too. (It’s been great watching people go from being undergrads to being practicing psychologists, or holding other positions of importance in the real world!)

Concern: “salary is partially determined by these evaluations (I think), so professors/lecturers have greater incentive to give higher grades.”

Yes, you are correct. Even though evals are not supposed to be the sole determinant of teaching, sadly, those numbers may be the only representation of teaching on my yearly review. I am not a number! There is an incentive to give higher grades only if there is a belief that doing so will result in better evals and thus performance increments. I can’t give you any statistics on this one, and I wouldn’t want to. I don’t want to imply that my colleagues are so shallow. Rather, in working with them on the AASUA Teaching and Learning Committee and in other groups, going to teaching seminars put on by University Teaching Services, and in talking with them one-on-one, I find them--to a person--to be hardworking, dedicated, and committed to doing the best teaching job they possibly can. This is not puffery; I am not stoking anyone’s ego. If the University of Alberta were not seriously interested in the importance of teaching, I would have thrown in the towel and left.

Claim: “Even to this day, most believe that Harvard grades are meaningless.” I have no data on this, and cannot speak to this. Even if the grades are meaningless, I know for a fact that a degree from Harvard is not meaningless. In fact, it can be a ticket to more money than I’ll ever see. I know your University of Alberta degree has value; there are too many people working hard for the reputation of the whole university go down the drain. And I don’t think that’s going to change.

To be sure, the issues you have raised are important ones, and they are being discussed and considered on campus (and on other campuses, too). I hope I have not dismissed your valid comments, concerns, and criticism. Instead, I’ve tried to pull the curtain aside and let you hear my thoughts and ideas. I’m impressed that you have been considering these issues, and have brought them forward for discussion. That’s what I wanted in this blog, and boy did I get it--thanks!

Why aren’t you studying?

Update 3/21/2009: Just found out about the website GradeInflation.com. Are instructors inflating grades, are students getting better, are teaching techniques improving, or is it something else?

The Question on Higher Grades and Teaching Evaluations

In a recent open comment, Anonymous had some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. In my last post, I tried to make the case that I don’t always give out “higher grades,” although there is a tendency for higher grades to appear in some of my classes. This post addresses the second of several claims/questions/concerns.

Question: “Do you think your preoccupation about evaluations are one reason why you are willing to give out relatively higher grades?”

You know what people’s greatest fear is? Public speaking. So how would you like to teach a course? That’s what my graduate supervisor asked me one day. Now, it wasn’t a question. No, it was more of a prediction about the future: You’re going to teach a course. Gulp.

Back in my day, there weren’t any how-to seminars for graduate students to learn about teaching like they have now. The preferred method was to throw you in the deep end and walk away, leaving you to thrash around, coughing and sputtering, waving your arms frantically. This is, of course, terrifying. How are you supposed to improve? How do you know what you are doing wrong, or maybe even, doing right? The answer came a month after my first course ended: teaching evaluations.

The students I taught were very understanding, and gave me some really good constructive feedback on improving my teaching. Incredibly, the students who I had been trying to teach had ended up teaching me some valuable lessons of my own. (I know, this sounds like every Hollywood movie set in a classroom that’s ever been filmed. If anyone is interested in buying my script, please get in touch with my agent.)

So, obviously, reading evals is like eating chips--bet you can’t eat just one! Am I addicted? Am I so desperate to hear nice things about myself that I will pander to students by pumping up their grades?

Answers: 1) I dunno. 2) Geez, I sure hope not.

As it turns out, the AASUA’s Teaching and Learning Committee has been looking at the issue of the validity of teaching evaluations for the past couple of years. As it also turns out, I know this because I’m on this committee. So I have some actual (but general) answers--not just facetious ones.

Is there a statistical link between higher expected grades and evaluations of teaching? Yes. “Class-average grades are correlated with class-average student’s evaluations of teaching, but the interpretation depends on whether grades represent grading leniency, superior learning, or pre-existing differences” (Marsh & Roche, 1997, p. 1194). So what is the correlation? Unsurprisingly, there is a range, which usually goes from 0.10 up to 0.30; the “best estimate” is taken to be probably about 0.20 (Marsh & Roche, 1997). That’s a correlation, but it’s a pretty weak one. In terms of the overall variance in evaluation scores, grade expectations account for less than 10%. If an instructor decides to pump up his or her ratings by inflating grades (and risking his or her career), the payoff isn’t there--there are too many other things that influence the ratings.

Coming back to me, though (because it is all about me). Am I inflating grades to get good evaluations?

Answers: 1) I dunno. 2) Geez, I sure hope not.

I don’t want to know. I do a lot of statistics on the performance of my classes. (I’ll eventually post about how I use point-biserial correlations to analyze exam performance. Your eyes are guaranteed to glaze over! Woot!) I know, for instance, that student evaluations of me are positively correlated with their evaluations of the textbook (r = 0.498 last time I calculated). This is useful information: it’s really important to find a textbook students like. But, duh, I know that anyway. Why is this related to me, is it just spurious?

But I have not, do not, and will not calculate or correlate my evaluations with student performance. I’m not going to take any student rating of my teaching and compare it with the median, mean, or mode of the marks in any of my classes. Why? I don’t want to know. If I’m unconsciously, unknowingly giving out higher marks to get better evals, that’s one thing. But if I’m doing that willfully, consciously, that’s unethical--it’s just wrong.

It may look like I’m preoccupied with evaluations. I do mention them in class, and even put the evaluation date on the syllabus (we are supposed to tell students when evals are going to take place, ya know--I read the fine print). But telling you about how good my evals are, I think, makes my job harder: your expectations increase. In contrast, if I tell you that I really suck and then I kinda don’t, maybe you’ll be happy and give me a good rating.

So maybe it works against me. You think you’ve got this instructor who thinks he’s hot as snot, but turns out to be awful. So you burn me on the evaluations. I welcome that. As long as you tell me what I did wrong, I can still learn and improve. I can try harder next time. Maybe someday I can be as good as the instructors who inspired me to go into this psychology business in the first place, like my graduate supervisor.

I’ll address the remaining concerns expressed by Anonymous in my next post.

Why aren’t you studying?

References:
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187-1197.

The Question on Higher Grades

In a recent open comment, Anonymous made some pretty serious accusations (sorry, sorry, “questions”!) about grades and evaluations. I’d like to address those questions--not just in another comment, but in full postings. This post addresses the first of several claims/questions/concerns.

Claim: “...you also give relatively higher grades.”

So, relative to other instructors, I’m taking it? The University, specifically GFC, has approved grade distributions for different undergraduate courses. The fine print says this: “These distributions are provided for guidance in your grading. It is not necessary for the grades in a particular class to follow any of the distributions exactly.” (Unless an instructor is grading on the curve with the help of a spreadsheet, it’s impossible to get these exactly anyway. And I don’t grade on a curve.)

Instead, I focus on the expected medians for each course level:

1st year = B-
2nd year = B
3rd year = B
4th year = B+

If my classes don’t match these, well, I don’t know what happens. So far, nothing yet.

Anyway, here are the actual medians for the last 12 courses I’ve taught:

1st year: B, B
2nd year: B+, B+, B, B, B-, B+, B+
3rd year: B
4th year: A-, B

Any patterns? Am I consistently giving higher grades? It looks like the 100-level courses are a bit higher than expected. Why? Major components of that course (20% of the overall mark) consist of easy marks (Information Literacy, Research Participation) that boost students’ grades. These components are out of my hands; I don’t do any marking, I just accept the results as they are. So should I make my exams harder to compensate for these “free” marks? Of course not. Class means on my exams in that course are around 65%, and I don’t want them any lower than that.

Let’s skip to my 400-level course. Yup, I recently had a class earn a median of A-. They all deserved it. It was the best bunch of students I’ve ever had in that course, and I was really happy to give the marks I did. Their term papers were great, and their exams were outstanding. Didn’t even know the median was so high until it popped out of my spreadsheet when I was filling in the final grade forms. (And look, another 400-level class only got a median of B.)

It looks like there’s something funny going on in my 200-level courses. Yup, the grades are a bit high, tending to a median of B+, whereas GFC expects a B. That’s not a huge difference--in terms of the percent cutoffs I use, 72% is right in the middle of my “B”, whereas 76.5% is the middle of my “B+”. That’s a difference of 4.5%. Still, for a class of over 200 to have a grade that’s almost 5% “too high” is significant.

So, why the high marks? In my 200-level perception course, the textbook I used was extensively revised a few years ago, and the testbank of multiple choice questions that comes with it was really--how should I put it?--simplified. Because these questions make up about half of the exam, the marks went up by a few points. In my 200-level cognition course, the textbook I use is written by the same person who wrote my perception textbook. This textbook was also recently revised. Guess what the testbank is like?

It’s tough to rewrite dozens of exam questions, but I’m slowly working on it. Realizing that the marks have been increasing, I’ve also been slowly changing the percentage cutoffs for each letter grade. I don’t want to make huge changes all in one term--that’s not fair to those students. But it’s also not fair to give them inflated grades compared to other terms.

I’d like to think that my teaching improves over time--but is this reflected in students’ grades? If that were universally the case, wouldn’t instructors near retirement have sky-high marks in their classes, and wouldn’t graduate students teaching their first class have rock-bottom marks? Hmm, unless those sneaky novice instructors are inflating their students’ marks.

But that’s the topic of my next post.

Why aren’t you studying?