The Research: Primary vs. Secondary

As a scientist, I do research. The first thing the word “research” brings to mind is probably experimental research. But this is only one method under the broader umbrella term of empirical research, which can include other methods like surveys, for example.

Another way of dividing up research into different kinds is into primary and secondary. In primary research, you collect original data; you’re discovering something no one else has ever known (you hope!). In secondary research, you are going through data that has already been collected. Maybe you are looking for something specific, or maybe you want to do a (formal, statistical) meta-analysis. (This doesn’t mean that every time you do a Google search, you’re doing secondary research--but secondary research might employ an Internet search now and then. More likely, I’ll use PsycINFO or MEDLINE.)

I do a lot of secondary research in prepping my courses. For example, when I created my lecture on synesthesia, I did a lot of secondary research--searching for studies, reading and analyzing them, and synthesizing the information in a systematic, coherent way. (At least, I hope it’s coherent! ;-)

I also do some primary research. It’s not something that I’m required to do in my role as Faculty Lecturer (but it can be a lot of fun to do). In fact, the University makes it hard for contract academic staff to do primary research: we are not allowed to apply for research grants. As you can imagine, having no money makes it kinda hard to do research. Unless: a) you’re rich, b) you have a sugar daddy, or c) a publishing company comes to you with a bunch of free stuff and asks if you’re interested in using it to do a study.

Late last year, I had the opportunity for option c). In the next few posts, I’ll describe the steps in the research process, ending up with a summary of my results.

Why aren’t you studying?

The Awards: 5

I've been named to the Department of Psychology's Honour Roll with Distinction for all three courses I taught last term. Thank-you to everyone, and special thanks to those who went to the trouble of giving written comments. I'm not going to post "best-of" student comments this time because (a) I've done that before, (b) there weren't many comments that, er...cry out for a response (most were constructive and helpful, which is great!), and (c) I don't want to reinforce anyone trolling for their comments to be posted in this blog (getcher own blog, eh?).

This time, I want to congratulate my colleagues who were named to the Honour Roll:

  • Brown, N. (PSYCO 405 X5)
  • Dixon, P. (PSYCO 258)
  • Friedman, A. (PSYCO 212)
  • Hurd, P. (PSYCO 400/409)
  • Masuda, T. (PSYCO 241 B1, PSYCO 305)
  • Passey, J. (PSYCO 105 B1, PSYCO 233)
  • Schimel, J. (PSYCO 105 B4)
  • Spalding, T. (PSYCO 105 B3, PSYCO 405 B2)
  • Snyder, M. (PSYCO 403 B2)
  • Westbury, C. (PSYCO 339)
  • Wylie, D. (PSYCO 267 B2)
And those who were named to the Honour Roll with Distinction:
  • Busink, R. (PSYCO 436)
  • Caplan, J. (PSYCO 403 B4)
  • Colbourne, F. (PSYCO 403 B1)
  • Gagne, C. (PSYCO 532)
  • Hurd, P. (PSYCO 414/505)
  • Kuiken, D. (PSYCO 415)
  • Lee, P. (PSYCO 105 S1)
  • Mou, W. (PSYCO 403 B3)
  • Mullins, B. (PSYCO 104 B2)
  • Nicoladis, E. (PSYCO 323)
  • Noels, K. (PSYCO 300)
  • Passey, J. (PSYCO 241 S1, PSYCO 405 B1)
  • Singhal, A. (PSYCO 377)
  • Spetch, M. (PSYCO 485)
  • Todd, K. (PSYCO 475)
  • Varnhagen, C. (SCI 100)
  • Watchorn, R. (PSYCO 323)
  • Wylie, D. (PSYCO 405 B3)
Quite a list, isn't it? I think the criteria are pretty stringent (see below for details); that means the Department has a lot of great teachers. I am humbled to be included among them.

Here are the criteria for the awards:
1. The course section median response was equal to or greater than 4.0; for Honors with Distinction, the course section median response was greater than 4.0 and at least 45% of the students agreed strongly that the instructor was “Excellent;” For classes with fewer than 10 students enrolled, the majority of students responded “Agree” or “Strongly Agree”; for Honors with Distinction, the majority of the majority responded “Strongly Agree”;
2. At least 60% of the class responded to the questionnaire;
3. There were no abnormalities in the grade distributions (e.g., distributions skewed too high or too low);
4. Instruction was conducted in accord with the ethical standards of teaching as outlined by the APA and CPA.

Why aren't you studying?

The Value of Your Degree

I’ve posted about The Best Job in the World, according to a number of metrics. Now, a new study has been released, looking at the earnings of people who hold a bachelor’s degree.

Researchers at Georgetown University Center on Education and the Workforce analyzed US Census Bureau data on over 3 million bachelor’s degree-holders who graduated over the last 40 years. Specifically, they recorded the median salaries they earned in 2009; this means that they took a cross-section of people currently in the workforce (it didn’t just look at people who graduated in 2009).

I’m sad to report that the field with the lowest median salary is Psychology and Social Work (P&SW) at $42K (range: $29K to $53K). Sob. Below is the breakdown within P&SW (from The Chronicle of Higher Education):









The lowest is counseling psychology, ringing in at a paltry $29,000 per year. At the top of P&SW is I/O psychology (those who work in large companies, measuring and improving performance and/or wellness). I guess those companies pay pretty well; the pay is almost double that of the poor counseling psychology graduates.

But wait--what are those lowly counseling psychology-degree-holders actually doing? They’re not working as counseling psychologists. Why not? Generally, you can’t, not with just a Bachelor’s degree. Maybe they just got their BA and are now working in retail. On the other hand, you can work in I/O psychology with “just” a bachelor’s degree.

Also, the US economy isn’t in great shape. The data came from 2009, when the job situation was pretty grim--not that it’s great today. It’s possible that some people had their salary cut, or at least not increased recently. Still, the numbers above are based on full-time, full-year workers with a Bachelor’s, not part-time workers.

Finally, those who hold higher-level degrees like a Master’s or Ph.D. earn more. According to the report, median earnings of those with a graduate degree in P&SW was $60K, moving P&SW up to the third-lowest field.

So, did you make a bad choice of major? Should you have taken engineering (overall median: $75,000)? Or computing science (overall median: $71,000). Maybe we’re just all in this for the love of it.

Why aren’t you studying?

The Random Facts 2

No, still no Twitter account. Why? I can't come up with a good name. Is “Sh*t my prof says” taken? So, more random facts (and some questions).

  • First, the Taco del Mar by my house closed, then the Taco Bell closed. What have I done to offend the taco gods? I am currently leaving offerings of money at the nearby Mucho Burrito...
  • never get three dental fillings a couple of hours before teaching a class
  • Me: “What are you watching?”
    My wife: “Um, Keeping up with the Cardassians.”
    I wonder to myself, hmm...Elim Garak, or Gul Dukat?
  • Is a public toilet seat still warm from the previous occupant gross, or kinda nice?
  • Oh, and don’t hold anything over a toilet that you want to keep.
  • Have Cadbury Crème Eggs gotten a lot sweeter since I was a kid, or have I just gotten more sensitive? That’s what I thought.
Do you have any random facts or questions?

Why aren't you studying?

The Spring Term, 2011

I've written about Spring Term before, describing my misconceptions about the kind of students who take courses in Intersession (Spring and Summer Terms), and about their reasons for taking them. This time, a little bit about one of the greatest challenges in taking an Intersession course: time.

Courses in Spring (and Summer) Term last for 6 weeks. Unless you've signed up for a 3-week Spring or Summer Term course. That's right: first day of class to final exam in 3 weeks. I've taught 3 weeks courses before, at another institution. It's 3 hours of lecture every day, Monday to Thursday. Then on Friday there's a midterm and another lecture to round out the 3 hours. It's brutal. Don't even think about taking two of these monsters at the same time. Your social life will go out the window and you'll have to catch up on your favourite TV shows later. It's just go to class, read the textbook, go to class, read the textbook. So, in contrast, a 6-week course doesn't sound too bad.

In 6-week courses, there's "only" 70 minutes of class every day, and an exam "only" about every 2 weeks. Let's compare (labs aside, assuming a full course load):

  • "regular" Fall/Winter Term: 770 minutes (about 12.8 hours) per week in class (five courses)
  • Spring/Summer Term: 700 minutes (about 11.7 hours) a week (two courses)
For me, here are my in-class teaching hours (not counting prep work, of course):
  • Fall/Winter: 7.8 hours/week
  • Spring/Summer: 11.7 hours/week
(Net result? I have to work harder in Intersession.)

The point of all this silly number crunching? Don't take Intersession courses lightly. Be prepared to devote your time appropriately--these courses still count toward your GPA. It's tough to concentrate on the yada-yada of your instructor when the sun is shining and the birds are chirping. *sigh*

If you've taken Intersession courses before, what are your tips for making it through?

Why aren't you studying?

The Earth Day

Happy Earth Day, Earth! And to everyone on it! (Sorry, astronauts on the ISS, you don’t count. But hey, you’ve got an awesome view out the window. Of the Earth.) It's good to have a reminder about the value of the Earth, our environment, and ways to protect it.

At work, I strive to be environmentally friendly:

  • No recyclable paper gets thrown in the garbage. Any paper that goes into the recycling almost always has both sides used. I admit that I do use a lot of paper--exams, mostly. You wanna help me recycle exams? Heh, nice try. They all go to be shredded and then recycled.
  • Dead batteries from the wireless mics I use get dropped off in an ECOS box, along with my empty printer cartridges.
  • I turn off the lights in my office when I go to class; I also turn off the lights in the bathrooms in Bio Sci when I leave. (Sorry about leaving you in the dark that one time, guy in the stall. Oops!)
Am I a paragon? Of course not. Apparently, I’ve committed the worst kind of enviro-crime possible, twice: having kids. Um, sorry?

Speaking of family, here are some things we do:
  • We don’t buy bottled water; every family member has their own reusable aluminum bottle (so no fighting over the one with zoo animals on it!).
  • In my house, I’m famous for eating food that’s past its expiry date. Long past. Hey, I hate to waste food. What's a little borborygmi?
  • Every garbage day, I put out more blue bags than garbage bags--in fact, no one else on my block puts out as many blue bags as I do. It takes me over an hour a week to clean, rinse, sort, and otherwise prep all of my family’s recyclables.
Here’s a funny story. Before the City of Edmonton’s Blue Bag curbside pickup recycling program, there was the blue box curbside pickup recycling program. Not only did you put your recyclables in a, well...blue box, you had to sort everything, too: cardboard vs. plastic vs. clear glass, etc. So when the recycling-pickup person came, they tossed the cardboard into one compartment of the truck, plastic into another, and so on. One recycling day, the recycling-pickup person had (oddly) left behind a large piece of cardboard. When I picked it up, I saw that they had written me a note of appreciation, thanking me for the good job I had done sorting and arranging my recycling. Aww! Thanks, recycling-pickup person!

Could I do better? Absolutely. I could bring my own rechargeable batteries to use in the wireless mics. I could try to always remember to shut off the computer and turn off the lights in my classrooms, if it’s the last class of the day. I could buy products with less packaging to reduce my recyclables.

What are you doing to help save the Earth?

Why aren’t you studying?

The End of Term

Just a quick reminder, in case you've forgotten: the end of term is approaching.

Yeah, yeah--terms papers this, studying that, yada yada. But there's another reason to remember the last day of term. I ask that, if you have any concerns about the marking of any assignment or exams in the course, you bring them to me or the TA before classes end. I'm going to be heads-down marking term papers, and your TAs are students with exams to prepare for, too. Although I've asked them to keep their usual office hours up until the final, they may just have to write an exam during one of their office hours.

Yes, according to Department of Psychology policy, it is your right to wait up until some unspecified time before the day of the final exam to bring your concern forward:

"With the exception of term work for which students did not receive feedback before the posting of final grades, students must initiate a request for reevaluation of term work with the instructor prior to the day of the final exam or in the case of courses without final exams, before the posting of final grades."
So on or after the day of the final exam, however, pfft! that's it. Even if we made a marking mistake, we are not allowed to change it after that day. If there is a significant concern, please don't wait until the last minute. Note that this doesn't apply to simply asking questions to clarify something; you can do that at any time--even after the course is over.

Why aren't you studying?

Teh Wrnog Mnoth

Only a week into this month, and already this isn't my month. It must be someone else's month. Let's see what's gone wrong so far...

  • Major virus scare: I thought a rootkit had infected all of my computers. This happened to my office computer in 2008, so it's not unprecedented. My email app was very slow, so I got a packet sniffer to check out my IP traffic. What are all these connections to 1e100.net!? (Oh, um, I get it. "1e100" means a 1 with 100 zeros behind it: a googol. Get it?)
  • So, no virus, but Gmail is insisting on rebuilding my All Mail folder every time I start Thunderbird, downloading 30,000+ messages. That'll slow things down a bit. Darn you, Gmail!
  • Wrote an important email, and misspelled the address. It didn't get bounced back until hours later. Darn you, Gmail!
  • Getting my coursepack together for spring term, I photocopied a couple of chapters of a book. Then, instead of emailing me the results as a PDF, the photocopier ate everything. Twice.
  • Due to an apparent glitch in HERO (the UofA's online ethics submission/review platform), my ethics application was not reviewed. Instead of taking 5 days, it took 5 weeks. In the meantime, my hands are tied, and I can't get started on my research project.
  • Ordered a couple of things online--both companies sent me the wrong item. Grrr!
  • My PVR stopped talking nicely to my TV over HDMI. Banged my head against the wall (actually, the floor) for hours on that one. Decided to switch to digital cable and get an HDPVR. It didn't talk nicely to my TV over HDMI. The good-humoured tech who came to take a look at it liked the fact I knew what "HDMI" was. But he still couldn't fix it.
  • Tried to get a delicious Nestea from a vending maching right before class and was denied. I got my money back, but it wouldn't give me any delicious Nestea. (And no, I don't want that Brisk crap.)
  • Daughter #1 made me spill my coffee this morning. Oh, the humanity!
Notice a pattern? (OK, except for the last one.) Yup, technology is giving me fits. I'm not superstitious. But if this is your month, you can have it back.

Why aren't you studying?

The New Logo

You may not have noticed, but the University of Alberta has quietly changed its logo. The changes in the "shield" and "wordmark" are subtle, but they're there: compare the new one (top image) to the old one (bottom Linkimage). Without any fanfare, the new logo appeared on the UofA home page on March 7, 2011 (heh, they even remembered to change the l'il favicon, too). This change came after a staggering 100,000 people (me included) participated in research to evaluate the new visual identity.

It's odd that there was no big press conference about this. I don't think it's because they're not proud of the new art scheme, but rather because they don't want to draw attention to the old one. I mean, lookit it! It's got Times Roman font--yuk! That's almost as ubiquitous as Helvetica. Times Roman is so, you know, authoritative (maybe even authoritarian). And you don't want a university looking too, like, authoritative or whatever.

So, why the change? Rumour is that some people thought the old logo was too stodgy for university-choosing kids coming to the UofA homepage; it should be updated to be more distinctive on the web. But hold on, who starts by going to homepages anymore? And what about all the existing (old-logo) letterhead? (We're assured that it won't be thrown out, but used up and then replaced with the new design.) OK, but there are also a lot of electronic documents that will have to be updated.

I'm not opposed to change, as long as there's a good reason for it. But I didn't see a pressing need for change here. On the one hand, the logo (which has been around for ages--anyone know how long?) is being changed (sorry, "refreshed"). On the other hand, administration is going back to 1908 and promoting Henry Marshall Tory's assertion that, for the University of Alberta, "The uplifting of the whole people shall be its final goal."

I thought we were in the middle of a budget crisis. Why go to all this trouble and expense? This doesn't look like belt-tightening, it looks like a way to spend a bunch of extra dollars. Will this attract significant numbers of new students, who otherwise would have presumably gone to Waterloo because of its spiffy new logo?

What do you think? Is the new logo sufficiently kewl? Is it not kewl (or different) enough? Did you even notice?

Why aren't you studying?

The Gmail (update)

I’ve been having a problem using the Mozilla Thunderbird (“Tb”) email client to access Gmail via IMAP. I don’t want to delete a message and sent it to [Gmail]\Trash, because any message in there will be deleted after 30 days. Instead, I want to archive deleted messages in another folder, [Imap]\WebTrash but this hasn’t been working. I can set Tb to do this correctly, but after a restart, deleted messages will end up in [Gmail]\Trash. Grr! Turns out this is because Tb has a bug.

Tb bug 533140: “Cannot specify custom trash folder using Gmail IMAP ([Gmail]/Trash is always used regardless of trash folder selection at Server Settings, because Tb currently ignores trash folder selection if Gmail IMAP in order to avoid unwanted problems)” since 2009-12-05.

Here’s a solution from Gmail engineer RyanTaylor:

The first step is to create your own Trash folder:
1, In your Thunderbird, create a new folder under your GMail account for your own Trash, let's call it "MyTrash" for now.
2, Go to your account settings (Right click on the account / Settings...) and then select Server Settings.
3, Set the "When I delete a message", "Move it to this folder:" and select your MyTrash folder.
4, Click OK.

Then make sure that you unsubscribe the GMail Trash folder:
1, Go to your subscription settings (Right click on the account / Subscribe...)
2, Find the Trash folder under [GMail], select it and click on "unsubscribe".
3, Click OK.

At the last step, you need to restart Thunderbird. When the trash icon is appearing in your newly created label, then it should to be working.
It works for me now, yay! (AICT sent me the fix, but HT to Dr Connie Varnhagen for finding it first and telling me about it!)

Finally, here’s one more resource from the MozillaZine Knowledge Base on “Using Gmail with Thunderbird and Mozilla Suite (Troubleshooting and Gmail Quirks)”. That's right: "Gmail Quirks"!

Why aren’t you studying?

The Guest Lecture

I gave a guest lecture on Wednesday. This is not something I do very often--because no one asks me, that's why. It's a strange feeling, stepping into "someone else's" class, taking over like you own the joint. There's a whole class of students who are used to a certain way of doing things, then suddenly there's some new person who does things all differently. Like using PowerPoint, or something else very strange. And maybe those students are taking that class because they certainly don't want to take my class. (Potential nightmare: walking into the classroom and everyone groans and says, "not that guy".)

This time, I filled in as a favour to Dr Elena Nicoladis. She had a graduate student's candidacy exam to attend, so she couldn't make it to her PSYCO 323: Perceptual and Cognitive Development class. (That's a pretty good reason for missing a class. Not like my reasons: because I gotta go pick up my laundry, or because my fish has the hiccups.) So she asked me, and I couldn't say no. See, she's currently the Department of Psychology's Associate Chair (Undergraduate Program), which makes her...well, kinda-sorta my boss. What, I'm gonna tell her, "Naw, why should I? Help you? What's in it for me? Forget it."? Because when it comes time to renew my contract, she might tell me, "Naw, why should I? Help you? What's in it for me? Forget it."

There's another reason why she asked me--besides the fact that I'm her minion--and that's because I once taught PSYCO 323. Well, twice, actually, way back in 1996 and 1997. OK, that's not exactly true either. I co-taught it twice, with Dr Katherine Robinson. (Co-teaching, that's another strange experience--sometimes one instructor shows up and teaches, and sometimes the other one shows up. It's like flipping a coin, but you never lose. Sorry, I had to say that. Kathy bought me a coffeemaker as a wedding present, so I wanted to say something nice about her!)

Digging up those old lectures was not easy. They were 14 years and many word processors ago (my dearly beloved Ami Pro 3.0). Turns out, Ami doesn't like current versions of Windows, and Windows doesn't like Ami much either. I spent a lot of hours looking for filters that I could shove down Word's throat so it would be able to read my old files. After spending far too much time, I got it to work. A-ha! Now I didn't have to create any lectures from scratch. Except...those old lectures? Crud. Utter crud. Totally outdated. And boring. Rats!

After way too many hours of updating my knowledge of perceptual development, I finished my new lecture. Now it was slightly less cruddy, and no longer totally outdated. (Of course, this is sort of a violation of my self-imposed moratorium on creating new lecture material this year, but it's technically for someone else's class. I love finding loopholes in my own rules.) I had to fill an 80-minute class, because I was told that my lecture was on the 10th of March. Except...it wasn't. At the last minute, Dr Nicoladis told me that the lecture was actually on Wednesday the 9th, which means a 50-minute class. But--but--but! That means I'd have to cut my lecture down by almost half! It's really hard to cut out material--almost as hard as creating it in the first place. So I took a hard look at my lecture. What to cut? Ironically, I cut out the only remaining bits from my original 14-year-old lectures. *sigh*

Why aren't you studying?

The Gmail

Because of my awesome specialness, I've been chosen to transition to Gmail. OK, maybe I was randomly selected, I dunno. Anyway, I had finished marking my pile of midterms and was waiting to swap with my TA, so I had a bit of time and decided to go ahead and activate my UofA Gmail.

I'm not new to Gmail; I've been using it since the summer of '04. Back then, Gmail was in beta, and you could only get an account with an invitation which was very highly prized by Googlers. (Yup, I'm bragging: "I've been on Gmail since you were in elementary school, la-la-la.")

I got my invitation by promising to send a postcard about Edmonton to whoever sent me an invite. I sent out two postcards; one person stiffed me, but the other sent me an invitation. And...big whoop. It's email. I've been doing email since, well...I sent my first email in 1989 ("since you were in diapers, la-la-la"). I'm not crazy about Gmail's odd way of labeling messages instead of physically putting them in different folders. The concept of archiving everything and actually deleting nothing was novel, and getting a whopping 1GB of storage free at that time put Hotmail and Yahoo! to shame. But then there's the fact that it's web-based, and I'm pretty fond of my desktop email client (I love me my Thunderbird--the software, not the wine) because of its customizability and extensions.

Anyway, here's how my transition went: hair-pullingly bad. You have to change your password, which is a nuisance--now I've got to update account info and passwords in all my password-management software, argh! Next, I (apparently) wasn't able to change or update my existing account in Thunderbird--I had to set up a new one. That's a hassle. What about all of the email messages in my old account? (Turns out, you can request that they transition all of that over to your new Gmail account, but it could take "days or weeks".)

So OK, I set up Thunderbird to use IMAP, tried logging on, and got an error message. WTF? Did I not enter my new password correctly? Did I not enter all the server settings right? Argh! Turns out, I had to go to a Google webpage and enter a CAPTCHA, I guess to prove my human-ness. That would have been good to know in advance, AICT. So it's working, but now I have to sync all of my computers with Gmail's IMAP servers to download and reindex 25,000+ messages. That takes a little while. Like, hours.

If you don't or can't use an standalone email client, you can use the Gmail web interface. It's no different than regular Gmail. It is a bit of a pain if you already have a Gmail account, because now you have to log out of one account if you want to log in to the other account. And don't forget to update the info in your password-management software, which tends not to understand what's going on if you have two logons to the same website, argh!

Finally, after you change over to Gmail, there's no going back. You won't be able to log in to the old UofA webmail--it will fail. Eventually, though, everyone will have to transition; it's just a matter of time.

Is this a fabulous new experience? No. I had my email setup just the way I liked it, and now I've got to chase down bugs and get used to Gmail's weird labeling-instead-of-filing process. But this process was about saving money in the first place. And now we also get cool Google Apps like Calendar, Docs, and more. I've been using all those already ("since before you hit puberty, la-la-la"), but now everyone else gets to as well. Maybe we'll all be more productive. FYI, "Google ate my term paper" is not an acceptable excuse.

Why aren't you studying?

The Awards: 4 (part 2)

So many good comments, I had to split them into two posts. Last time, I covered my perception class. This time, let's hear from students in my 100-level course.

(Warning: As always, remember that snarkiness and sarcasm filters are now OFF.)

Intro psych:

"created a sort of a sexist environment that made a lot of people in class think it was ok to say things that were even more inappropriate. For example, _such_ gendered language e.g., always 'Mom, Dad, husband, wife instead of spouse, parent', encouraging ppl to shout out stereotypes like 'Women are always pmsing, crying, talking, getting what they want.' Saying that the fluffy light Psychology magazine was aimed at Women & that he would never read it."
(I'll go one better. How about I just say "persyn"? Sure, it'll lead to confusion, but it's better than me talking about my wife all the time. And I will ensure that everyone in class submits their responses in writing for my approval before they are allowed to speak in class. Oh, and Psychology Today is closer to Modern Bride than Car & Driver. I know, I measured it on the rack. But I still do actually read Psychology Today.)

"...made me not pay attention in class, he is not good with keeping my attention. I resorted to doing crossword puzzles and checking twitter."
(I'm so proud! The Behaviourist Approach crossword is a good place to start. Mind Hacks has a great list of psychology and neuroscience on Twitter. Glad I could stimulate your desire to learn more about psychology, using new media!)

"We always had interesting and interactive things to do in the classes. It was really helpful and made the classes enjoyable to attend."
(And if you get bored, you could always check Twitter.)

"I feel as though it would be very difficult to pass this course without reading the text"
(Your feelings serve you well, Padawan. Plus, I kept saying how the majority of the exam questions come from the textbook. But a Jedi has no use for such things.)

"I refuse, on principle, to read the entire textbook to do well in this class."
(What a coincidence: I refuse, on principle, to give a good mark to anyone who doesn't read the entire textbook in this class.)

"I didn't like the fact that every chapter was assigned for reading instead of assigning pages, which would have been more helpful."
(That's not helpful. I'm going to assign individual words. Now that's helpful.)

"Only include questions from notes because not everyone can afford to purchase textbook."
(Then why assign a textbook? I'm not going to teach a class that doesn't have any form of required reading. Hey, I know! To save you money, I could have put a copy of the textbook on reserve for you.)

"Also, the textbook that was on reserve in Cam library was very useful. I would have failed the course without it. It also helps students who are more 'economically conscious'"
(Oh right. I did put the textbook on reserve for you. Too bad I'm not currently allowed to do that. Don't be mad at me--it's not my fault. Direct your concerns to UofA administration. Thanks.)

"Don't make the infolit assignments due on a Friday or Saturday."
(You can do the assignments on a Thursday. Or Wednesday. Did you know that? You don't have to do them on the exact day that they are due.)

"The tutorial was out of date for Assign #3 and VERY difficult to follow. The instructor should have notified students that the tutorial did not match teh new library database & given written updated instructions."
(Unfortunately, I can't access the infolit assignments. I have nothing to do with them; I am required by the Department of Psychology to have them in my intro psych course. But good thing you waited until the end of term evaluations to bring this up, though. That way, there's absolutely nothing I can do about it.)

"Lecture moves way too fast for those of us who don't want to print off an entire novel of fill-in-the-blanks and are instead taking notes manually."
(So for the three of you, I should slow way down so you can copy the notes? OK, but only as long as it doesn't bother the other 261 students in the class. I don't want them tuning out and checking Twitter.)

"I would describe this course as a mile wide and an inch deep."
(That's a 100-level course for ya. Well, that's a "survey course" for ya. I have added material that goes into greater depth, but then I can't cover everything in the textbook, too.)

"not a big fan of the textbook/lecture content differences"
(Not a big fan of the vague, unhelpful comments. Do you mean the textbook contradicted the lectures? Or the fact that I don't just read out of the textbook to you, and instead include topics that many think are interesting, like the psychology of happiness?)

"Have a clicker for miniature in class assignments, not graded, and inexpensive, used as a learning tool"
(... That's actual, serious, pedagogical advice for me. I will strongly consider that. Thank you.)

"Stop with the lame jokes. Not everyone is 4 years old."
(OK, I'll try to aim higher: 6-year-olds. I've got one I can practice on. Fart sounds are popular.)

Why aren't you studying?

The Awards: 4

OK, so I got on the Department's Honour Roll with Distinction for all three courses I taught in Fall, 2010. Yada yada. Now--you know 'em, you love 'em--on to student comments. (Warning: As always, remember that snarkiness and sarcasm filters are now OFF.)

Perception:

"I really, really wanted to write something ridiculous just to be featured on WAYS, but I decided that might hurt your chances at getting honor roll again, which you truly deserve...[I] don't give "pity laughs" to profs trying to make jokes...with that in mind I can say I laughed a lot, which kept class interesting and made me want to be here."
(Aw, gee, thanks. And here you are on WAYS, even though you didn't write anything ridiculous. Others did, however. BTW, don't worry about your comments hurting my chances at getting on the honour roll--my abysmal teaching will take care of that.)

"Don't mock previous student evaluations at the beginning of the course under the guise of being fair. What are the important concepts? Emphasize these. What is the purpose of this class? I don't feel that I undertand perception any better than when I first began this class. Teach more. I'm not paying for you to distill notes from a textbook, I can do that myself. SPEAK LOUDER!!!"
(I will only mock evals here on my blog, 'K? "Teach more", eh? I'll have to look into that. In the meantime, learn more. AND DON'T WAIT UNTIL THE COURSE IS OVER TO TELL ME TO SPEAK LOUDER!!!)

"I found the instructor did not treat students with respect. He was arrogant and mocked students with legitimate comments/questions."
(WTF? Here's what treating students without respect would actually be like: "That's a stupid question. You're stupid. Don't waste my time with your stupidity." I don't do that; I've never done that. You must be confusing me with some asshole.)

"Notes are overly wordy."
"Lecture skimmed topics."
(I'll reduce the number of words, but at the same time go into more depth. Necessarily, polysyllabic entities will be employed. Antidisestablishmentarianism!)

"Notes were very disorganized & hard to follow. I am still unsure of what the purpose of this course is. Nothing memorable about information provided because it was delivered in a dull way. I strongly dislike perception because of you."
(Well, then, mission accomplished. I hope you dislike it even more after you finish taking my Advanced Perception course this term.)

"I am near the end of a 4 yr degree and this has been one of the most enjoyable classes I have taken so far...not so much for the content but for the excellent presentations & entertaining lectures."
(Well, then, mission accomplished. I hope you enjoy it even more after you finish taking my Advanced Perception course this term.)

"I really appreciate how Dr. Loepelmann knew beforehand that some topics were harder to grasp than others and took the time to explain it to us in different ways (approaches) so that we understood the material. He is an excellent prof; one of the best one I've ever had in university. I specifically chose this class because he's teaching it."
(Thanks--it's all part of my teaching philosophy: Teach more.)

More comments coming soon!

Why aren't you studying?

The Open Comments: 2

I really should do this more often than once every *cough*twoyears*cough*. This is an "open comments" post. I've got midterms on the brain (and I bet you do, too), so I'd like some mid-term evaluation from you.

This is an opportunity for me to ask "How am I doing?" and an opportunity for you to provide some formative evaluation--as opposed to the summative evaluation at the end of term.

Should I speak up? Should I slow down? Let me know.

Why aren't you commenting?

The Random Facts - 1

Why? Because I'm too lazy to set up a Twitter account.

  • When you feel how cold your spouse's hands are, it's not a good idea to ask, "What, are you a dead person?" Actually, it's never a good idea to ask that.
  • All that snow we got in January? That almost paralyzed the city? Yeah, it wasn't a record amount for January. That record was set in 1971, with 66 cm. Hmm, why don't I remember that? Oh, right--I was in my crib. (No, not my house, yo. Like, an actual crib.)
  • A sump pump is an important thing to have in your house. It's even more important if it's actually working. Luckily, I have the ears of a hawk, and heard mine gurgling and buzzing its last. $600 later...
  • Norovirus is not my favourite virus. When one person in my family gets it, we all get it, falling like dominoes. (How I managed to not miss teaching a single class, I dunno.) I'm not crazy about Coxsackievirus, either. Or influenza or rhinoviruses.
  • There is no such thing as "one cookie left."
Do you have any random facts to share?

Why aren't you studying?

The Letter of Reference

Deadlines are fast approaching! For many graduate programs, the deadline is early February or March. That's why I've been so busy lately. No, I'm not applying to get into grad school. Been there, done that, got the degree(s). I'm busy because I'm writing letters of reference for students who want to further their education--grad school, rehab med, med school, and even law school.

It's a good idea to prepare yourself long before your application deadline--years before. This includes thinking about who you will potentially ask for a reference. Here are some tips:

  • First, what kind of class did you take? A small lab class? That instructor would be able to say things about your hands-on research abilities. A class with a lot of written assignments/papers? In that case, your ability to structure your ideas and communicate them effectively could be described. A class with multiple choice exams only? Um, this is what you'll get: "Dear Institution: This person apparently took my class and seems to have gotten a grade of X. Cheers."
  • Request a letter of reference from an instructor who knows you well. That is, did you just go to class? OK, that's great, but unless you asked a lot of intelligent, insightful questions in class--and mentioned your name a lot--that's not enough.
  • Did you make an effort to go to the instructor's office to ask any questions you had? Your extra effort in traipsing all the way to some office out in the armpit end of campus will be noted, and remembered.
  • Oh yeah--one more: Did you get an excellent mark? Like, at least an A? (That goes without saying, I hope.)
I won't write a letter of reference for just anyone. My criteria are spelled out in my FAQ. The periodical University Affairs has a good articles on letters of reference, including How to Ask for a Reference Letter (this one is about letters of reference for jobs, but the general advice still applies). There are also articles on how to write them, and How to Properly Turn Down a Reference Letter Request (in case you're wondering why someone might say no).

The best part for me (and, okay, for students too, I guess) is hearing that they were accepted. That makes me happy--that I've been able to help someone on their way, just like others helped me. Getting presents--like the one in the photo--upon getting accepted is definitely not required. (But thanks anyway, E.H., the cookies are yummy--and congrats again!)

Gotta go finish up one more letter of reference now...

Why aren't you studying?

The Cover Page

In my Perception class, students are allowed (actually, encouraged) to decorate the cover page of their lab assignments. My TA and I pick out the best two or three, and I hand those back personally in class. (Last term, the class even clapped for the chosen ones.) If your cover is chosen, you won't have to rummage through the piles of labs to find yours--it's hand-delivered to you in your seat. Like you're in a luxury box at Rexall Place or something. Hey, you earned it!

For those that aren't chosen, however, there still can be a benefit. If your cover really stands out, that makes it a lot easier to find among the sea of plain white cover pages. And, as an extra side benefit, this is an application of the visual attentional phenomenon of popout. Eh? Eh? Clever, huh?

I promised my class this term that I'd post some photos of the best cover pages from last term. Some recent covers were so memorable that I took pictures of them with my phone. My dumbphone. Because it's not a smartphone, see? Plus: I'm mad at my dumb dumbphone. I don't have any pictures to show because my 0.3 megapixel dumbphone took the world's worst pictures of cover pages. No, really--some of them were so blurry even I didn't know what some of them were. Me, photoshopped to look like Han Solo? Actual Halloween candies taped to the cover? No, maybe it's a LOLcat. So, sorry about that.

Anyway, here are some dos and don'ts for cover pages:

  • origami: it's just going to get squashed and ripped--forget it
  • macaroni: frowned upon--what, are you still in grade 2?
  • fusilli: ah, now you're talking--can you make a fusilli Jerry? (no bonus marks for that)
  • glitter: past TAs have nixed glitter and sequins--glitter glue may be okay, but ask first
  • $5 bills: no, no, no--the university says that money must not be submitted with assignments (just drop it off at the Provost's office)
  • drawings/sketches: if you have the time, patience, and ability to actually draw something, well, I've got a soft spot for that
Why aren't you studying?

What I Did on my Christmas holiday

It was, as usual, a busy time these past couple of weeks. At the end of December, I tried to blitz through my marking as quickly as I could, to get final grades submitted before everything shut down for the holidays. Then there was Christmas: made a turkey, the kids went crazy ripping their presents open, and my wife still hasn't opened the iPad I got her (it...scares her).

Then there were a bunch of furlough days, when I was not supposed to be working. So, on my furlough days, I did some work. Waitwaitwait--nothing related to the University (of course), but my contract work for Nelson Education. I'm a consultant for their higher education division, working on the websites for their Canadian (or "Canadianized") psychology textbooks. (Full disclosure: yes, I do get paid for this. And yes, I do use Nelson textbooks in two of my classes But I do not get a single cent in kickbacks for choosing their textbooks. And the textbooks I've chosen are not Canadian ones, so my consulting work has nothing to do with those books.)

In between furlough days, I also started planning out a research study. Because of my good relationship with Nelson, I got a very nice offer from them. Susan, the publisher's local representative (we call 'em "book reps"), and her manager offered to give students in my perception course free access to their integrated online etextbook/study guide/online lab called CengageNOW. It's a bit surprising to hear about publishers giving anything away...it seems to good to be true. There is one catch: they're not doing this to show that they really are a sweet, generous, kitten-loving company; they're doing this to get some direct feedback from students on CengageNOW. And there's another catch: they only had 60 free access codes to give away. We decided that the fairest way to distribute them in my class of 214 students was random selection. Those are pretty good odds: a 28% chance of getting a free textbook. You're welcome! Oh, and my research part? I don't want to say too much, so let me just say that I want to see how well students do. That's all for now.

I also let myself have a little fun. A favourite activity that my girls like to do is bake pretzels from scratch. Because they're only 6 and 2 years old, that means I have to do most of the work, while they just try to roll out a few balls of dough and sprinkle way too much salt on them. But that's OK--I don't mind eating these misshapen, overly salted pretzels because they're made with love (sniff). Um, also, the pretzels taste really good after you've been outside shoveling a mountain of snow for (I kid you not) three hours. Oh, I had help--from a 6-year-old and a 2-year-old armed with a princess snow shovel and a Dora snow shovel, respectively. (As you can see from the picture above, I'm rockin' the snowblower while my crew takes yet another coffee break.)

Please tell me your holiday was more exciting than this. Please!

Why aren't you studying?

The Furlough Days

Today is the first of a bunch of "furlough days" at the UofA. So I will be trying really hard to not do any work, to show solidarity with my colleagues. Um, even though I already checked (and answered) email. And then I, er, started writing this blog entry. But from this point on, I will not be doing any course prep for next term, or answering email, or even--

Wait, scratch that. This isn't about having "days off" without pay. I can't just just take six days off and do no prep work at all--there's no way I'd be ready for the first day of class. It's about taking a pay cut, to help bail the University out of the great big, deep hole the administration dug, getting caught with their pants down with bad investments in the Great Recession. (Sorry about the mixed metaphor. Meh, I don't care--who's reading this anyway? Besides my mom.)

Anyway, I'll just pretend that I'm not doing any work. If you don't get an answer to your email, here are some classic posts that answer some FAQs:

  • Need some help getting into my classes, which are all currently full? I've got an answer for you.
  • Do you want to get your final grade "bumped up"? Just read this post from last year.
Why aren't you studying? (Like, for next term...)

The Eggnog Latte

Ah, a nonfat eggnog latte. It's a small indulgence that I allow myself every December after classes end. The caffeine keeps me going, because my kids are still conspiring to deprive me of sleep. The latte also helps get me into the holiday frame of mind--which is difficult, because I'm surrounded by so many term papers.

Don't get me wrong, I like these term papers. They're pretty interesting. But marking them takes up a huge lot of time. I do my best to prepare as much as I can beforehand: final exams ready to go, syllabi for next term completed, Christmas shopping done. I don't check my personal email, I don't read the newspaper, I don't watch TV. About the only thing that gets in my way is: snow. It does have to be shoveled. (Of course, it's snowed now for 5 days in a row, grr!)

What I'm trying to avoid is procrastination. I remember being a student and, after classes ended, getting the sudden and overwhelming urge to arrange all my CDs in alphabetical order. It's got to be done, right? Might as well do it now. Psychology Today has a list of 10 things to know about procrastination that you might want to check out--after you're done studying.

Why aren't you studying?
(I know, I know, you're going to ask "why aren't you marking?" Right? Someone else beat you to it. I beat you to it, too. It is OK to take a break--just as long as it doesn't last all day.)

The Copyright

You've heard about the expiration of the UofA's Access Copyright licence, right? It's been on ExpressNews and The Gateway, ya know. OK, here's the upshot, in bullet points (just like in class!)

There are a couple of important implications of all this. One affects coursepacks. As an instructor, I am not allowed to create a coursepack unless the UofA has a licence. (I could track down each copyright holder and negotiate with each directly, but, yeah, that's not going to happen. It's enough hassle to fill out the Access Copyright Log every term.) In January, I teach two classes that don't have textbooks--just coursepacks. So how will that work? In a clever bit of trickery, the coursepacks will actually be printed/published/assembled in 2010, so they are covered under the about-to-expire licence. Ha-HA! Take that, Access Copyright.

The second change is that required textbooks will not be available on reserve. A couple of key words in that last sentence are "required" and "reserve." Any recommended textbooks can remain on reserve. But if a textbook is required, it has to go on the regular shelves (or it will be sent back to whomever lent it to the library in the first place), so you can't take it out for an hour at a time to photocopy it, you naughty students! You can, however, er, take it out for three weeks and photocopy it. I mean, read it. In a typical 14-week class, though, only five students would be able to borrow the book. If students are fast "readers," more students could have access to the book, but even if it takes one day to "read" the book, only about 100 students could take it out. And my class is bigger than that.

Due to popular demand, I've put copies of textbooks for my courses on reserve. This term, it's been especially important for my perception course (PSYCO 267). I've assigned "Virtual Labs" that run off a CD-ROM that comes with every copy of the textbook. But if you bought the book used, it is probably missing the CD--and you can't buy the CD by itself. (You could buy the eTextbook which has access to the labs online, but that still costs about $70.)

In a stroke of good luck, however, I am onto a special opportunity provided by Nelson Education, the Canadian publisher of that book. An opportunity that could potentially save 60 PSYCO 267 students in my class next term quite a bit of money. And I might get the opportunity to do a study I've wanted to do for a long time, but on a much bigger scale than I ever hoped. But, I've probably said too much already...

Why aren't you studying?

The Gay Bisanz Memorial Turkey Drive

Dr Gay Bisanz taught me developmental psychology as an undergraduate. It wasn't a required course, but I took it anyway--I wanted to take as many psych courses as I could. It was a good choice. Sure, she taught me specific things about how people grow and develop, and general general things about science, psychology, critical thinking--but she also showed me the importance of giving back to your community. Sadly, Gay Bisanz died of cancer on June 1, 2005.

Gay started the Department of Psychology's now-famous Turkey Drive. People in the Department--academic staff, support staff, post-docs, students, and more--give money that is directed to CBC Edmonton's Turkey Drive for the Food Bank. But beyond that, some people bake cookies for sale, make jewelry, and donate items for raffle--there are a lot of very creative ways to part people with their money. Some instructors have volunteered to catch a pie with their face if their class contributes more money than any other class.

Last year, a total of $6,531.91 was collected. This year, the Turkey Drive goes from November 24 to December 8. Stop by the Psychology General Office (BS P-217) and buy a cookie, or a raffle ticket for one of the really nice gift baskets up for grabs. (The 2-for-$1 white chocolate and macadamia nut cookies are my personal weakness.)

Why aren't you studying?

The Exam Statistics: The Q Score

This is my final post on the topic of exam statistics. Previously, I described my use of the mean, difficulty scores, and point-biserial correlation. This time: the dreaded Q score. (Just to clarify: I'm not describing the other "Q Score", which represents the public's familiarity with--and appeal of--a person, product, company, or television show. That's not dreaded at all.)

The dreaded Q score is not a statistic that I regularly receive with all my other exam stats. I have to put in a special request. It's extra work for the people over at TS&QS, which means there's an additional cost that must be paid by the Department of Psychology. TS&QS has to go into their database of exam results for my class and perform a statistical comparison between two (or more) given exams.

Here's where the dread comes in: Why would I want to statistically compare two (or more) students' exams? If I suspect them of cheating, that's why. Sometimes, the cheating is blatantly obvious. The cameras in the classrooms (you know about those, right?) may clearly show one person peering over at the exam of another. Other times, it's not so obvious. Why is that guy jittering in his seat, looking everywhere except at his exam? Maybe he's nervous, or has exam anxiety. Why is that girl acting squirrelly, shifting her eyes back and forth? Maybe she drank too much coffee, has caffeine overload, and now really, really has to pee. Whatever the case, the exam proctors will not interrupt any student taking the exam. Nope. We'll just let you do what you do. If that's cheating, so be it.

However, at the end of the exam, the answer sheets from any suspicious students are set aside. (Think you can fool us by not leaving at the same time, or handing in your exams to different proctors? Tsk. You don't know how many eyes are watching, do you?) Those answer sheets will be analyzed, and I will get the dreaded Q score. I don't want to say too much about how it works, so suffice it to say that it gives a probability that cheating has occurred, compared to chance. Maybe I'll write more about how I have to deal with cheating in another post, but for now, let's just say it involves a lot of dread.

Although I have caught several cheaters over the years, I'm glad I've never had to deal with anything like what happened in Professor Richard Quinn's class recently. Yeesh.

(Cartoon by Frank Cammuso. It's important to give credit where credit is due. Otherwise, it's like, um...cheating.)

Why aren't you studying?

The Mouse

That's right, a mouse. I've got a mouse in my office. Well, it's probably not here full-time, but it does drop in and visit. Speaking of dropping, that's what's in the photo: droppings. See the two little black sesame-seed looking things? Mouse poop. On my desk. (Describing it with a food metaphor is kinda making me queasy. Bleah.)

A couple of weeks ago, there was a knock at my door. A couple of jolly fellows were putting mouse traps in everyone's offices in the Psychology wing. I told them I hadn't seen any mice, but they were quick to point out two little black grains of rice (bleah) on the floor. They didn't clean up the poop.

At this point, the lightbulb went off in my head. Oh, yeah. The chocolate bar that I left on my side table the other day. I came in to find it half eaten. I wasn't pleased as I threw the remainder away (Swiss dark chocolate!)--I figured the cleaning staff had seen it an gotten a bit hungry. Nope. Those must've been mouse teeth marks.

OK, so now: mouse trap. The problem is that it hasn't been working. I come into my office in the morning and regularly find more poops. On my desk. Of course you know, this means war! I don't want to get hantavirus. So yesterday I went out and got a couple of better mousetraps, put some cheese in them (this is what cartoons have taught me: mice love cheese), turned out the lights and left for the day. Heh-heh-heh, I laughed menacingly.

Today I opened the door to my office hesitantly. What would I find? Answer: nothing. OK, not exactly nothing. No mouse. No cheese. Yup, the l'il sucker ripped off my cheese. But at least the mouse traps were still there. This means I'm now helpfully feeding the mouse that's running around on the second floor. Dr Snyder, whose office is just down the hall, recently saw it looking at him from his bookshelf, but he wasn't able to catch it. In my office, however, the mouse prefers my desk. Evidence? Another poop. Probably left right after polishing off those two bits of cheese. Can a mouse be impertinent?

I suppose there's some joke in here somewhere about a psychologist and a mouse, but I'm drawing a blank. Do you know any good ones?

Why aren't you studying?

The Exam Statistics: The Point Biserial Correlation

I'm continuing my explanation of the reams of statistics I get about multiple choice exams. Last time, I explained exam item difficulty scores. (Fascinating, no?) This time: point biserial correlation coefficient, or "rpb". That is, "r" for the correlation coefficient (why, oh why is it the letter r?) and "pb" to specify that it's the point biserial and not some other kind of correlation. Like, um, some other kind.

If I've constructed a good exam item, it should be neither too hard nor too easy. It should also differentiate among students. But I can't tell how well it does that just by looking at the difficulty score. Instead, there's a more complex measure, the rpb. In general, I need a correlation index for a categorical variable with a continuous variable. More specifically, I want to correlate the categorical variable of a test item (i.e., whether a student answered the test item correctly or incorrectly), with the continuous variable of the student's percent score on the examination. Got that? I didn't think so.

Let me try again. Student A did well on the exam, getting 90% correct. Student B did not do so well, getting only 50%. If I look at any given exam question, in general, student A should be more likely to answer it correctly than student B. This is not the same as difficulty, because I'm not simply looking at what proportion of the class answered the question correctly. I'm correlating each student's score with their performance on each question. The key to all this is the word "should" in the sentence above.

If an exam item is poorly constructed for whatever reason, good students may do worse on it than students who did worse on the exam. That is, the better you are overall, the less likely you are to answer it correctly. That is not supposed to happen. The rpb gives me this information for each question on the exam. Experts in exam construction recommend that the rpb should range from 0.30 to 1.00. Anything question getting a rpb lower than 0.30 means that I will take a look at it and try to figure out why that's happening.

And if the rpb is negative, well...it's a negative correlation. That's the worst case I described: better students are doing worse answering this question; and poorer students are doing well. I won't use any questions getting a negative rpb again unless I can figure out why it's happening. Maybe I can tweak the question, maybe I have to rewrite it to ask about the same knowledge in a different way. Or maybe I'll just give up entirely, go and get a coffee, and check out some LOLcats.

Why aren't you studying?

The Exam Statistics: The Difficulty

In my last post, I discussed how I analyze the mean in my (multiple choice) exams. This time, I'm going to look at difficulty. This is not directly related to the mean. Huh? Isn't it the case that, the more difficult the exam, the lower the mean? Well, yes. But that's not the "difficulty" I'm writing about.

Among all the pages and pages of results I get from Test Scoring & Questionnaire Services is the "DIF" score or difficulty of each question. It's actually the proportion of the class who answered that question correctly. DIF=1.000 means that everyone got it right, but DIF=0.250 means that only 25% of the class did. But it's not really "difficulty," is it? If a question is really difficult, fewer people will answer it correctly and the number should decrease. So, really, it shouldn't be called difficulty, it should be called easiness. But, look, it's just called "difficulty," OK?

You might be thinking that I want everyone to answer every question correctly, right? Um, sorry to rain on your ice cream, but...no. It's really, really unlikely that everyone was able to learn absolutely everything in the course, and was also able to remember and apply that knowledge on an exam perfectly correctly for every question. What an exam should do is assess each student's learning of the material, and provide some way of differentiating among all students. If all questions are answered correctly, the exam itself has failed.

I went to a seminar last year at which a renowned expert in testing and exam question construction gave a talk. After it was over, I talked to him about DIF scores--specifically, what should they be? The general rule is that an exam question is doing a good job of differentiating among students if it's at least 0.300. That is, at least 30% of the class should be getting each question correct. There is no guideline for the upper end, but at another seminar, I heard an instructor say that she liked to put at least one DIF=1.000 question on each exam as a confidence booster. Yup, a gimme. I thought that was a pretty nice thing to do, so I try to include at least one high DIF question on every one of my exams, too.

So difficulty is related to the mean in that, the higher the DIF, the higher the mean on the exam overall. The mean is good for evaluating the overall performance of the class. But I also need to evaluate the questions on my exams, so I get the DIF score for each one. If the DIF is too low, the question either gets killed (*snff*), or rewritten to clarify it. Oh, and if I ever get DIF=0.000, it means I've keyed in an incorrect answer. Ooops.

Why aren't you studying?

The Exam Statistics: The Mean

With the first round of (multiple-choice) midterms over, I'm now swimming in data. I want to tell you about some of the stats I go through to assess and improve my exams. Unfortunately, I'm too late to celebrate (the first) World Statistics Day. But I don't feel too bad. At least statistics has a day. It's not like there's a "Psychology Month" or anything. Oh, look--yes there is. And I'm late for that, too. Moving on...

This installment is about the (arithmetic) mean, or, if you insist, the "average." I post the class mean of every exam because you demanded it! Really, though--what use is it to you? For classes that don't grade on the curve, you don't need to know the mean (or standard deviation) to determine your absolute standing in the class. Just take your percentage correct, and see what grade that corresponds to in the syllabus. Right?

Yes, that's important. But don't you want to know how everyone else did, too? Sure you do. "Did everyone think that exam was a killer, or just me?" We want to compare ourselves to other people. Some students even want to know what the top score was. "Did anyone get 100%?" "Am I the best in the class?"

The mean also serves another purpose, when there are multiple forms of an exam. In larger classes, multiple forms of an exam are used to discourage cheating (or at least, to make it more difficult). Typically, there is one form that has the questions arranged in order of topics (e.g., questions based on the first lecture and textbook chapter first, followed by questions on the second lecture and chapter, etc.). The other forms will have the questions in a random order. Are students who get the scrambled forms at a disadvantage? Or, put another way, is there a benefit to answering questions in a sequence that reflects the arrangement of the learning materials? If so, that wouldn't be fair, would it?

The data from every exam includes the means from each form. They are usually a little bit different. But is that difference a fluke, or is it due to the ordering of questions? Hmm, sound like a job for...statistics! The data also includes the results of an ANOVA (analysis of variance) that compares the means to each other. That is, are any differences statistically significant? The answer: No. I've never had a difference at p < 0.01 or even p < 0.05. That means any differences are small; they are due to chance.

The bottom line: It doesn't matter which form you get. Isn't science cool?

Why aren't you studying?

The Coffee

When I went to high school, it wasn't cool to drink coffee. Coffee was dark, scary, and bitter. Sure, my family would have Kaffeezeit ("coffee time") on the weekends, but I was just in it for the Kuchen.

When I started university, I drank a cup of tea with milk and sugar every single morning. Even though I had a lot of 8:00 classes (because they were good classes only offered at that time, that's why), one cup of English breakfast tea was all the caffeine I needed. Some of my friends became desperate around exam time, and dipped into the go-juice. It was hilarious to watch as the normally non-caffeine consumers' eyes got really big after having a big cuppa joe. Then, they'd study like the dickens. This proved that coffee was a dangerous, dangerous substance.

Then, I started graduate school. Sure, getting a graduate degree is pretty demanding. Maybe I'd have an extra cup of tea once in a while. But the sheer, stark terror of almost having to go into the Real World was enough stimulus for me--no coffee, thanks. Maybe just a bit more sugar.

Then, one term I was Dr. Dawson's teaching assistant. He made me come to the class (I dunno, to learn something I guess); because he was on my supervisory committee, it's not like I could say no or anything. The first day of class, I met him at his office and we went to the class together. But not before he poured himself a cup of freshly made French-pressed coffee. And then he insisted on giving me a cup. It's not like I could say no or anything. The worst thing was that the coffee was: black. As black as night. No milk, and certainly no sugar (isn't that freebasing?). After that, I brought my own mug and poured sugar into it first--milk was too conspicuous.

Now, I'm neither a connoisseur (yes, I drink instant--please forgive me), nor a coffee-hound. Usually, I get by with only one cup of coffee. But if I've had a bad sleep (which does happen), you'll see me toting a cup from one of the fine local purveyors. Not my usual mug-o-water; not tea; not vodka. I have gone over to the dark side. With lots of milk and sugar.

Why aren't you studying?

The Comics

Hey, who doesn't love comics? Not me! No, I don't not love comics. Um. Here are some of my favourite web comics.

PhD Comics is about grad.students who seem to have a problem finishing their theses. (Want to know how to drive a grad.student insane? Ask her if she's finished her thesis yet. Hee!) Even if you're not a grad.student, it's still pretty funny, poking fun at all kinds of academic matters. (This one made me LOL.) There are 3 comics per week (Mon/Wed/Fri).

xkcd isn't an abbreviation or acronym--it's just the title of a webcomic, aimed at people who have the ability to think. This rules out a lot of people who just won't "get it." Some gags require knowledge of science. Gasp! Conveniently comes out 2 times per week (Tues/Thurs).

Lab Bratz isn't just for lab ratz (er, rats). Has gags on academia, but doesn't require a degree to get the joke. Only 1 per month.

Do you have any favourite web comics? (To A.K.: Yes, I know about Salad Fingers, which is technically a cartoon. And yes, I read your blog!)

Why aren't you studying?

Find It