Jonathan Eisen has been ripping on RateMyProfessors lately with a hashtag, #BoycottRateMyProfessors. Good.
RMP is a terrible source for any kind of evaluation of professors — it’s more of a place where disgruntled students can vent, which is fine. I do pay attention to complaints, since they’re information I can use to improve, but they’re more useful when they’re on an evaluation form rather than on a website I don’t read.
But there are two big problems here. One is that Google algorithms take RMP seriously as a source for information on academics.
Send this message to @Google, which continues to use its algorithms to put RMP high on almost any search for an academic. The embedded racism and sexism of these evaluation systems must be acknowledged and challenged.
— Monica H Green (@monicaMedHist) June 28, 2018
But worst of all…the goddamn chili peppers. You’re supposed to rate your professors on their “hotness”. It’s flamingly sexist.
And before the sniggers come in about me being jealous because my students definitely do not rate me as “hot” — I’d be even more annoyed if they had put one of those stupid red peppers next to my name. Sorry, students, “eye candy” is not and never has been in the job description.
Here’s a really good summary of the problem from the person who started flaming those peppers.
Yes, one can glean useful information from the students’ comments, although the signal-to-noise ratio is poor. The site attracts lots of students who are unhappy and want to blame someone other than themselves, so there’s a lot of invective. The “hot pepper” nonsense is particularly irksome and irrelevant. Most students lack the filters necessary to dig out the occasional nuggets of helpful data, resulting in their being misled. And some items are almost certain to lead you astray, like the super-positive student comment about one teacher that “C students are getting A’s!!!” Yeah, I got some of that instructor’s students in a subsequent class. They had their A’s, but no knowledge.
My favourite rating that I received on there back when I was teaching was one that gave me all 1s, complained about my hair, and accused me of being the worst prof ever to teach [course that isn’t even part of my faculty, much less one I’d ever taught].
I was professor before RMP came online, but I retired and went back to school. I became the verifier for our school. Though many of the comments were harsh, most of them were true, of the faculty I knew. The peppers were unfortunate – some were given to people who were anything but ‘hot’ – I think to be ironic.
I saw a comment there once that said, “Do not take this professor, he does not grade on the curb.” Obviously left by someone who needs to be graded on a curve.
Is “Rate My Professors” ever used for administrative or job evaluation purposes?
PZ, “RateMyProfessor” is a joke. A fun joke, maybe, but still a joke. I’m nearly 70 years old and I get a “hotness” red chili pepper on this site, which my wife thinks is hilarious! And I get very good ratings – but these come from no more than a handful of students who happen to find this site and post their ratings. At my school, we do very systematic student surveys on courses and the performance of professors, and these can be useful if done well. You’ve still got to realize that student comments are biased in a very particular way with respect to how much they “like” a professor – but properly interpreted they can be useful. RateMyProfessor, on the other hand, is meaningless. Any institution that even looks at these rating as a method of evaluation is badly misguided. (Ken Miller, Brown Univ.)
Ditto, although I think there’s room for improvement because we use the same survey instrument for all courses in a one-size-fits-all process that sometimes misses significant differences between in-person, on-line, and hybrid courses. Also, the student surveys are treated as just one component of faculty review and not determinative. My colleagues and I find such feedback helpful and will continue to do so as long as college management doesn’t go nuts and start mining the student surveys for the most disgruntled remarks. Our peer review process helps to keep that under control. And RateMyProfessors.com is never referenced in formal reviews and serves mostly as fodder for hallway gossip and colleague-teasing, since some of the more egregious student comments are hilarious (as long as no one takes them seriously).
Huh. The peppers appear to have vanished rather suddenly. Either that, or no one in my department is rated “hot” anymore. I also noticed that the reference to “hotness” has been scrubbed from the Ratings Categories page, where it used to be cited under Overall Quality (with a note that it was excluded from the Overall Quality score). Is the pepper dead? Looks like it.
I just checked (5 minutes ago) RateMyProfessor.com for my undergrad school and didn’t see any peppers on any of the bio profs (men or women). Then again, I also found several ratings of a professor who left awkwardly in 2004, and students who rated the prof in 2007. I know he didn’t come back, and they weren’t super generic ratings, so I don’t know if these were old students (long graduated) retrospectively rating this guy, or some kind of weird trolling.
Clearly not useful at all.
The most damning indictment of RMP is that last I checked, my horrible Dynamics professor didn’t even have an entry.