Michael Huemer, assistant professor of philosophy at the Univ. of Colorado, Boulder, critically reviews student evaluations of faculty. (Link via Marginal Revolution.) He concludes they are pretty rotten and have some serious side effects but that they are used because 1) they are cheap, 2) they seem objective, and 3) there are no good alternatives. (See also this new paper that concludes, ". . . these results suggest that about half of the variation in student opinion survey scores used by universities for promotion, tenure, and teaching award decisions may be due to the easiness of the course and the sexiness of the professor." Link via SCSU Scholars.)

I agree there are no good alteratives that are also cheap, but there are alternatives. Here are three:

1. Adaptation of a technique used by some businesses, the "secret shopper." Faculty are told that there is a small chance that in each of their classes one student is a secret shopper. A shopper student is working for the provost or perhaps the college dean. He takes the entire course, does the assignments, and takes all the tests. But the administrator removes the grade from the student's transcript after the student hands in a detailed written evaluation of the class. The student's evaluation of the course should then be much less affected by the professor's grade. (But the student's grade could be compared to the class average to establish that the student actually tried.)

2. Adaptation of how professors' research is evaluated, the referee report. Videotapes and course materials are sent to faculty at other universities--but in the professor's field--for review.

3. Faculty members from the same university--but outside the professors' department--review videotapes and course materials.

Each of these has some weaknesses. But I'd guarantee that someone who scores highly on all three is actually an outstanding teacher and someone who scores low on all three is not. That's something the current system does not guarantee.

But it would be expensive and difficult to do any one of these, let alone all three, so I also guarantee that universities will continue to use the current system, with little or no change.

Even though it reminds one of the old joke about the drunk who loses his wallet in the bushes but who looks it for under the streetlight a block away.