Recall (see here and here for previous postings) that to argue that grade inflation has occurred, it is not sufficient to simply show that grades have risen. It must be shown grades have risen without a corresponding increase in learning and student achievement. And that is difficult to do because there are really no good independent measures of student learning, apart from grades.

Some have argued that the SAT scores of matriculating classes could be used as a measure of student ‘ability’ and could thus be used to see if universities are getting ‘better’ students, thus justifying the rise in grades.

But the use of SAT scores as a measure of student quality or abilities has always been deeply problematic, so it is not even clear that any rise in SAT scores of incoming students means anything. One reason is that the students who take the SAT tests are a self-selected group and not a random sample, so one cannot infer much from changes in SAT scores. Second, SAT scores have not been shown to be predictive of anything really useful. There is a mild correlation of SAT scores with first year college grades but that is about it.

Even at Case, not all matriculating students have taken the SAT’s. Also the average total SAT scores from 1985-1992 was 1271, while the average from 1993-2005 was 1321. This rise in SAT scores of incoming students at Case would be affected by two factors, the first being the re-centering of SAT scores that occurred in 1995. It is not known whether the pre-1995 scores we have at Case are the original ones or have been raised to adjust for re-centering. This lack of knowledge makes it hard to draw conclusions about how much, if at all, SAT scores have risen at Case.

Alfie Kohn cites “Trends in College Admissions” reports that say that the average verbal-SAT score of students enrolled in all private colleges rose from 543 in 1985 to 558 in 1999. It is also the fact that it was around 1991 that Case instituted merit scholarships based on SAT scores and started aggressively marketing it as a recruiting tool. So it is tempting to argue that there has been a genuine rise in SAT scores for students at Case.

Another local factor at Case that would influence GPAs is the practice of “freshman forgiveness” that began in 1987. Under this program, students in their first year would be “forgiven” any F grades they received and this F would not be counted towards their GPA. This is bound to have the effect of increasing the overall GPA, although a very rough estimate suggests only a 1-2% increase. This practice was terminated in 2005.

The Rosovsky-Hartley monograph points to the fact that many more students in colleges are now enrolled in remedial courses than was the case in the past, arguing that this implies that students are actually worse now. But again, that inference is not clear. Over the recent past there has been a definite shift in emphasis in colleges of now wanting to retain the students they recruit. The old model of colleges recruiting more students than they needed and then ‘weeding’ them out using certain courses in their first year, is no longer in vogue, assuming that there was substance to that belief and it is not just folklore.

Now universities go to great lengths to provide assistance to their students, beefing up their advising, tutoring, and other programs to help student stay in school. So the increased enrollment of students in remedial courses may simply be the consequence of universities taking a much more proactive attitude to helping students, rather than a sign of declining student quality. All these measures are aimed at improving student performance and are another possible benign explanation for any rise in grades. In fact, all these remedial and assistance programs could be used to argue that a rise in grades could be due to actual improved student performance.

Alfie Kohn argues that taking all these things into account, there is no evidence for grade inflation, that this is an issue that has been blown way out of proportion by those who have a very narrow concept of the role of grades in learning. Kohn says there are many reasons why grades could rise:

Maybe students are turning in better assignments. Maybe instructors used to be too stingy with their marks and have become more reasonable. Maybe the concept of assessment itself has evolved, so that today it is more a means for allowing students to demonstrate what they know rather than for sorting them or “catching them out.” (The real question, then, is why we spent so many years trying to make good students look bad.) Maybe students aren’t forced to take as many courses outside their primary areas of interest in which they didn’t fare as well. Maybe struggling students are now able to withdraw from a course before a poor grade appears on their transcripts. (Say what you will about that practice, it challenges the hypothesis that the grades students receive in the courses they complete are inflated.)

The bottom line: No one has ever demonstrated that students today get A’s for the same work that used to receive B’s or C’s. We simply do not have the data to support such a claim.

In addition to the factors listed by Kohn, psychologist Steve Falkenberg points out a number of other reasons why average grades could rise. His essay is a particularly thoughtful one that is worth reading.

Part of the problem in judging whether grade inflation exists is that we don’t know what the actual grade distribution in colleges should be. Those who argue that it should be a bell curve (or ‘normal’ distribution) with an average around C are mixing up a normative approach to assessment (as is used for IQ tests and SATs) with an achievement approach.

IQ tests and SATs are designed so that the results are spread out over a bell curve. They seek to measure a characteristic (called “intelligence'”) that is supposedly distributed randomly in the population according to a normal distribution. (This assumption and the whole issue of what constitutes intelligence is the source of a huge controversy that I don’t want to get into here.) So the goal of such tests is to sort students into a hierarchy, and they design tests that spread out the scores so that one can tell who is in the top 10% and so on.

But when you teach a class of students, you are no longer dealing with a random sample of the population. First of all, you are not giving your assessments to people off the street. The students have been selected based on their prior achievements and are no longer a random sampling of the population. Secondly, by teaching them, you are deliberately intervening and skewing the distribution. Thirdly, your tests should not be measuring the same random variable that things like the SATs measure. If they were, you might as well give your students their grades based on those tests.

Tests should not be measures of some intrinsic ability, even assuming that such a thing exists and can be measured and a number assigned to it. Tests are (or at least should be) measuring achievement of how much and how well a selected group of students have learned as a result of your instruction. Hence there is no reason at all to expect a normal distribution. In fact, you would expect to have a distribution that is skewed towards the high end. The problem, if it can be considered a problem, is that we don’t know a priori what that skewed distribution should look like or whether there is a preferred distribution at all. After all, there is nothing intrinsically wrong with everyone in a class getting As, if they have all learned the material at a suitably high level.

In fact, as Ohmer Milton, Howard Pollio, and James Eison write in Making Sense of College Grades (Jossey-Bass, 1986): “It is not a symbol of rigor to have grades fall into a ‘normal’ distribution; rather, it is a symbol of failure — failure to teach well, failure to test well, and failure to have any influence at all on the intellectual lives of students.”

There is nothing intrinsically noble about trying to keep average grades unchanged over the years, which is what those who complain about grade inflation usually want to do.

On the other hand, one could make the reasonable case that as we get better at teaching and in creating the conditions that make students learn better, and as a consequence we get students who are able to learn more, then perhaps we should raise our expectations of students and provide more challenging assignments, so that they can rise to greater heights. This is a completely different discussion. If we do so, this might result in a drop in grades. But this drop is a byproduct of a thoughtful decision to make learning better, not caused by an arbitrary decision to keep average grades fixed.

This approach would be like car manufacturers and consumers raising their standards over the years so that we now expect a lot more from our cars than we did fifty years ago. Even the best cars of fifty years ago would not be able to meet the current standards of fuel efficiency, safety, and emissions. But the important thing to keep in mind is that standards have been raised along with the ability to make better cars able to meet the higher standards.

But in order to take this approach in education, it requires teachers to think carefully about what and how we assess, what we can reasonably expect of our students, and how we should teach so they can learn more and learn better. Unfortunately much of the discussion of grade inflation short-circuits this worthwhile aspect of the issue, choosing instead to go for the quick fix like putting limits for the number of grades awarded in each category.

It is perhaps worthwhile to remember that fears about grade inflation, that high grades are being given for poor quality work, have been around for a long time, especially at elite institutions. The Report of the Committee on Raising the Standard at Harvard University said: “Grades A and B are sometimes given too readily — Grade A for work of no very high merit, and Grade B for work not far above mediocrity. … One of the chief obstacles to raising the standards of the degree is the readiness with which insincere students gain passable grades by sham work.”

That statement was made in 1894.

POST SCRIPT: Cindy Sheehan in Cleveland tomorrow

Cindy Sheehan will speak at a Cleveland Town Hall Meeting Saturday, March 25, 1-3 pm

Progressive Democrats of Ohio present Gold Star Mother and PDA Board Member Cindy Sheehan at a Town Hall Meeting on Saturday, March 25, 2006 from 1 – 3 p.m. at the Beachland Ballroom, 15711 Waterloo Road in Cleveland’s North Collinwood neighborhood. (directions.)

Topic: Examining The Cost of Iraq: Lives, Jobs, Security, Community

Panelists include:

US Congressman Dennis Kucinich, OH-10
Cindy Sheehan – Gold Star mother & activist
Tim Carpenter, National Director, Progressive Democrats of America
Francis Chiappa, President, Cleveland Peace Action
Paul Schroeder, NE Ohio Gold Star Father and co-founder of Families of the Fallen For Change
Farhad Sethna, Immigration attorney and concerned citizen

Before jumping to the conclusion that a rise in average grades must imply inflation (see my previous posting on this topic), we should be aware of the dangers that exist when we are dealing with averages. For example, suppose we consider a hypothetical institution that has just two departments A and B. Historically, students taking courses in A have had average grades of 2.5 while those in B have had 3.0. Even if there is no change at all in the abilities or effort of the students and no change in what the faculty teach or the way that faculty assess and grade, so that the average grades in each department remain unchanged, it is still possible for the average grades of the institution to rise, simply because the fraction of students taking courses in B has become larger.

There is evidence that this shifting around in the courses taken by students is just what is happening. Those who are convinced that grade inflation exists and that it is evil, tend to interpret this phenomenon as game playing by students, that they are manipulating the system, choosing courses on the basis of how easy it is to get high grades rather than by interest or challenge.

For example, the ERIC report says “In Grade Inflation: A Crisis in College Education (2003), professor Valen E. Johnson concludes that disparities in grading affect the way students complete course evaluation forms and result in inequitable faculty evaluations. . . Students are currently able to manipulate their grade point averages through the judicious choice of their classes rather than through moderating their efforts. Academic standards have been diminished and this diminution can be halted, he argues, only if more principled student grading practices are adopted and if faculty evaluations become more closely linked to student achievement.”

This looks bad and the author obviously wants to make it look bad, as can be seen from his choice of the word ‘manipulate’ to describe the students’ actions and the way he implies that faculty are becoming more unprincipled in their grading practices. But there is no evidence for the evil motivations attributed to such students and faculty. In fact, one can look at the phenomenon in a different way. It is undoubtedly true that students now have many more choices than they did in the past. There are more majors and more electives. When you offer more choices, students are more likely to choose courses they are interested in and thus are more likely to do better in them.

Furthermore, even if students are choosing courses partly based on their expectation of the grade they will receive in it, we should not be too harsh in our judgments. After all, we have created a system in which grades seem to be the basis for almost everything: admission to colleges and graduate schools, honors, scholarships, and financial aid. As I said, grades have become the currency of higher education. Is it any wonder that students factor in grades when making their choices? If a student tries to balance courses they really want to take with those that know they can get a high grade in order to be able to maintain the GPA they need to retain their scholarships, why is this to be condemned? This seems to me to be a sensible strategy. After all, faculty do that kind of thing all the time. When faculty learn that the NIH or NSF is shifting its grants funding emphasis to some new research area, many will shift their research programs accordingly. We do not pour scorn on them for this, telling them that they should choose research topics purely based on their interests. Instead, we commend them for being forward thinking.

It certainly would be wonderful if students chose courses purely on the basis of their interest or usefulness or challenge and not on grade expectations, but to put students in the current grade-focused environment and expect them to ignore grades altogether when making their course selection is to be hypocritical and send mixed messages.

What about the idea that faculty grading standards have declined and that part of the reason is that they are giving easy grades in order to get good evaluations? This is a very popular piece of folklore on college campuses. But this question has also been studied and the data simply do not support it. It does seem to be true that students tend to get higher grades in the courses in the courses they rate higher. But to infer a causal relationship, that if a faculty member gives higher grades they will get better evaluations, is wrong.

People who have studied this find that if a student likes a course and a professor (and thus gives good evaluations), then they will tend to work harder at that course and do better (and thus get higher grades) thus bringing about the grades-evaluations correlation that we see. But what tends to determine how much a student likes a course and professor seems to depend on whether they student feels like she or he is actually leaning interesting and useful stuff. Students, like anybody else, don’t like to feel they are wasting their time and money and do not enjoy being with a professor who does not care for them or respect them.

Remember that these studies report on general trends. It is quite likely that there exist individual professors who give high grades in a misguided attempt to bribe student to give good evaluations, and that there exist students willing to be so bribed. But such people are not the norm.

To be continued. . .

POST SCRIPT: And the winner is. . .

Meanwhile, there are Americans who have already have decided which country the US should invade next in the global war on terror, even if they haven’t the faintest idea where that country is on the globe or why it should be invaded. Even Sri Lanka gets a shot at this particularly dubious honor.

Here’s an idea for a reality show, along the lines of American Idol. It will be called Who’s next?. The contestants will be the heads of states of each country and this time their goal will be to get voted off because the last remaining country gets bombed and invaded by the US. The judges could be Dick Cheney (to provide the sarcastic put-downs a la Simon Cowell. Sample: “You think we’re going to waste our smart bombs on your dumb country?”), Donald Rumsfeld, and Condoleeza Rice.

Fox television, my contract is ready and I’m waiting for your call.

There is nothing that gets the juices flowing in higher education academic circles than the topic of grade inflation. Part of the reason for this passion may be because grades and test scores, and not learning, seem to have become the currency of education, dominating the thinking of both students and faculty. Hence some people monitor grades as an important symptom of the health of universities.

But what is curious is that much of the discussion is done in the absence of any hard data. It seems as if perceptions or personal anecdotes are a sufficient basis to draw quite sweeping conclusions and prescriptions for action.

One of the interesting things about the discussion is how dismayed some faculty get simply by the prospect that average grades have risen over time. I do not quite understand this. Is education the only profession where evidence, at least on the surface, of a rise in quality is seen as a bad thing? I would hope that like in every other profession we teachers are getting better at what we do. I would hope that we now understand better the conditions under which students learn best and have incorporated the results of that knowledge into our classrooms, resulting in higher achievement by students. Any other profession or industry would welcome the news that fewer people are doing poorly or that fewer products are rejected for not meeting quality standards. But in higher education, rising grades are simply assumed to be bad.

Of course, if grades are rising because our assessment practices are becoming lax, then that is a cause for concern, just as if a manufacturer reduces the rejection rate of their product by lowering quality standards. This is why having an independent measure of student learning and achievement to compare grades with has to be an important part of the discussion.

Grade inflation is a concept that has an analogy with monetary inflation, and to infer that inflation (as opposed to just a rise) in grades has occurred implies that grades have risen without a corresponding increase in learning and student achievement. But in much of the discussion, this important conditional clause is dropped and a rise in grades is taken as sufficient evidence by itself that inflation has occurred.

Let’s take first the question of whether average grades have actually risen. At Case, as some of you may know, beginning January 2003, the GPA cutoffs to achieve honors were raised to 3.56 (cum laude), 3.75 (magna cum laude), and 3.88 (summa cum laude) so that only 35% of students would be eligible for honors. (The earlier values were 3.20, 3.50, and 3.80 respectively.) This measure was taken because the number of people who were graduating with honors had risen steadily over the years, well above the 35% originally anticipated when the earlier bars were set.

A look at grade point averages at Case shows that it was 2.99 in 1975 (the first year for which we have this data), dropped slowly and steadily to 2.70 in 1982, rose to 3.02 in 1987, stayed around that value until 1997, and since then has oscillated around 3.20 until 2005, with the highest reaching 3.27 in 2001. The overall average for the entire period was 3.01 and the standard deviation was about 0.70. (I am grateful for this and other Case data to Dr. Julie Petek, Director of Degree Audit and Data Services.)

It is hard to know what to make of this. On the one hand we could start at the lowest point in the grade curve and say that grades have risen by half a letter grade from 1982 to 2005. Or we could start at 1975 and say that grades are fluctuating in the range 2.70-3.30, or about half a standard deviation about the mean of 3.0.

What does the data say nationwide? Henry Rosovsky and Matthew Hartley, writing in a monograph for the American Academy of Arts and Sciences are convinced that inflation has occurred. For evidence of grades increasing, they point to various nationwide surveys that show that average grades rose by 0.43 from 1960 to 1974; that in 1993 the number of grades of A- or higher was 26%, compared to 7% in 1969; and the number of C’s dropped from 25% to 9% in that same period; and that averages rose from 3.07 in the mid 1980s to 3.34 in the mid 1990s.

In this last result, it is interesting to note in another study that grades rose on average only at selective liberal arts colleges and research universities, while they declined at general liberal arts colleges and 
comprehensive colleges and universities, and in the humanities and social sciences

The Rosovsky-Hartley monograph has been criticized for depending on research that itself depended on studies that used surveys, and such studies can be questioned on the fact that it is not clear how reliable the responses to such surveys are, depending as they do on self-reporting.

A 2003 ERIC Digest of the literature on this topic finds results that that cast doubt on the basic question of whether average grades have even risen. For example, “Clifford Adelman, a senior research analyst with the U.S. Department of Education, reviewed student transcripts from more than 3,000 colleges and universities and reported in 1995 that student grades have actually declined slightly over the last 20 years.” (my emphasis). His study of 16.5 million graduates from 1999-2000 also found that 14.5% of these students received As while 33.5% received grades of C or lower.

What is significant about the Adelman study is that he used actual student transcripts, not surveys, and thus seems to me to be more reliable.

It seems from this data and other studies that average grades have not increased across the board, but it is plausible that they have increased at selective liberal arts colleges and research universities. The Rosovsky-Hartley monograph says that “In 1966, 22 percent of all grades given to Harvard undergraduates were in the A range. By 1996 that percentage had risen to 46 percent and in that same year 82 percent of Harvard seniors graduated with academic honors. In 1973, 30.7 percent of all grades at Princeton were in the A range and by 1997 that percentage had risen to 42.5 percent.”
Even though it has not been conclusively established suppose that, for the sake of the argument, we concede that at least at selective liberal arts colleges and research universities (such as Case) have seen a rise in average grades. Is this automatically evidence of grade inflation? Or are there more benign causes, such as that we getting better prepared and more able students now, or our teaching methods have improved? Another important issue is whether Case’s experience of rising grades is part of a national trend or an exception.

To be continued. . .

POST SCRIPT: Where’s the balance?

Over at Hullabaloo, Tristero catches the Washington Post in a blatant act of bias in favor of atheistic science. The Post article says:

Scientists said yesterday they have found the best evidence yet supporting the theory that about 13.7 billion years ago, the universe suddenly expanded from the size of a marble to the size of the cosmos in less than a trillionth of a second.

Tristero points out that the article completely fails to mention the controversy around this question, that there is another side to this story, that the big bang is “only a theory” since no one was there to actually observe this event and prove that it happened.

And not a word of balance from the other side, as if the sincere faith of millions of Americans in a Christian God didn’t matter at all to the Post’s editors.

I just hate it when the media reports carefully vetted scientific data as fact and not as just one of many valid points of view. I’m not asking for them to ignore the opinions of these so-called scientists, but they really should report the fact there’s a lot of controversy about whether this kind of evidence is valid. Like, were you there, huh, Mr. Hotshot Washington Post?