Untangling the truth about test scores: Reign of Error part 4

Untangling the truth about test scores: Reign of Error part 4

Fourth in my series about Diane Ravitch's Reign of Error. Earlier installments can be read here (chapter 1), here (summary of chapters 2-4), and here (Chicago context of chapters 2-4).

We've all heard it.

Our schools are failing and our test scores are terrible. They're terrible nationally and they're terrible internationally. It's the teachers' faults, and maybe sort of the principals' faults too; for sure the unions are to blame, and definitely our terribly, terribly low expectations. Our kids are trapped in failure factories, and the system is protected by the terrible teachers' unions because teachers want less work and more pay, shorter days, shorter years, no accountability, and no assessments of their students whatsoever so that no one finds out how lazy and terrible they are.

That's the story. We've all heard it. It's told very loudly in Chicago by our mayor, by the CPS CEO, by the Chicago Tribune on an almost daily basis, by charter operators, and by many intelligent citizens of Chicagoland. We'll be hearing a lot more about it, too--now that Paul Vallas is back, that great wreaker of creative disruption and school system dismemberment. Paul Vallas, recently booted as the Bridgeport CT schools chief, is going to be Gov. Quinn's new running mate.

But I digress.

Diane Ravitch has a beef with this story of school failure and terrible test scores. She thinks it isn't true.

But how can you just think something isn't true--if there's all kinds of data that proves it? Aren't our test scores terrible? Aren't we failures internationally? Haven't we seen all about this in Waiting for Superman? Isn't this the greatest security risk our nation now faces?

Ravitch has marshalled a bunch of data to demonstrate that this story is in fact false. This is the burden of chapters 5, 6, and 7 of Reign of Error.

We've been hearing about poor test scores for so long, it's just a given in our collective national consciousness. So when someone says, "data shows that American students are learning more and testing better than they ever have," it's rather arresting. It sure doesn't seem right.

But education historian Ravitch knows what she's talking about.

The test scores under discussion are from the National Assessment of Educational Progress (NAEP), a non-high-stakes standardized test that's been randomly administered since the 1960s. Ravitch served on the independent governing board overseeing this assessment for 7 years.

There are 2 uses of these scores. Scale scores establish a trend line. Achievement scores are divided across four levels: "advanced" (A+ level of achievement), "proficient" (solid achievement, like an A through B+ range); "basic" (roughly equivalent to a B-C level); and "below basic" (corresponding to a D student).

A common misuse of this data by corporate ed reformers is to confuse achievement level with grade level, assuming that those who are not "proficient" are below grade level (47-48).  David Guggenheim, the director of Waiting for Superman, claimed using NAEP data that 70% of 8th graders could not read at grade level. In this statistic he included everyone below "proficient." NAEP, however, doesn't report grade level. "Actually," Ravitch corrects, "76 percent on NAEP are basic or above, and 24 percent are below basic." Guggenheim's claim is dramatic, but not anywhere close to accurate.

Ravitch spends several pages (49-54) presenting changes in NAEP scores since 1973 and demonstrates that scores in both reading and math have improved--in reading slowly and steadily, and in math dramatically. In fact, students in every racial and ethnic group have seen large gains.

To wit: for white 9 year olds, math test scores have gone up 25 points since 1973; Hispanic 9 year olds, 32 points; and black 9 year olds, 34 points. In reading, white 9 year olds have gained 14 points; Hispanic 9 year olds have gained 25 points; and black 9 year olds have gained 34 points. This is just one little slice of the data she offers in chapter 5--all of it encouraging.

Chapter 6 turns to the achievement gap that is narrowing, but remains, between white and minority students.  Scores for minority students have risen dramatically--of note is the fact that "black student achievement was higher in 2009 than white student achievement in 1990." (56) Yet in the same years white students have also been making gains, thereby maintaining the gap. The main issue, however, is summed up at the start of the chapter: achievement gaps "will remain large if we do nothing about the causes of the gaps." (55)

These gaps "begin long before kindergarten" (59) and have everything to do with income.  The income achievement gap is in fact far larger than the racial achievement gap, and has been growing for the last 25 years. (60) Schools alone cannot close gaps created by poverty. And, Ravitch adds, "So long as our society is indifferent to poverty, so long as we are willing to look the other way...there will always be achievement gaps." (62)

Concluding her analysis of testing and its significance in chapter 7, Ravitch looks at international test data. Despite all the hand-wringing of politicians who foresee our imminent doom as indicated by our test scores, Ravitch assures that our international test-score standing is nothing new. In fact, since the mid-60s when such testing began, U.S. performance has always looked sketchy.

U.S. scores have not declined, but remained constant. And if they are sorted for poverty, scores are actually in the upper ranks of all the nations who administer the tests. Data confirms that U.S. low-poverty schools perform among the best in the world; performance drops as poverty rises. (64-65)

But how much do these standings mean anyway?

Keith Baker, a former analyst with the Department of Education, examined international test data to find out how well high-scoring nations have performed economically in the last 50 years. After all, the "Chicken Littles" have been claiming national disaster since that time. (71) There was no correlation between high test success and national wealth. There was a correlation, however, between lower test scores and successful democracy.

The unique qualities of American public education are sought after by other nations--even Finland and yes, even China. Our schools "cultivate a certain 'spirit'......'ambition, inquisitiveness, independence, and perhaps most important, the absence of a fixation on testing and test scores.'" (71) (I can only assume he is speaking relatively here.) On measures of creativity, American students "clobbered the world," says Baker.

Chinese-born professor at the University of Oregon Yong Zhao worries that those unique aspects of American public education are at risk: "In our eagerness to copy nations with higher test scores, we may sacrifice the qualities of individualism and creativity that have been the source of our nation's economic, social, and technical success." China is trying to become an "innovation-driven knowledge society" and leave behind its past as a "labor-intensive, low-level manufacturing economy" which has relied on "schools that force students to memorize correct answers on standardized tests and [regurgitate] spoon-fed knowledge." (69)

"If China, a developing country aspiring to move into an innovative society, has been working to emulate U.S. education, why does America want to abandon it?" asks Zhao. (69)

Ravitch concludes chapter 6 with observations that underscore her entire book. "More testing does not make children smarter. More testing does not reduce achievement gaps. More testing does nothing to address poverty." (72) It does, however, undermine creativity and innovation, that vital spirit of inquiry, and intellectual freedom.

When we lose those things, my farmyard friends, the sky really will be falling.

If you want a better Chicago Public Schools system please like my Facebook page and join me there for more discussion. You can also follow me on twitter @foolforcps.

Sign up for your weekly dose of education from the fool for CPS by typing your email address in the box and clicking the "create subscription" button. My list is completely spam free, and you can opt out at any time.

Filed under: Uncategorized

Leave a comment