Testing Results & Grad Rates, Oh My!

It’s test score time, the annual ritual in which the state finally releases school by school test score results and everyone pays attention for a moment.  The scores don’t look great, on the whole — we already knew that — and they may not even mean much, reports Linda Lutton.  Oh, and newly accurate (lower) graduation rates, too.

Many third-graders fail key reading standard Tribune: At Chicago Public Schools’ Guggenheim Elementary, 85.3 percent of third-graders failed to meet state reading standards, the worst performance in the state. Yale Elementary, also a CPS school, came in second with 77.4 percent, followed by North .

Researchers: test scores released today mislead the public WBEZ:  Today is the day the public gets to see how its schools are doing. Illinois is releasing test scores for every school in the state. Increasingly, researchers say the way the state reports those scores is misleading. WBEZ’s Linda Lutton tell the story.

Top schools have longer days Sun Times:  The 10 highest-ranking suburban neighborhood elementary schools all have longer days for kids than the typical Chicago public school — but shorter ones than those advocated by Mayor Rahm Emanuel and city public school officials.

Year-round classes lift Lindblom Sun Times:  Chicago Public Schools gained two high schools among the state’s 100 top-scoring ones this year, bringing the system’s total to seven.

Chopin beats odds with classic formula Sun Times: Why did 100 percent of seventh- and eighth-graders at Chicago’s Chopin School pass their state reading and math tests last school year? Put that question to this year’s eighth-graders in the 96-percent low-income school, and their first answer is just one word: Looping.

The big chill Tribune: The report cards for Chicago schools were nothing to take home to mom and dad, either. And yet, what’s consuming the Chicago Public Schools and the Chicago Teachers Union these days? A drawn out fight over how long children should be in school.

Illinois graduation rates tumble under new formula WBEZ:  Illinois is releasing report cards for every school in the state today. The annual report card release shows high schools struggled. The state changed how it calculates the graduation rate this year—to bring it in line with federal guidelines. The effect: hundreds of high schools saw their graduation rates tumble.

Graduation rates slump across Illinois high schools Tribune: Despite its academic struggles, North Chicago Community High School could count on at least one piece of good news: Students were getting their diplomas. The school reported a 90.3 percent graduation rate in 2010.


Leave a comment
  • The test score discussion largely ignores regression toward the mean within the context of test scores for the thousands of students who are taking the ISAT and PSAE. I generally do not like to talk about this issue because the mathematics of it was developed by Sir Francis Galton in 1886. It was part of the eugenics movement that did much damage to people with disabilities and was part of the reason the Nazi's wanted to exterminate people with disabilities.

    But despite its dubious origins Galton's analysis has significant importance to the long term examination of ISAT and PSAE average scores. There is a relatively good paper on this issue available to readers of this blog at http://economics-files.pomona.edu/GarySmith/aveTestScores.pdf . As this article puts it " In educational testing, the “true score” is the statistical expected value of a person’s test score. A person’s observed score on any single test depends on the questions asked, the person’s health during the test, and even the person’s luck when unsure of the answer. The difference between an observed test score and the true score is the error score. Observed scores regress toward the mean because those who score highest on a test are likely to have had positive error scores that put their observed scores farther from the mean than are their abilities."

    The concept of regression to the mean also has impact on value added analysis or gain scores. For example students who are given pre-tests and posts test after remedial interventions on average can be expected to improve on a second test even if the instructor “had no more than passed his hand over the students’ heads before he retested them” according to R L Thorndike's early 1960s study of remedial interventions. So I think every article on the complex issue of the testing results overtime should be include a discussion of what Galton discussed over one hundred years ago.

    Linda Lutton's WBEZ piece attempted to discuss the difference between raw scores and the cut scores used by ISBE for ranking related to standards, but did not discuss regression to the mean. Duaa Eldeib Tribune article on 3rd grade test scores also did not discuss regression to the mean. We should not abuse Galton's discovery, but we do have to take it into consideration when looking at massive data sets we get from ISBE every year.

    Rod Estvan

  • In reply to Rodestvan:

    so, an outlier on the first measure trends toward an average on the second, and if it outlies on the second, it will have been closer to the mean on the first…in other words, we're right back where we started….

  • In reply to Rodestvan:

    Rod - I love that someone who is not a statistician actually brings up regression to the mean! Great point. You are totally right that both programs which target students who scored very low or those who scored very high have to take into account regression to the mean. For the remedial programs, students will tend to look like they improved at the second testing, even if they did not actually learn anything additional. For gifted programs students will look like they got worse at the second testing, making the program look like a dismal failure.

    This is a totally legitimate issue, but in the black and white world of CPS the administrators don't care about these *sticky* statistical issues because they consider them to be ways that researchers muddy the waters, not as legitimate issues that need to be addressed when determining program efficacy. They'd rather look at raw scores, because that makes most remedial programs look successful! - "wow, all those kids who volunteered to come to our program are doing better on tests than the kids who decided not to come. Amazing. We're doing such a great job in our programs." And don't even bother trying to mention the issue of selection bias...

  • Brizard, the minstral show, is being paid big dollars to mug for the camera?!?!?!?! Brizard is here for only one purpose, be the point person for the continuation of Ren 2010. You know, closing schools... and giving contracts to private charter schools. Brizard, instead of rolling up his sleeves and telling his master Rahm, that the right thing to do is not privatization, he mugs. Brizard, the Grim Reaper with a smile! Appopriate!

  • Chopin? Are their results possible at other schools? Would their technique of looping be useful elsewhere?

Leave a comment