Monday, September 14, 2009

When Good News Really Isn't

I'm currently reading a fascinating book by Dr. Daniel Koretz, a psychometrician at Harvard's Graduate School of Education, called Measuring Up: What Educational Testing Really Tells Us. In the book, Dr. Koretz talks a lot about the pitfalls of high-stakes standardized testing schemes such as those required by the No Child Left Behind Act. He devotes an entire chapter to the topic of score inflation.

I was reminded of this book today when I read an article in the New York Times about how New York State has reduced the passing score for its math test from 60% correct in 2006 to a mere 44% today. An investigation by the NYT found that a student who randomly guesses on all question now has an 89% chance of receiving a passing score.

Federal tests do not show the same kind of dramatic increase in passing rates that the New York state tests have in recent years. In fact, math scores have been stagnant on the 8th grade NAEP exams since 2003 and 4th graders have only made minimal progress. SAT math scores in the state have actually dropped by 18 points since 2005.

The jump in scores on state tests helped 97% of schools in New York City earn ratings of "A" or "B" on their state Dept. of Ed. report cards. Does anybody seriously believe that 97% of NYC schools actually are doing a good job at educating their students? Nearly 40% of all students in the city do not complete high school, including 49% of African-Americans and 52% of Latinos. Nearly 3/4 of those who do manage to graduate and enroll in college require remediation in at least one subject.

Families deserve to know the truth about how their students are faring. It is unethical to lower the bar and then trumpet the "progress" that has been made :-(

3 comments:

Tertium Quid said...

I have put up a post about our first two weeks of homeschooling. I hope you are well.

JJ Ross said...

I don't have citations to offer but I remember from grad school in education and more importan;y my experience working for a large school board in the 1980s, that standardized test scores on any test tend to "improve" over the years. Even without this sort of lowering the bar.

I can't recall specifics of the effect (I was young and it wasn't my concern at the time) but my impression even then was that the whole system was set up to benefit both the seelers and the buyers of the test, in neat election-term blocks of time.

Crimson Wife said...

Dr. Koretz discussed one study that compared scores for the same schools before and after the state switched from one achievement test to another. The 3rd graders were allegedly working at a 4th grade level on the old test the last year it was required. Then the first year after the switch, scores dropped to a mid-3rd grade level on the new test. After a couple of years, scores had risen back to a 4th grade level on the new test. However, when those same students were tested with the old test, they only scored in the mid-3rd grade level. These tests were considered similar in difficulty so the difference he attributed to coaching/teaching to the test.