Low-scoring students have not gained in the era of test score “accountability”
“We had appendicitis and they gave us massive chemotherapy.”
Last of three articles about what test scores show about the reading skills of Massachusetts and Boston students.
By Alain Jehlen
Retired teacher Kathy Greeley, who worked at a struggling Boston school for two years, knows from first-hand experience that schools don’t always do right by all students. But did MCAS and the MCAS graduation test help them do better?
She doesn’t think so.
“We had appendicitis and they decided to give us massive chemotherapy,” a colleague told her.
Due to low MCAS scores, the BPS central office aborted the teachers’ own successful efforts to improve their school and fired most of them.
Did MCAS help?
But some policymakers still believe test score “accountability” is the key to helping children learn better. They insist that MCAS has led to huge jumps, especially for students who used to leave school with low reading skills. Thanks to MCAS, they say, that no longer happens.
Is it true? Has MCAS lifted up the bottom, the students with the poorest skills?
Thanks to a nationwide testing program, we can answer that question. The answer is no.
The National Assessment of Educational Progress has been testing students in English and math for decades. The scores aren’t reported for individuals or schools, so there’s no incentive to teach to the NAEP test, as there is for MCAS. Since 1998, NAEP scores have been reported separately for Massachusetts (and other states), in addition to the whole country. Since 2003, they’ve been reported for the Boston Public Schools.
The reports are broken out by race, ethnic group, income, and other groups.
Most important for this article, NAEP reports the scores of students at the 10th, 25th, 50th, 75th, and 90th percentiles of each group. A student at the 10th percentile scored lower than nine-tenths of students in the group.
These are the students who are supposed to have benefited most from the pressure that school districts feel to raise their scores.
If MCAS helped the students with the weakest skills, the progress should show up for the students in that bottom 10th percentile.
There are many things students should learn in school that are not measured by a standardized test. But if the pressure of testing did any good, it should at least show up as better performance of the skills that are on the test.
See for yourself
For Massachusetts as a whole, here is the chart for eighth-grade reading showing all the percentiles, bottom to top.
The last scores are for 2022 so they’re affected by the pandemic. But look at the 2019 scores. Do you see dramatic improvement, especially for the 10th percentile students?
Rise and fall
Scores went up a bit in the first few years, which coincided with a major increase in school funding, concentrated in low-income districts. But then state funding leveled off. So did the scores.
In 2010, the pressure from MCAS intensified. A new law gave state officials the power to take over schools and whole districts if their scores were low. The state seized control of four individual schools and three districts, making the threat real for all low-scoring districts. But scores showed no improvement. And in 2019, the last year before the pandemic, they sagged, especially for the students in that bottom 10th percentile.
In short: Test-based accountability failed.
Remember that we’re looking at 2019 scores, pre-COVID, so the pandemic is not the reason for the decline.
It would be good to look at twelfth-grade scores, too, but NAEP’s twelfth-grade testing is sparse. The only scores available are for 2009 and 2013. Here they are:
Going back to the eighth-grade scores, let’s look at subgroups of students.
Supporters of test-based accountability say that the groups that benefit most are those that score below average: Black and Latinx students, low-income students, students with disabilities, and English learners. Here are the charts showing how these groups have fared in the era of testing.
A note on the English learners graph:
Most subgroups wound up about where they started out, but English learners scored lower at the end than at the beginning. Why?
To be counted as an English learner, you have to score low on an English test, so its no surprise that English learners do badly on the English MCAS. Once they become fluent in English, they’re no longer English learners. If English learners improved their English skills enough to get out of the group, you wouldn’t see that in the scores.
Also, the scores of English learners are affected by the rate of immigration from non-English-speaking countries. If there are many recent immigrants, the scores will be lower than if most immigrant students have had a few years in America to learn English. I haven’t compared this chart with immigration rates. The falling scores could be due to rising immigration.
Boston scores
Now let’s look at Boston Public Schools scores. NAEP scores for Boston have only been available since 2003, so we can’t tell what happened earlier. But since 2003, they pretty much track the Massachusetts scores. Coincidentally, 2003 was the first year that seniors had to pass MCAS to get their diplomas.
NAEP has not reported twelfth-grade scores for BPS.
Most groups peaked just before 2010, the year the state ratcheted up the possible punishments for districts with low scores. Then, with the pressure at its fiercest, scores slipped.
Chemotherapy didn’t cure the appendicitis.
A different approach
What might work better to help all students get a good education?
Schools can’t wipe out the enormous obstacles thrown up by poverty and racism, but Kathy Greeley has seen real progress in schools when teachers are able to work together, learn from each other, and get the help they ask for. She says:
“It continually astounds me how rarely schools ask teachers and other staff for their input on what schools need.
“I don’t mean, ‘Do you need a new whiteboard or more math books?’ I mean, ‘What do you need to do your job to the best of your ability? What do underperforming students need to be successful? How would you address the achievement gap? How would you prioritize these various initiatives?’”
That’s an excerpt from Greeley’s book, Testing Education: A Memoir, due to be published April 22 by University of Massachusetts Press.
Failure at the top
In her book, Greeley relates how she and other staff at a Boston school worked through a wide range of difficult problems, from poor reading instruction to inconsistent school discipline and tensions between staff and administrators.
But the district administration, under former Superintendent Carol Johnson, shut all their efforts down because their math scores were low. Instead, officials sent in new teachers and a new principal, who lasted four months, followed by more turmoil, until finally the school was absorbed by another school.
Greeley’s school was one of many where test score “accountability” inflicted harm on the children it was supposed to help.
