Experts weigh in on how to think about standardized exams

Thousands of New York City kids are advancing through school with solid report cards yet failing standardized test scores. The disparity leaves parents worried about their children and critical of teachers, schools, and the tests themselves. If Johnny is passing his classes but failing the state reading exam, Who is wrong—The test or the teacher?

Reasonable people can disagree. And do they ever.

The Ink talked to education experts about the merits and pitfalls of standardized testing in an effort to illuminate this confusing issue.

 

The first thing you might ask when you see a bad score is this: Is the test even reliable?

Experts know what parents don’t: the tests aren’t great at measuring much. Test scores can only check a student’s progress at a single moment in time: a three-hour snapshot of reading or math abilities out of a 180-day academic year. And in New York, the tests changed twice in five years, which means teachers have had little time to learn and adjust to new requirements and material.

But that doesn’t mean they aren’t reliable. The tests are actually very predictive of one thing: whether a student will pass next year’s exam. And research shows that the tests are reliable in the sense that a student will perform just as well on test day as any other school day, said Howard Everson, a professor in educational psychology and director of the Center for Advanced Study in Education at City University of New York.

So don’t brush away a bad test score just because little Johnny had a tough day. He would probably get the same score if he took the exam any other day.

In that sense the tests are very reliable, Everson said, but that doesn’t mean they are successful.

“The real controversy is, are they valid,” Everson said.  “Are they testing the things that the kids are taught, are they testing them in ways they are familiar, and are they given under conditions under which the kids have a chance to do their best?”

The answers to those questions are unclear, he added. And the meaning of a poor test score is unclear, too.

 

If people disagree on how valid the tests are, what are they even measuring?

Professor Aaron Pallas, who teaches at the Columbia University Teacher’s College, said New York’s standardized tests are only nominally predictive of future college-readiness.

Before the minimum score for a passing grade was raised in 2011, only 50 percent of students who consistently passed the standardized tests in grades three to eight went on to college. After the 2011 change, nearly 75 percent were college-bound. Nothing about the test changed then, but it became more difficult to pass and the number of students doing so fell dramatically. Still, similar numbers of students were attending college.

The takeaway message was that the tests didn’t matter when it came to college readiness. Some students who failed went to college anyway. Some who passed didn’t go. That trend continued even after the bar was raised, though a larger percentage of the much smaller number of students passing the exams were college-bound, Pallas said.

A second change to the exam in 2013, which completely overhauled the test to align with the national “Common Core” standards, caused another dip in scores across New York City. Pallas said the drop probably occurred because the tests haven’t been optimized to test what students are learning—but that’s not anyone’s fault, he added. It should take time, potentially several years, for teachers, students, and test-makers to get used to the new standards and fairly align what’s on the exam with what’s taught in class.  

That means a poor test score could be because the new exam isn’t optimized, yet. And a bad test score won’t necessarily keep a kid out of college–though three out of four children passing the exam will end up pursuing a higher degree, so it’s still a goal for students to strive toward.

 

If the test isn’t optimized, should parents even bother worrying about a bad score?

Pallas said he would put a lot of stock in teacher feedback. The test scores are so volatile from year to year and between schools that good grades and glowing reviews could easily tell you more about a student’s performance, he added.

Experts agree that the exams don’t do the best job at measuring what students are learning, largely because of curriculum differences across states, Everson said.

 

Why do states keep testing anyway?

The test scores do provide an early warning signal for schools to identify teachers and students who might be struggling, said Sean Corcoran, a professor of education economics at New York University. Intervention at the earliest sign of trouble can keep a student on track.

For teachers, the scores should be the “beginning of a conversation” when students consistently perform poorly, Corcoran added. He said the scores can show valuable information about teacher performance, too, if they are looked at over a number of years. However, the scores are rarely looked at in comparison with last years’ results. Teacher evaluations tend to simply use a statistical algorithm to assign a certain amount of points to a score from the current year and ignore past performance completely, Corcoran said.

Still, the history of a teacher’s test scores can be seen by digging up the data. Schools and parents alike could look at an instructor’s history of scores to get an idea for how well that person is teaching (though the recent upheaval in test format would still likely have major impacts on any teacher’s scores since 2011).

So much is riding on the test scores that they have become embroiled in controversy: Teacher evaluations that affect future job prospects and tenure rely heavily on the test results. Student placement in classes is at least partially determined by standardized exam scores. Even school funding can be won or lost by the fluctuating metrics.

Most of all, proponents claim standardized tests are the most objective feedback on teacher performance that exists. Every student in a state answers the same questions on the same day. The performance should reflect their knowledge.

But the problem is that curriculum is set at the local, not state, level, and students at different schools learn different things. And the tests only go after math and English skills, neglecting student learning in science, history, and the arts.

The tests do give districts and the state more power to hold failing schools and failing teachers accountable. Before state standardized exams, there was no external metric to evaluate school performance and schools were left to succeed or fail with little consequence for the institution.

 

So, we should keep these tests?

Neither Pallas, Everson, nor Corcoran advised tossing the exams. They are still an important tool, they argued. And if the tests can be aligned with what students are taught, the scores will gain even more usefulness.

Give teachers and test makers time to work it out, Pallas suggested. And, until then, Johnny’s good grades are probably enough to ease a parent’s worries and might be a more accurate measure of how much he’s learning. But he should keep trying to pass that state standardized test next year.