The state Department of Education released its proposed teacher evaluation regulations in early March. After much review, one thing is clear: the NJDOE has not answered the question of how to measure student growth accurately, and even more important, how to fairly and accurately attribute that growth (or lack of it) to a specific teacher.
NJDOE officials shouldn’t be embarrassed that they were unable to solve this mystery. As some of them have explained, no state has figured this out. And if the regulations pass as currently written, New Jersey won’t either.
The issue of measuring student growth is complex, so we’ll only deal with one part of the problem—student growth percentiles, or SGPs. Under the proposed regulations, SGPs would be calculated for math and language arts teachers in grades 4-8.
How would it work? From a student’s performance from year to year on the NJASK, the state will calculate that student’s percentile rank based on his or her test performance compared to other students across the state who have performed similarly. The individual student is assigned a number by the state known as that student’s growth percentile. The teacher will be assigned a score based upon the median of all of his or her students’ growth percentiles. That score will count as 35 percent of that teacher’s summative rating.
As noted researcher Dr. Howard Wainer notes, this is “one of those ideas that only sound good if you say them fast.” Author of Uneducated Guesses—Using Evidence to Uncover Misguided Education Policies (Princeton University Press, 2011), Wainer notes that dealing with percentiles means it is a zero sum game; if one student goes up it must be counter-balanced by another who goes down.
“Such a system is hardly calculated to build a cooperative learning environment,” Wainer explains. “Moreover, it does not allow us to see overall trends, only comparative ones.”
He likens this exercise to measuring the height of a classroom full of children.
“Some would grow faster than others, and the percentile changes would reveal this, but it would miss the big fact that all are growing. Both aspects – absolute growth and comparative growth – are important, but the focus on just percentile growth diminishes the more important aspect of absolute growth,” says Wainer.
A distinguished research scientist for the National Board of Medical Examiners, Wainer also questions the NJDOE’s assertion that “no educator is ever ‘penalized’ for teaching students at any achievement level” because even “students at the highest end of proficiency can also show growth.”
How can this be true? Common sense tells us that if a student gets a pre-score of 40 percent there is a lot of room to improve. If another gets a pre-score of 95 percent there isn’t. And Wainer believes that the use of percentiles makes things worse.
“Score distributions have lots of people in the middle and they get sparse in the extremes, so a small gain in the middle of the distribution jumps you over a lot of people,” Wainer explains.
“For example, a 10-point gain in the SAT from 790 to 800 jumps over about 4,000 to 5,000 people (out of the 1.5 million who took it) whereas a 10-point gain in the middle of the distribution, from say 500 to 510 jumps over 54,000 other people, so the gain in percentile rank is more than five times greater in the middle than in either tail.”
It’s worth noting that even Colorado, the state that has led the charge toward student growth percentiles, is still not using SGPs to evaluate teachers. The NJDOE should heed the lessons of other states and the expertise of statisticians and find a fairer and more reliable way to incorporate student growth into a teacher’s evaluation.
And that’s how it sounds when you say it slowly.
Up next: the problems of student growth objectives (SGOs)