A pattern [has been] uncovered by a USA TODAY investigation of the standardized tests of millions of students in six states and the District of Columbia. The newspaper identified 1,610 examples of anomalies in which public school classes — a school’s entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests.Such anomalies surfaced in Washington, D.C., and each of the states — Arizona, California, Colorado, Florida, Michigan and Ohio — where USA TODAY analyzed test scores. For each state, the newspaper obtained three to seven years’ worth of scores. There were another 317 examples of equally large, year-to-year declines in an entire grade’s scores.
The article describes the methodology employed in analyzing these test data, an approach that is fairly orthodox among testing experts and education researchers:
[USA Today] compared year-to-year changes in test scores and singled out grades within schools for which gains were 3 standard deviations or more from the average statewide gain on that test. In layman’s language, that means the students in that grade showed greater improvement than 99.9% of their classmates statewide.
The higher the standard deviation, the rarer that improvement is. In dozens of cases, USA TODAY found 5, 6 and even 7 standard deviations, making those gains even more exceptional.