Los Angeles Unified School District rates schools using valued-added methodology

The L. A. Times reports:

In a dramatic turn for the country’s second-largest school district, Los Angeles Unified released school ratings based on a new approach that measures a school’s success at raising student performance — the first in a series of high-stakes moves that will thrust the district into the center of the national debate over education reform.

Next month, the district will take the more controversial step of providing thousands of teachers with confidential ratings of their performance using the same approach, known as value-added. The district is also negotiating with the teachers union to include such measures in teachers’ formal performance reviews, an effort the union bitterly opposes.

I have posted about the value-added modeling debate previously… See this post, as well as this white paper on the statistics underlying value-added modeling.

April 8 K-12 news round-up

• New York City schools chancellor Cathie Black has resigned her post (apparently with encouragement from NYC Mayer Mike Bloomberg) after just three months on the job. Black was a controversial pick because she completely lacked experience either as an educator or K-12 administrator. [Huffington Post]
• Education Week interviewed Kelly Gallagher, a former secondary English teacher and author of the new book Readicide, about declining reading skills (and declines in time spent reading) among American youth.
• A growing number of parents are opting out of allowing their children to participate in standardized testing, leading some researchers and educators to wonder about the effects these opt-outs have on schools’ test performance. [CNN]
• USA Today profiles Jacob Barnett, a gifted 12-year-old who is wowing fellow physics students (and professors) at Indiana University. (I acknowledge this is not strictly speaking a K-12 news story but it’s interesting nonetheless!)
• “Children of color are four times more likely than their white peers to be born into a poor family and suffer a lifetime of consequences, ranging from diminished academic standing to increased financial insecurity, a report released Thursday found.” [Huffington Post]

Speaking more than one language might benefit your brain

NOTE: This update is cross-posted with my Cognitive Science Blog.

NPR reports:

Research suggests that the growing numbers of bilingual speakers may have an advantage that goes beyond communication: It turns out that being bilingual is also good for your brain…

The idea that children exposed to two languages from birth become confused or that they fall behind monolingual children is a common misconception, says Janet Werker, a psychologist at the University of British Columbia who studies language acquisition in bilingual babies.

“Growing up bilingual is just as natural as growing up monolingual,” said Werker, whose own research indicates babies of bilingual mothers can distinguish between languages even hours after birth.

“There is absolutely no evidence that bilingual acquisition leads to confusion, and there is no evidence that bilingual acquisition leads to delay,” she said.

Werker and other researchers say the evidence to the contrary is actually quite strong. Instead of holding you back, being bilingual, they say, may actually be good for you.

Debating the value of college admissions ‘test prep’ programs

Education Week reports:

Derek Briggs, the chairman of the research and evaluation methodology program in the school of education at the University of Colorado at Boulder, has examined how much SAT score increases can be attributed to the effect of test prep.

Test-prep programs generally include three elements: a review of test content, practice on test questions, and orientation to the format of the test. In 2009, in cooperation with NACAC, Mr. Briggs reviewed three national data sets and found the average effect of commercial coaching is positive, but slight. Test-score bumps were more in the neighborhood of 30 points (on a 1,600-point scale at the time), far from what some in the industry claim. He does point out that there may be specific programs that are more effective than others, but evidence to support that is weak.

Considering the results of Mr. Briggs’ and earlier studies, NACAC concludes that test-prep activities and coaching have a “minimal positive effect on both the SAT and the ACT.”

FULL DISCLOSURE: Dr. Briggs is on the faculty at the U. of Colorado at Boulder, where I am a Ph.D. candidate. I have studied with Dr. Briggs in the past.

International Summit on Teaching reveals U. S.’s approach to K-12 education differs greatly from other nations

Writing in the Washington Post, Stanford professor Linda Darling-Hammod reports on the International Summit on Teaching, a first-of-its-kind event held in New York City this past month:

Evidence presented at the s ummit showed that, with dwindling supports, most teachers in the United States must go into debt in order to prepare for an occupation that pays them, on average, 60% of the salaries earned by other college graduates. Those who work in poor districts will not only earn less than their colleagues in wealthy schools, but they will pay for many of their students’ books and supplies themselves.

And with states’ willingness to lower standards rather than raise salaries for the teachers of the poor, a growing number of recruits enter with little prior training, trying to learn on-the-job with the uneven mentoring provided by cash-strapped districts. It is no wonder that a third of U.S. beginners leave within the first five years, and those with the least training leave at more than twice the rate of those who are well-prepared.

Those who stay are likely to work in egg-crate classrooms with few opportunities to collaborate with one another. In many districts, they will have little more than “drive-by” workshops for professional development, and – if they can find good learning opportunities, they will pay for most of it out of their own pockets.

Meanwhile, some policymakers argue that we should eliminate requirements for teacher training, stop paying teachers for gaining more education, let anyone enter teaching, and fire those later who fail to raise student test scores.

USA Today investigates DC Public Schools’ standardized testing anomalies

USA Today has published a major investigative article on standardized testing anomalies at District of Columbia public schools, many of which were praised (and financially rewarded) by former chancellor Michelle Rhee.

In just two years, Crosby S. Noyes Education Campus went from a school deemed in need of improvement to a place that the District of Columbia Public Schools called one of its “shining stars.”

Standardized test scores improved dramatically. In 2006, only 10% of Noyes’ students scored “proficient” or “advanced” in math on the standardized tests required by the federal No Child Left Behind law. Two years later, 58% achieved that level. The school showed similar gains in reading.

Because of the remarkable turnaround, the U.S. Department of Education named the school in northeast Washington a National Blue Ribbon School. Noyes was one of 264 public schools nationwide given that award in 2009.

USA Today goes on to explain that the paper launched in investigation of  DC Public Schools because of the national attention DCPS has received during and after the tenure of Chancellor Rhee; indeed, the paper reports, DCPS received tens of millions of dollars in federal funding based on its purported improvements in test scores. But there was a problem:

A USA TODAY investigation, based on documents and data secured under D.C.’s Freedom of Information Act, found that for the past three school years most of Noyes’ classrooms had extraordinarily high numbers of erasures on standardized tests. The consistent pattern was that wrong answers were erased and changed to right ones…

Noyes is one of 103 public schools here that have had erasure rates that surpassed D.C. averages at least once since 2008. That’s more than half of D.C. schools.

Erasures are detected by the same electronic scanners that CTB/McGraw-Hill, D.C.’s testing company, uses to score the tests. When test-takers change answers, they erase penciled-in bubble marks that leave behind a smudge; the machines tally the erasures as well as the new answers for each student.

In 2007-08, six classrooms out of the eight taking tests at Noyes were flagged by McGraw-Hill because of high wrong-to-right erasure rates. The pattern was repeated in the 2008-09 and 2009-10 school years, when 80% of Noyes classrooms were flagged by McGraw-Hill…

On the 2009 reading test, for example, seventh-graders in one Noyes classroom averaged 12.7 wrong-to-right erasures per student on answer sheets; the average for seventh-graders in all D.C. schools on that test was less than 1. The odds are better for winning the Powerball grand prize than having that many erasures by chance, according to statisticians consulted by USA TODAY.

USA Today on why teachers cheat

USA Today reports:

Teachers cheat sometimes and so do principals, according to academic studies. Why it happens and how often — and the seriousness of efforts to stop it — are open to debate. Punishment varies from state to state, too. In an investigation of standardized testing in six states and the District of Columbia, USA TODAY found that an infraction such as casually coaching one student can carry nearly the same punishment as deliberately changing answers for a whole class.

The consequences can be drastic: Cheating can cost school districts thousands of dollars for makeup tests, set back the careers of gifted teachers and create confusion for schools and parents over a child’s academic progress.

In an Arizona State University survey published last year, more than 50% of teachers and other educators admitted to some kind of cheating on Arizona’s state tests. The authors of the online survey of more than 3,000 educators defined cheating broadly — from accidentally leaving multiplication tables on classroom walls to changing answers.

Findings of widely-cited Gates Foundation study on teacher effectiveness called into question

Late in 2010, the Bill & Melinda Gates Foundation, which has sponsored much research into teacher effectiveness, released the Measures of Effective Teaching report (PDF). The report’s analysis was developed using a statistical methodology called value-added modeling, which attempts to quantify the impact an individual teacher has on students’ learning outcomes by examining student standardized test scores over time and across different classes. A number of education reformers, including Arne Duncan, U. S. Secretary of Education, have touted value-added modeling in general and the Gates Foundation report in particular as important new tools for understanding teacher effectiveness.

Now, an analysis of the Gates Foundation report by a U. of California at Berkeley economist challenges the conclusions of the Gates Foundation report:

The Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project seeks to validate the use of a teacher’s estimated “value-added”—computed from the year-on-year test score gains of her students—as a measure of teaching effectiveness. Using data from six school districts, the initial report examines correlations between student survey responses and value-added scores computed both from state tests and from higher-order tests of conceptual understanding. The study finds that the measures are related, but only modestly. The report interprets this as support for the use of value-added as the basis for teacher evaluations. This conclusion is unsupported, as the data in fact indicate that a teachers’ value-added for the state test is not strongly related to her effectiveness in a broader sense. Most notably, value-added for state assessments is correlated 0.5 or less with that for the alternative assessments, meaning that many teachers whose value-added for one test is low are in fact quite effective when judged by the other. As there is every reason to think that the problems with value-added measures apparent in the MET data would be worse in a high-stakes environment, the MET results are sobering about the value of student achievement data as a significant component of teacher evaluations.

The pros and cons of teacher seniority rules

Writing in the Huffington Post,  Los Angeles middle school teacher, blogger, and published author Heather Wolpert-Gawron summarizes the most commonly-heard pros and cons of public K-12 teacher seniority rules, and offers some ideas about how teacher seniority and tenure can be reformed in the future.

The the full list at The Huffington Post.

USA Today on standardized test scores that are “too good to believe”

USA Today reports:

A pattern [has been] uncovered by a USA TODAY investigation of the standardized tests of millions of students in six states and the District of Columbia. The newspaper identified 1,610 examples of anomalies in which public school classes — a school’s entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests.Such anomalies surfaced in Washington, D.C., and each of the states — Arizona, California, Colorado, Florida, Michigan and Ohio — where USA TODAY analyzed test scores. For each state, the newspaper obtained three to seven years’ worth of scores. There were another 317 examples of equally large, year-to-year declines in an entire grade’s scores.

 

The article describes the methodology employed in analyzing these test data, an approach that is fairly orthodox among testing experts and education researchers:

[USA Today] compared year-to-year changes in test scores and singled out grades within schools for which gains were 3 standard deviations or more from the average statewide gain on that test. In layman’s language, that means the students in that grade showed greater improvement than 99.9% of their classmates statewide.

The higher the standard deviation, the rarer that improvement is. In dozens of cases, USA TODAY found 5, 6 and even 7 standard deviations, making those gains even more exceptional.