Math scores & loss aversion

Enhancing the Efficacy of Teacher Incentives
Loss-aversion incentives increased math test scores between 0.2 and 0.4 standard deviations
In recent years, a number of U.S. states and school districts have implemented teacher financial-incentive plans, also known as merit pay, with the goal of increasing student achievement. Some past studies have shown that such reform attempts, which pay teachers bonuses after their students hit certain goals, have had limited effects on student achievement.

In Enhancing the Efficacy of Teacher Incentives Through Loss Aversion: A Field Experiment (NBER Working Paper No. 18237), co-authors Ronald Fryer, Jr., Steven Levitt, John List, and Sally Sadoff find that using an alternative "loss aversion" incentive -- with teachers being paid bonuses in advance and asked to give money back if students don't achieve specific results -- significantly improves student achievement. Their results also suggest that loss-aversion incentives might be used in the corporate world in the pursuit of profits.

Although earlier studies have confirmed a correlation between teacher quality and student achievement, the challenge to date has been identifying quality teachers and providing proper incentives for all teachers to successfully strive for improved and lasting student achievement. At least ten states and numerous school districts in the United States have adopted programs that reward teachers with extra pay after students achieve certain goals on tests or report cards, but these "traditional" incentive programs generally have not had large effects on long-term student performance.

Fryer and his co-authors conduct a field experiment of teacher incentives using the concept of loss aversion -- that is, by framing incentives as losses rather than gains. They worked with schools in Chicago Heights, Illinois, which is located thirty miles south of Chicago and has nine K-8 schools with a total of about 3,200 students, during the 2010-11 school year. Chicago Heights' schools are made up of primarily low-income minority students who struggle with low achievement rates.'

In cooperation with school administrators and the teachers' union, the authors randomly selected 150 volunteer teachers and divided them into two main categories. The "gain" group was subject to traditional merit-pay incentives distributed after student achievement levels were determined and met; the "loss" group was subject to loss-aversion incentives that gave bonuses in advance, with the stipulation that money would be returned by teachers if students didn't hit stipulated test goals at the end of the school year. With a pool of $632,960 to distribute in incentive payments, the authors further subdivided the "gain" and "loss" groups in order to measure individual-based and team-based teacher incentives.

Using benchmarks from prior student test scores and final end-of-school-year ThinkLink Predictive Assessment test results, the authors find that loss-aversion incentives increased math test scores between 0.2 and 0.4 standard deviations, or the equivalent of increasing teacher quality by more than one standard deviation. The traditional "gain" incentives yielded "smaller and statistically insignificant results." Similar patterns were found in reading test scores -- and in both individual-based and team-based teacher incentive approaches. The authors did not identify any other factors, such as student absenteeism or outright cheating in test scores, which could explain the differences in achievement results.
--Jay Fitzgerald

Gold Star teachers

I like this idea:

Proposals for a Cost-Conscious Era: "Gold Star Teachers"

By Rick Hess on October 13, 2010 9:30 AM

For decades, the go-to school improvement recipe has been to reduce class size. Any challenge to this status quo encounters a buzz saw of opposition from parents and teachers who like small classes. That's why national teacher-student ratios are down to 15:1 today. Yet the research backing across-the-board class reduction is thin, at best. International evidence shows no simple relationship between class size and student achievement. Some high-performing nations boast middle or high school class sizes of 40 to 50 students. Small classes are costly and the need to keep adding bodies forces school systems to be less selective and training to be less focused.

Given that 55% of K-12 spending funds teacher salaries and benefits, you can't cut costs without boosting the productivity of good teachers--which requires increasing class size. But trying to sell that argument to parents or teachers is a dead end. Hence, the Gold Star program offers teachers who are at least reasonably effective the opportunity, should they so choose, to teach more kids per class and to be rewarded for taking on a larger workload. Such a state-level program would offer a chance to reshuffle the incentives and create a productivity-enhancing dynamic.

Teachers whose students post larger-than-normal gains for at least two consecutive years would be eligible to opt into the program. While I have consistently explained that value-added data systems have real limitations, they do provide a systematic way to identify teachers whose students are at least improving in math and reading at better-than-average rates. This gives some assurance that these teachers are at least reasonably effective. Participating teachers would teach up to 50% more students than normal--say, 36 students rather than 24--and would be rewarded for their increased workload. Continued participation would depend on a teacher's students continuing to make larger-than-normal gains. Given data limitations, states would be advised to pilot such programs in grades four through eight.

While parents prefer small classes in general, small classes also frustrate parents whose children can't get seats in the class of a heralded teacher. The Gold Star program lowers these barriers by allowing access to the most effective teachers for more kids. Given the choice between a Gold Star Teacher serving more children and the alternative, many or most parents will likely prefer the larger class. But it is essential that it be a parental choice and not an administrative fiat.

Teachers and taxpayers would also win big. On average, given current teacher salaries and benefits, increasing class size by one student saves something like $3,000; so allowing a talented teacher to instruct 36 rather than 24 saves up to $36,000. Awarding the teacher half that amount yields an $18,000 productivity bonus (a 35% bump for the median teacher). The state and district would split the other $18,000. Even on a trial basis in grades four through eight, such a program could help states shave school spending by two or three percent--tallying hundreds of millions in some cases while rewarding excellent educators.

Parents have choice, teachers have choice, and spending declines.

Offhand, I don't think this approach would necessarily interfere with professional learning communities, but I don't know.

Here’s a clue about why our schools are in trouble

If I didn't know anything else about our nation’s public schools, this one sentence from the WSJ article about merit pay in Denver would go a long way in explaining the trouble with our education system.

Nationwide, most teachers are paid based on two factors: education and experience.

Teachers take risks when they decide to teach middle-school math

The headline in the WSJ article says “Denver Teachers Object to Changes In Pay-for-Performance Plan”

Yawn

But what really caught my eye when reading this story was this sentence:

But the biggest rewards will go to early- and midcareer teachers -- and to those
willing to take risks by working in impoverished schools or taking jobs few
others want, such as teaching middle-school math.

No one who reads KTM regularly is surprised to find that teaching middle-school math is considered a job desired by few teachers. It must be darn hard to teach students who’ve emerged from elementary school not having mastered such things as fluency with multiplication tables, long division or fractions. These are considered by many mathematicians to be critical stepping stones on the way to higher-level mathematics. Instead, middle school math teachers might typically find students who have mastered things like math journaling, sorting colored manipulatives as a way to demonstrate they understand the “concept” of multiplication and pizza-slice fraction analysis.

Middle school math – where the math sh*t hits the fan.

PS – Actually, the entire article is worth reading.

The Denver teachers may decide to strike while the Democratic Convention is in town.

That could create some awkward moments. The Democrats don't want to anger
teachers unions, which are key allies. Nor do they want Denver's plan to fall
apart.

Merit pay did make a difference.

Before the plan took effect, she said, "we almost never sat down with our
principals to say, 'Where are our students now, where do they need to be and how
do we get them there?' This really has changed the culture in our schools."

Teacher's Union Opposes Salary Increases

In Washington D.C. a strange thing is happening. A proposal to improve the educational cesspool that is the public school system in the Nation's Capital is being torpedoed by the national teachers unions (despite support from the local union leaders and teachers).

I haven't seen the story reported in any US newspapers, and I wonder why not, since I read education columns quite regularly. If I were any more cynical than I already am, I might think there is a plot by the national unions to keep this particular proposal quiet. Or perhaps I just missed the US press stories due to summer vacations and what not. Anyway, the British news weekly, The Economist, had a short story two weeks ago.

The story begins with a typical indictment of the District's schools -- hugely inefficient, extremely high cost, abysmal test scores, and outrageous student behavior. And then the teachers:

Teachers are virtually unsackable and paid by seniority. Such incentives attract the lazy and mediocre and repel the talented or diligent.

But here's where it gets interesting, the solution is somewhat novel in education reform:

Ms Rhee [Michelle Rhee, School Chancellor] is thrashing out a deal with union leaders that would raise teachers’ wages dramatically. Starting salaries would leap from about $40,000 to $78,000, and wages for the best performers would double to about $130,000 a year. In return, teachers would lose tenure and be paid according to merit, measured in part by their students’ results. Current teachers would have a choice: they could join the new system or stay in the old one. New hires would have to join the new system. Over time, the quality and morale of teachers in Washington should soar. “Imagine the kind of talent the hard-pressed system could attract,” drools the Washington Post.

But wouldn’t all this require a huge expansion of the school budget? Perhaps not. The current system is staggeringly inefficient. The city employs an army of educational bureaucrats and has twice as many schools as it needs. It pays to heat and air-condition some schools that are only a quarter full. Insiders reckon that, within a few years, the new pay deal could be wholly financed by cutting waste. And in the short term private donors are willing to shoulder much of the cost.

The plan’s boosters call it revolutionary, in that it applies to public schools a principle—reward good work and you get more of it—that every other employer has known for centuries. But it will be still-born if the Washington teachers’ union does not agree to it. Local union leaders rather like the idea of higher pay, but the big national unions are appalled at the notion that any teachers might give up tenure. Fearing an unwelcome precedent, they are leaning on the local union to kill the deal.

I can imagine the union drooling all over the pay hikes, but like most behemoth bureaucracies, they'd like the cake too, please. Big pay hikes, no accountability, lifetime tenure. Still, it is disheartening to see the difficulty the District is having getting this approved. If merit pay is a nonstarter in a district performing as badly as Washington D.C., what hope have we in a nominally high performing suburb?