Top Researchers Offer New Tool for Educational Recovery Efforts: Education Recovery Scorecard

On October 28, researchers from Harvard and Stanford universities jointly released the first nationally-comparable view of district-level learning loss during the pandemic.

For 29 states and the District of Columbia, researchers analyzed standardized test scores, examining both state tests and national NAEP scores to allow for comparisons across states. More states will be added in the coming weeks, as they report their state standardized test results from last school year. 

The researchers used both NAEP and state standardized test scores to measure losses in academic performance. Merely comparing state pass rates among states won’t give you accurate comparisons of students’ academic performance. States choose different tests and set different proficiency benchmarks, making apples-to-apples comparisons impossible. NAEP results are comparable across states. The researchers relied on NAEP results to set a comparable scale among states. 

They also examined the amount of time students in each district spent learning remotely and the amount of federal pandemic relief funds received. Though time spend in remote learning was associated with greater learning loss, losses also varied among districts that spend the same amount of time in remote learning. More research is needed to examine all the factors involved in students’ pandemic learning experiences–economic insecurity, access to broadband, illness and death rates in local communities–in order to disentangle the combined effects on learning.

How the Scorecard Works

While the overall news–as has proven true time and again–shows significant learning losses,  it’s important for parents and local education leaders to dig deeper into how their own schools were affected. 

That’s where the new tool comes in. 

Users can click on interactive maps to see district-level changes in math and reading achievement between 2020 and 2022. They can also view scatterplots showing how their district compares to others nationally. There’s plenty of supplemental information, including links to the Education Recovery Hub created by the Collaborative for Student Success, and Edunomics Lab’s Recovery Calculator, which estimates the amount of funds needed to help kids bounce back from pandemic-related learning losses.

“The pandemic was like a band of tornadoes that swept across the country,” CEPR Faculty Director Thomas J. Kane noted in a press release announcing the tool’s launch. “Some communities were left relatively untouched, while neighboring schools were devastated. The Education Recovery Scorecard is the first high-resolution map of the tornadoes’ path to help local leaders see the magnitude of the damage and guide local recovery efforts.”

Using the scorecard, parents and district leaders can see estimates of their local achievement losses, along with the number of students being served by catch-up efforts and an estimate of how much recovery those efforts are likely to produce for students. This allows districts to make sure their plans are truly proportionate to the task at hand.

“School districts are the first line of action to help children catch up. The better they know about the patterns of learning loss, the more they’re going to be able to target their resources effectively to reduce educational inequality of opportunity and help children and communities thrive,” said Sean Reardon, Professor of Poverty and Inequality, Stanford Graduate School of Education, in the launch announcement.

Recovery Will Take All the Resources We Can Wisely Invest

District leaders have received unprecedented amounts of federal funding to support recovery efforts. However, only 20% of those funds are required to be spent directly on learning recovery efforts. Plus, according to Kane and Reardon’s analysis, 65% of U.S. students attend school in districts where sustained learning losses likely require more investment than federal pandemic relief funds have provided.

This finding is likely to be met with skepticism. “I say this unapologetically: the schools have resources,” Chicago Public Schools CEO Pedro Martinez commented at a recent school board meeting. He noted he must have deeper conversations with principals about how they need to track how those resources are being used to support students’ recovery.

Chicago is investing a small fraction of its federal funds–$25 million–into high-dose tutoring, a research-backed strategy to help students recover academically. An outlier among high-poverty districts, Chicago has directed the largest share of its federal funds into salaries and benefits, mostly for existing staff. District officials say this strategy helps retain staff in the face of enrollment declines and will reduce the fiscal shock when the federal relief funds run out. However, in June, Edunomics Lab estimated that Chicago would need to spend more than $955 million on tutoring to help students catch up.

This yawning gap between estimates of what it will really take to catch students up and what districts are spending is not unusual. While states and districts are using some of their federal relief funds to add tutoring, summer school and extended days, many of those efforts are not yet large enough to address the full extent of learning loss. The Education Recovery Scorecard will provide  the hard data districts need regarding learning loss to guide spending on learning recovery that meets their students’ actual needs.

Harvard’s Kane insists it will take more than just school districts to help kids bounce back. 

“The whole village needs to hear the bell ringing, not just schools,” Kane said in a press release. “Mayors should organize tutoring efforts at local libraries. Community organizations should plan school vacation academies and summer learning opportunities. Governors should be funding and evaluating innovative pilots to provide models that everyone could use. We cannot wait for the Spring 2023 state test results next fall to tell us that we underinvested in recovery efforts. Many are happy just to get back to normal, but normal won’t help kids catch up.”