Pending further information from CREDO at Stanford and Excellent Schools Detroit, I’m taking down the Friday post about the New York Times’ Detroit charter schools story, “NYT Detroit Charter Story Misleads On Overall Results, Says Researcher.”

Here’s why:

There were several flaws in my reporting of the column, including (a) going through the NYT press office with interview requests rather than approaching reporter Kate Zernike and her editors directly and (b) agreeing to talk to a Stanford researcher who was not the main point of contact with Zernike during the reporting process. Hoping to talk with her directly, I also failed to read all of Zernike’s Twitter responses to criticisms of her work.

There were other points of confusion and miscommunication as well.

As became apparent Friday afternoon, it turned out that the NYT press office had not actually told Zernike that I was seeking an interview with her to talk about her account.

We finally spoke Friday afternoon, shortly after the original column was published, at which time Zernike was able to clarify and explain several questions that had come up about her piece.

It also turned out that the Stanford researcher I interviewed, James Woodworth, who criticized the way the NYT portrayed the CREDO study, was mistaken. He had not actually spoken to reporter Zernike — another CREDO researcher (who is out of the country) had done so.

And it turns out that Zernike’s unattributed description that “half the charters perform only as well, or worse than, Detroit’s traditional public schools,” was a reference not to the CREDO research but to research findings from another organization, Excellent Schools Detroit, in cooperation with the Education Policy Initiative at the University of Michigan.

According to ESD’s Armen Hratchian, “Ms. Zernike did not reference a specific source…If she did use [ESD data], I would say her quote is accurate… In terms of all charters above/below the [district] K8 average, 41 schools scored a 29% or higher on the 2016 ESD Scorecard while 42 scored a 28% or lower.”

It’s possible that Zernike’s description of the ESD data includes the same flawed grouping of results that was of concern with the CREDO data. (Based on ESD’s data, roughly the same number of K-8 Detroit charters exceed the district average as fall below it.) But I’ve been unable to determine the most accurate description of the ESD data and in the meantime it seems most appropriate to post this explanation of where things stand at present.

The original post is below:

NYT Detroit Charter Story Misleads On Overall Results, Says Researcher*

There was a fair amount of predictable howling and cheering in response to the New York Times’ big Detroit education story earlier this week.

For charter school critics, the front-page story was another bit of evidence that these semi-independent public schools aren’t nearly as good as has been claimed. Among charter school supporters, the Times story went out past the actual evidence and — combined with NPR’s problematic feature story on Rocketship charter schools — represented a confirmation of media bias.

But a few substantive and verifiable issues did come up in the response to the Times piece, including the story’s apparent mishandling of research evidence about charter school performance and its reliance on anecdotal information.

In fact, one of the researchers whose work is cited in the story says that education reporter Kate Zernike makes a massive if common mistake in her piece.

According to CREDO senior research associate James Woodworth, Zernike’s description of charter school performance is highly misleading.

“She’s combing categories together,” he said in a telephone interview. “We tell everyone not to do that.”

With this piece, Zernike and the Times were following up on a multi-day series from the Detroit Free Press that came out two years ago, after which many of the same objections were lodged.

As you may recall, Zernike rejoined the education team last year and, with the departure of Motoko Rich to the Toyko bureau, she is the main national education reporter for the paper.

Zernike penned a November piece on standardized testing changes in Massachusetts that was controversial in its description of how much or little the state was actually changing course.

The Times coverage of education issues, both nationally and locally, has been a frequent topic of this column.

In the piece, Zernike tells the story how public officials in charge of educating Detroit’s children focused much of their effort on creating more charter schools and choice for families living there, but the result hasn’t been nearly as transformative as hoped. There is “lots of choice, with no good choice.”

Zernike’s thesis: “Michigan leapt at the promise of charter schools 23 years ago, betting big that choice and competition would improve public schools. It got competition, and chaos.” She describes “unchecked growth of charters,” and “a glut of schools competing for some of the nation’s poorest students.”

It’s not just a few bad charters here and there. According to Zernike’s piece, research shows that ‘half the charters perform only as well, or worse, than Detroit’s traditional public schools.”

No one disputes the claim that charter schools in Detroit are a bit of a mess (as is the traditional school system there). There are too many schools for the number of students who need an education. Low-performing schools aren’t getting closed as quickly as they should. The geographic distribution of the schools is all wrong. The results aren’t nearly as strong as anyone would like them to be. There’s a lot of taxpayer money being spent.

Even some charter supporters are appalled at what they read. “Some charters in Detroit are bad for kids and no one is holding them accountable,” Tweeted the Education Post’s Valentina Korkes. They’re making a bad name for charters everywhere, which is *also* bad for kids.”

But others think that Zernike dismissed decades of previous problems that have hindered Detroit schools and continue to plague the community, or focused on the results and delivery mechanisms rather than the systemic issues. “Choice didn’t fail [in Detroit], public oversight did.” says Robin Lake, whose Seattle-based think tank the Center on Reinventing Public Education recently issued a report on Detroit’s broken system. “The problem is lack of oversight and investment across all schools.”

Zernike’s description of the CREDO research — “half the charters perform only as well, or worse, than Detroit’s traditional public schools” — is at the heart of the matter. And apparently Zernike gets it wrong here.

According to CREDO senior research associate James Woodworth, this description is highly misleading. “She’s combing categories together,” he said in a telephone interview. “We tell everyone not to do that.”

The reason is simple, according to Woodworth. The CREDO study breaks school performance down into three categories. In reading, 4 percent of Detroit charters did worse, 45 percent did about the same, 51 did better. According to Zernike’s combination, nearly half of the charters perform as well or worse than their district counterparts.  But “you could just as easily say that 96 percent of charters in Detroit did the same or better than traditional schools,” notes Woodworth.

To be fair, Zernike’s not alone in doing this, says the Stanford researcher. “We tell every reporter we talk to that when when you look at the measurement of the percentage of schools that do better, the same, or worse you need to present the information broken out into three categories rather than lumped together.”

“People do this all the time,” he says. “It depends on what their  agenda is.”

Zernike responded to some of her critics of her front-page story on Twitter, then apparently blocked at least one of them. In response to email requests, the NYT communications office did not make Zernike or her editors available for a phone interview.

In some senses, Zernike’s depiction of the CREDO study might not seem to make much of a difference, given the length and scope of the story and the harrowing details it includes. Who cares that she lumped two categories into one?

And yet, the CREDO study is at the center of the piece, and how it’s described justifies or contradicts the entire story. The Times can’t really write the same story if 96 percent of charters do the same or better than district schools, or even if only 4 percent of charters do worse.

Misrepresenting the research data, and then digging in and refusing to acknowledge the error, adds fuel to the bias argument, reinforces education writers’ longstanding problem handling research, and doesn’t serve readers well.

Related posts:
NYT’s Mass. Testing Story Contradicted By Other Outlets
NYT Race & Testing Piece Ignores Polling Data From Parents Of Color
With Viral “Rip & Redo” Video, Both The NYT & Success Academy Could Have Done Better
A “Punitive” Look At The NYT’s Latest Education CoverageNational Teacher Shortage?
Are Classroom Teachers Better At Scoring Tests?
NYT Includes Squishy Miami-Dade Layoff Figures
A Nagging Disconnect Between Vivid Anecdotes & Underlying Data

ABOUT THE AUTHOR

Alexander Russo

Alexander Russo

Alexander Russo is founder and editor of The Grade, an award-winning effort to help improve media coverage of education issues. He’s also a Spencer Education Journalism Fellowship winner and a book author. You can reach him at @alexanderrusso.

Visit their website at: https://the-grade.org/