PDK_100_5_Zhao_Art_554x350px

 

When evaluating an education initiative, look beyond the average to consider its effects on all students. 

 

In the same way that different people can have different responses to the same medical treatment because of individual variability in genes, environment, and lifestyle, students can have different reactions to the same educational treatment because of their individual conditions. In other words, what works for one student can hurt another. Because of this, interventions in education, be they policy or practice, can have both positive, intended, and desirable effects and, at the same time, adverse and unwanted side effects (Zhao, 2018). In the December 2018 Kappan, Rick Hess and Michael McShane describe such side effects as “happy (and not so happy) accidents” and consider how they played out in the education reforms of the Bush-Obama era. I believe the idea can also apply to the debate about school vouchers. 

In the 1950s, the free-market economist Milton Friedman, a Nobel winner in his field, proposed turning public education into a free market through school choice (Fowler, 2002; Friedman, 1955; Greene, 2011). The debate has become increasingly fierce over the years as states have implemented various forms of school choice, such as public charter schools, magnet schools, and cross-district choices, resulting in greater levels of privatization and marketization (Ravitch, 2013). The election of Donald Trump as U.S. president in 2016 and his appointment of Betsy DeVos as secretary of education have raised the stakes because, to many, their strong support for vouchers amounts to using public funds to support private, and even religious, schools (Willingham, 2017). 

Friedman, sometimes called the grandfather of vouchers, and his wife Rose Friedman launched the Friedman Foundation for Educational Choice (now EdChoice, where, as it happens, McShane is the director of national research) in 1996 to “promote school choice as the most effective and equitable way to improve the quality of K-12 education in America” (Forster, 2016). Opponents, however, argue that, among other things, school choice negatively affects public schools and results in a host of negative consequences such as distortion of the purpose of education, decreased civic engagement and commitment to the common good, and greater social segregation (Abowitz & Stitzlein, 2018; Labaree, 2018; Ravitch, 2013).  

Reviewing the evidence 

Setting aside the philosophical objections to and defenses of choice, as important as they may be, let’s consider whether school choice even leads to better achievement for participating students. Since the 1990s, the two sides have used the same imperfect body of empirical evidence to argue for and against choice. The central point of disagreement is the size of the effect of choice on student achievement, with proponents finding more significant positive effects than independent researchers (Carnoy, 2001; Dynarski, 2016; Rouse & Barrow, 2009). 

For example, the May 2016 EdChoice report A Win-Win Solution: The Empirical Evidence on School Choice (4th ed.) systematically reviews publicly available empirical studies that used the “gold standard” of research methods — random assignments of participants. The report’s author, Greg Forster (2016), concludes that “the empirical evidence shows that choice improves academic outcomes for participants and public schools, saves taxpayer money, moves students into more integrated classrooms, and strengthens the shared civic values and practices essential to American democracy” (p. 1). Even the most controversial form of school choice, vouchers to send children to private schools, is a “win-win” solution for everyone, he concludes. Similarly, voucher advocates at the University of Arkansas have found a positive effect of vouchers after “a meta-analytic and systematic review” of “the participant effects of private vouchers across the globe” (Shakeel et al., 2016, p. 1).  

Other researchers found much less significant effect. For example, Cecilia Rouse and Lisa Barrow (2009) reviewed the empirical evidence and concluded that the achievement gains for students in voucher programs are relatively small, with most not statistically different from zero. The significant effects reported in both reports have also been challenged. Christopher Lubienski pointed out that the reported significant effects of voucher programs in the two reports resulted from ideological bias and flawed research: “Both reports are marred by a number of serious problems and errors, including misrepresentations of the research literature, a failure to acknowledge the limitations of their approaches, not addressing the shortcomings of the theoretical underpinnings of vouchers, and the use of methods that bias the selections of the studies they utilize” (Lubienski, 2016, p. 1).  

In any case, this debate has focused to date only on the average effect of school choice on the students who participate in it. More helpful, however, would be a discussion of the varied effects of choice-based initiatives. In other words, even if such programs have a positive effect on average (which is under dispute), we still need to ask whether and to what extent they might also be harming certain students. And in fact, while there may be good reason to challenge Forster’s selection and analysis of previous research studies, it’s worth noting that even his report, though written by an advocate for school choice, offers clear evidence of such negative side effects.  

Different students, different effects 

Forster’s claim about the academic benefits of choice for participants is based on 18 studies using random-assignment methods. He concludes that “[T]his body of evidence shows that school choice benefits students” because: 

Fourteen studies find positive effects on school choice participants: six find choice had a positive effect across all students participating and another eight find choice had a positive effect on some student groups and no visible impact on other students. Two studies find no visible effect from choice. Two studies on Louisiana’s voucher program find that it had a negative effect. (Forster, 2016, p. 10) 

But Forster’s own data do not support his sweeping conclusion. Only six out of 18 studies found a positive effect across all students. Eight other studies Forster counts as having a positive effect actually had mixed results: positive for some and no visible effect on others. Further, two studies found an outright negative effect.  

Likewise, the University of Arkansas report finds that the effect sizes of voucher programs varied a great deal across countries and locations. Overall results “indicate that school vouchers have positive effects in both reading and math, but that these impacts are largest in programs outside of the U.S.” (Shakeel et al., 2016, p. 33). Further, they note that studies found no effect on reading in the U.S. but large positive effects in other countries, with the largest in Bogota, Colombia. Judging from the evidence, then, the effects of school choice, like all education interventions, is dependent on individual characteristics and contexts. 

If school choice expands its reach, we must carefully study its effects and side effects on students with a wide variety of characteristics. 

The contexts in which a treatment is implemented can moderate its effect or even cause adverse side effects. To date, there has been little empirical research into the ways in which contextual factors interact with school choice programs, but there has been no lack of speculation. Forster, for example, suggests that the significant negative impact of the Louisiana voucher program found by two studies (Abdulkadiroglu et al., 2015; Mills & Wolf, 2016) is the result of “poor program design and fear of future action from hostile regulators” that led to the low participation of private schools (Forster, 2016, p. 12), with participating private schools being more likely to be lower performers. Mark Dynarski (2016) argues that the negative effect may be due to the decline of quality of private schools and the narrowing gap in quality between public and private schools in Louisiana. The University of Arkansas report (Shakeel et al., 2016) found that publicly funded vouchers delivered larger effect sizes than privately funded ones, but the difference is largely influenced by the outsized effect of one publicly funded program in Bogota, Colombia. 

There are certainly other contextual variables — such as population density, transportation, culture, and the race, ethnicity, and religion of participating students — that can affect the degree to which school choice has a positive or negative effect on students. There is no reason, then, to conclude that school choice is a panacea. If it helps some students, it may hurt others. Parents and policy makers should neither embrace nor reject school choice without understanding its full range of effects and side effects. 

Tankers and leapers 

The effect of attending private schools supported by school choice varies a great deal among participating students and across cohorts. For instance, Martin Carnoy (2001) found that the standardized test performance of some students, termed “tankers,” fell more than 1.5 standard deviations when they switched to a private school, while the score of “leapers” rose by more than two standard deviations. Different sites saw inconsistent patterns of effects across cohorts, with the Washington, D.C., program being the only site that showed relatively consistent achievement gains across grades (Carnoy, 2001).  

What led to the decline among “tankers” and the improvements among “leapers”? Why was there so much inconsistency across cohorts? There is evidence that individual variability played a role.  

So which groups are most likely to experience negative side effects in a voucher program? Researchers from the University of California at Irvine found that a voucher experiment in New York City “did little to influence student achievement” but had “a small negative effect for a small group of high-performing students after the first two years of the program” (Domina & Penner, 2013, p. 24). A similar finding was also reported by another group of researchers (Barnard et al., 2003). The New York City experiment was also found to have a negative effect on non-Black students’ math achievement (Krueger & Zhu, 2004). Similarly, research into a voucher program in Charlotte, N.C., showed that Black students with a mother who dropped out of high school experienced negative effects, while the effect was positive for those whose mothers graduated from college and who lived in a two-parent home (Cowen, 2008).  

There may be adverse effects on students with other characteristics, but these studies do not systematically report them, perhaps because the researchers did not collect the relevant individual data or because they did not look for these patterns with the data they had. Because many voucher programs were designed to serve low-income, minority students in urban areas, the student population studied is more homogeneous than American schools as a whole. Thus, if school choice expands its reach, we must carefully study its effects and side effects on students with a wide variety of characteristics. 

Parent characteristics and choice 

Even if school choice leads to positive effects for all students who participate, not all parents are willing or able to participate. Certain characteristics of parents have been found to affect their decision to accept vouchers and send their children to private schools, further complicating our understanding of the effects of vouchers.  

In his analysis of the Milwaukee Parental Choice Program, John Witte (2001) noted that Black and Latinx parents are more likely than White parents to apply to voucher programs like Milwaukee’s, female parents are more likely to apply than males, lower-income families are more likely to apply than those of higher incomes, and mothers with higher levels of education are more likely to apply than less educated mothers. William Howell and Paul Peterson (2006) made both similar and different observations in The Education Gap: Vouchers and Urban Schools. In their studies, families with lower incomes, families of Black students, and mothers with less education were more likely to decline vouchers. And a recent study of the voucher program in Charlotte found that parents who attended some college and those living in two-parent homes were more likely to participate, while Black parents were less likely to participate (Cowen, 2008).  

In sum, although the nature of the effect is unclear and further study is needed to uncover a clear pattern, race, gender, education level, and family structure all appear to influence student participation in choice initiatives. Thus, certain children are more likely than others to experience the effects of choice, whether positive or negative. 

Side effects include . . .  

As much as we might want to seek a perfect solution for all students, one student’s medicine may very well be another one’s poison. As students’ characteristics and education treatments interact, negative side effects may occur. Funding private schools with public dollars probably does not affect all students positively in a uniform fashion. To date, studies of school voucher programs have found their effects to vary among different populations of students. 

Moreover, besides the side effects resulting from the interactions between students’ characteristics and education treatments, side effects also occur because of the broad range of desirable and potentially competing education outcomes. So far, evidence of the effects of voucher programs has been limited to a narrow set of outcomes such as academic achievement. Little, if any, empirical evidence has been collected concerning other equally important outcomes of schooling, such as preparing students for civic engagement and betterment of a shared society (Abowitz & Stitzlein, 2018; Labaree, 2018). Thus, we do not know their effects, negative or positive, on other important outcomes. It is, however, reasonable to believe that voucher programs and other forms of privatization of education can have negative side effects on individual students, the public school system, and the society (Labaree, 2018).  

References 

Abdulkadiroglu, A., Pathak, P.A., & Walters, C.R. (2015). School vouchers and student achievement: First-year evidence from the Louisiana Scholarship Program. Cambridge, MA: National Bureau of Economic Research. 

Abowitz, K.K. & Stitzlein, S.M. (2018). Public schools, public goods, and public work. Phi Delta Kappan, 100 (3), 33-37. 

Barnard, J., Frangakis, C.E., Hill, J.L., & Rubin, D.B. (2003). Principal stratification approach to broken randomized experiments: A case study of school choice vouchers in New York City. Journal of the American Statistical Association, 98 (462), 299-323. 

Carnoy, M. (2001). School vouchers: Examining the evidence. Washington, DC: Economic Policy Institute. 

Cowen, J.M. (2008). School choice as a latent variable: Estimating the “complier average causal effect” of vouchers in Charlotte. Policy Studies Journal, 36 (2), 301-315.  

Domina, T. & Penner, E.K. (2013). Distributional effects of a school voucher program: Evidence from New York City. http://emilykpenner.com/epkwp/wp-content/uploads/2013/07/Bitler_Domina_Penner_Hoynes_2013_NYC_Voucher.pdf. 

Dynarski, M. (2016). On negative effects of vouchers. Washington, DC: Brookings. www.brookings.edu/research/on-negative-effects-of-vouchers. 

Forster, G. (2016). A win-win solution: The empirical evidence on school choice. Indianapolis, IN: Friedman Foundation for Educational Choice.  

Fowler, F.C. (2002). Introduction: The great school choice debate. The Clearing House, 76 (1), 4-7.  

Friedman, M. (1955). The role of government in education. New Brunswick, NJ: Rutgers University Press. 

Greene, J.P. (2011). Why America needs school choice. New York, NY: Encounter Books. 

Hess, F.M. & McShane, M.Q. (2018). The happy (and not so happy) accidents of Bush-Obama school reform. Phi Delta Kappan, 100 (4), 31-35. 

Howell, W.G. & Peterson, P.E. (2006). The education gap: Vouchers and urban schools. Washington, DC: Brookings Institution Press. 

Krueger, A.B. & Zhu, P. (2004). Another look at the New York City school voucher experiment. American Behavioral Scientist, 47 (5), 658-698.  

Labaree, D.F. (2018). Public schools for private gain: The declining American commitment to serving the public good. Phi Delta Kappan, 100 (3), 8-13. 

Lubienski, C. (2016). Review of A Win-Win Solution and The Participant Effects of Private School Vouchers across the Globe. Boulder, CO: National Education Policy Center. 

Mills, J.N. & Wolf, P.J. (2016). The effects of the Louisiana Scholarship Program on student achievement after two years. New Orleans, LA: Education Research Alliance for New Orleans. 

Ravitch, D. (2013). Reign of error: The hoax of the privatization movement and the danger to America’s public schools. New York, NY: Knopf. 

Rouse, C.E. & Barrow, L. (2009). School vouchers and student achievement: Recent evidence and remaining questions. Annual Review of Economics, 1 (1), 17-42. 

Shakeel, M.D., Anderson, K.P., & Wolf, P.J. (2016). The participant effects of private school vouchers across the globe: A meta-analytic and systematic review. Little Rock, AR: University of Arkansas Department of Educational Research.  

Willingham, A.J. (2017, May 24). How to make sense of the school choice debate. CNN. www.cnn.com/2017/05/24/us/school-choice-debate-betsy-devos/index.html. 

Witte, J.F. (2001). The market approach to education: An analysis of America’s first voucher program. Princeton, NJ: Princeton University Press. 

Zhao, Y. (2018). What works may hurt: Side effects in education. New York, NY: Teachers College Press. 

 

NOTE: This article was adapted from What Works May Hurt: Side Effects in Education by Yong Zhao (Teachers College Press, 2018). 

 

Citation: Zhao, Y. (2019). Side effects in education: Winners and losers in school voucher programs. Phi Delta Kappan, 100  (5), 63-66.

ABOUT THE AUTHOR

default profile picture

Yong Zhao

YONG ZHAO  is Foundation Distinguished Professor at the University of Kansas School of Education, in Lawrence. His most recent book is Teaching Students to Become Self-Determined Learners .