Good education policy making: Data-informed but values-driven

Broadening the definition of important educational outcomes will enable decision makers to think more clearly about what they really value.

Consider a perennial debate about the allocation of children to classrooms within schools. Should students be grouped into classrooms by achievement, or not?

Discussions typically start with predicted effects on student achievement. In mixed-achievement classes, the concern is that teachers may teach to the middle-achieving students and fail to meet the needs of both low and high performers. Another possibility is that they may devote disproportionate attention to high-achieving students, doing an even greater disservice to lower-achieving students, or they may help low-achieving students at the expense of high achievers. If students are grouped by achievement, claim advocates of this approach, teachers can target their lessons to each set of students, therefore raising the achievement of all groups.

In contrast, opponents of achievement grouping typically challenge the contention that low-achieving students will benefit from such groups. Instead, they argue, schools might consign low-achieving students to classrooms with the least skilled teachers, producing detrimental effects on achievement that will be further compounded by negative peer effects.

However, effects on achievement are not the only considerations policy makers should take into account. How, for example, will being consigned to lower-performing groups affect those students’ inclination to work hard? If students are grouped by achievement level, how might this influence their perceptions of peers with differing abilities, or their capacity to treat those peers with dignity and respect? Might achievement grouping have adverse effects on the development of the dispositions and skills needed for good citizenship? And how is this problem compounded when achievement groups correspond with other categories, such as race and socioeconomic background?

Educational goods and childhood goods

For almost 20 years — starting with No Child Left Behind in 2002 and continuing (albeit modified) with the Every Student Succeeds Act of 2015 — a narrow focus on student achievement, typically measured by standardized test scores in core subjects, has driven American education policy. Yet, as the achievement-grouping debate illustrates, we expect our schools to do more than teach students how to succeed on tests. For our students to be successful in the job market, participate effectively in the democratic process, and flourish more broadly, they need to develop capacities that go beyond those that are measured on standardized tests. However, we lack a shared vocabulary for talking about these values in a way that will enable decision makers to promote the full range of valued outcomes of education.

Educational goods include all the knowledge, skills, attitudes, and dispositions that enable a person to flourish as an adult and to contribute to the flourishing of others in a democratic society.

These outcomes, which we call educational goods, include all the knowledge, skills, attitudes, and dispositions that enable a person to flourish as an adult and to contribute to the flourishing of others in a democratic society. Although families and communities contribute to the production of educational goods, schools are the central public institution in the process. Thus, it makes sense for education policy makers to focus much of their attention on schools. Also important are positive experiences during childhood, such as play, wonder, and naive curiosity — experiences that we call childhood goods. Childhood goods may contribute to long-term flourishing, but they should also be valued for their own sake while children are in school.

Educational goods include the capacities to function in the labor market, to participate effectively in the democratic process, to make autonomous judgments about key life decisions such as occupation or religion, to develop healthy interpersonal relationships, to seek personal fulfillment, and to treat others with respect and dignity. Each of these requires various forms of knowledge and skill, along with certain attitudes and dispositions. Educational decision makers, and the public, may disagree about the degree to which schools should spend time developing these capacities (as opposed to leaving this work to families) and on the weights to place on any specific capacity. Nevertheless, the concept of educational goods gives educational decision makers at all levels — state and federal officials, district or school leaders, or teachers in individual classrooms — the language to think carefully about the broad set of valued outcomes of the educational process and how to make trade-offs among them.

Consider, for example, how a school determines what resources to devote to the arts. The current emphasis on student test scores forces supporters of the arts to make the case that greater exposure to art and music contributes to higher test scores in math and reading. Yet arguments of that type are unlikely to succeed because directly investing in those core subjects would be a more efficient way to improve students’ knowledge of math and reading. Art and music do, however, contribute distinctly to other important capacities (Heller, 2017). Attention to a broader set of educational goods, including the capacity for personal fulfillment, could change the discussion and force decision makers at all levels to make explicit what trade-offs they’re willing to make among the valued outcomes of education.

Clarifying distributive values and trade-offs

Good decision making requires paying attention not only to the level and mix of educational goods, but also to how they are distributed across students or groups of students. Yet terms such as equity and social justice, which help draw attention to these distributive concerns, do little to help decision makers think about the trade-offs involved. Too often, these terms just mean something like “Whatever I think is good and just,” masking disagreements that might be resolvable if they were brought into the open.

To bring clarity to the conversation, we recommend focusing on three distributive values — equality, adequacy, and benefiting the less advantaged. The value of equality calls for reducing inequalities among individuals, schools, or districts; adequacy is concerned with ensuring that all reach some threshold level, without concern for the potentially unequal distribution of those above the bar, and benefiting the less advantaged similarly ignores inequalities at the top of the distribution while focusing on improving the prospects for the worse off.

Some educational decisions may promote all three goals, while others may require trade-offs among them. Consider, for example, a policy specifically designed to raise the level of educational goods for less-advantaged students in a manner that would raise the level of educational goods even more for advantaged students. In that case, for the sake of benefiting the less advantaged, one might accept an increase in inequality. Moving beyond vague terms like equity to be more specific about values brings out the range of the different distributive goals and makes clear the need for trade-offs.

Moving beyond vague terms like equity to be more specific about values brings out the range of the different distributive goals and makes clear the need for trade-offs.

While clear thinking about distributive values is essential to good decision making, evidence must also be brought to bear. Consider a state in which per-pupil spending is much higher in some districts than in others. Some state policy makers may argue that the situation is “inequitable” and push for greater equalization through policies designed to increase the spending in the low-spending districts and perhaps limiting it in the high-spending districts. Should you support that approach?

The answer depends partly on your values and partly on the evidence. Assume you place a high value on the distributive principle of equality. You still need to ask, equality of what? Let’s say your ultimate concern is with equalizing educational goods rather than just equalizing funding. If so, you need to gather evidence on the reasons for, and effects of, the unequal spending in different districts. Empirical research documents that districts with greater concentrations of expensive-to-educate students or higher costs of inputs — perhaps because of high costs of living or high land prices — would have to spend more per pupil than other districts to generate a given level of educational goods. If you can confirm that such characteristics account for the spending levels in high-spending districts, you might conclude that the current situation is acceptable. If, however, you find that the unequal spending levels cannot be attributed to differences in student needs or costs of inputs, you would deem the current spending levels unfair because the resulting pattern of educational goods across districts would be unequal. In that case, it would make sense to consider corrective policies.

But what if you care most about adequacy, and not at all about equality? In that case, you would need to turn to the evidence to determine whether districts in general have sufficient resources to provide an adequate level of educational goods. If they do, you need not be concerned that some of the districts are able to generate higher levels of educational goods than others. If the low-spending districts have inadequate funding, however, you might want to raise their spending up to an adequate level, while leaving the high-spending districts alone. Only if you value both adequacy and equality would you want to support policies that would raise the spending in the low-spending districts while limiting it in high-spending districts.

Accounting for other values

Producing educational goods, and distributing them well, are arguably the main priorities of education decision makers. Educational decisions are not made in a vacuum, however, and other values are likely to influence them. In addition to childhood goods, which we have already mentioned, other values that can enter the picture include respect for the democratic process, goods such as transportation and health care that might have to be foregone with greater spending on education, freedom of choice of residence and occupation, and parents’ legitimate interests in their own children.

Respect for the democratic process is relevant in school funding decisions, as is the value of spending on other goods. Values related to adults’ freedom to choose their occupation and residential locations might constrain decisions related to recruiting teachers and integrating schools, racially and economically. Most education policy makers would be unwilling, for example, to force people to move homes in order to integrate neighborhood schools, or to conscript teachers into particular schools, even if such policies were legal. Pursuit of educational goals must take place within a framework that respects adults’ freedom of occupation and residence.

Parents’ interests in their own children are widely understood to be relevant in policy discussions about charter schools and other choice programs. The question there for policy makers is how to trade off the interests of individual parents against the broader public interests in the education system as a whole. Parents’ interests can limit the ability of education policy makers to generate equal educational goods across a district or a state when wealthy parents have both the right and the ability to shift their children to private schools. The greater the proportion of parents who reject the public schools, for example, the harder it may be to generate political support for adequate levels of funding for the public system.

To see how our concept of childhood goods might matter, consider the phenomenon of bullying. In some cases, being the victim of bullying may damage a child’s academic performance. In other cases, if the victim reacts to bullying by retreating into her shell, avoiding social situations, and concentrating on her schoolwork, her academic performance might improve. However, the misery inflicted on the victim provides a compelling reason for a school to attend to bullying, regardless of its impact on her achievement. Second, consider debates about “drill and kill” strategies for teaching literacy and mathematics. Even if they work in some situations to improve achievement, principals and teachers should take into account the adverse effects of such strategies on the daily experiences of the child. That is, childhood goods matter.

Combining values and evidence

We must move beyond the current enthusiasm for data-driven decision making in the field of education policy. Although good education decision making is informed by good evidence, it should be driven by good values. The concept of educational goods provides a framework that decision makers can use to organize their own thinking about both values and evidence. Only with valued outcomes in mind can decision makers make good decisions among feasible policy options. Values alone, however, do not suffice. Decision makers also need to make use of high-quality evidence to determine the extent to which specific actions are likely to promote the valued outcomes. Such evidence must go beyond easy-to-measure outcomes such as student test scores and, instead, shed light on the broad set of relevant values.

And so we end where we started: with debates about achievement grouping. As is common in educational debates, the existing empirical studies on achievement grouping vary in rigor and in results. However, almost all of the rigorous research concerns effects on test scores — both the averages, and their distribution. Social science researchers have not paid much attention to the effects of achievement grouping on democratic competence, the tendency to treat others as equals, or on childhood goods. Because social scientists do not have good measures of those outcomes, research on what causes and impedes their development is limited. Without better measures, decision makers have to judge as best they can until social scientists develop measures of a wider range of the outcomes that matter and use them to investigate the likely effects of policies.

Reference

Heller, R. (2017). On the goals and outcomes of arts education: An interview with Lois Hetland. Phi Delta Kappan, 98 (7), 15-20.

Note: This article is based on the authors’ book, Educational Goods: Values, Evidence and Decision Making (University of Chicago Press, 2018).

Citation: Brighouse, H., Ladd, H., Loeb, S. & Swift, A. (2018). Good education policy making: Data-informed but values-driven . Phi Delta Kappan, 100 (4), 36-39.