FB_K1704_67_Crocco

 

Students need to be able to assess the trustworthiness of evidence — especially when deliberating thorny public policy issues.

By Margaret Crocco, Anne-Lise Halvorsen, Rebecca Jacobsen, and Avner Segall

Evaluating evidence. Defending claims. Making arguments. Although these words might conjure images of a courtroom, we’re referring to today’s classroom. Drawing upon evidence has become a central focus of recent curricular reforms across all subject areas. For example, the Common Core State Standards English language arts document, which includes literacy standards for history/social studies and other subjects, uses the word “evidence” 135 times.

But what is evidence? Is all evidence created equal? Despite the centrality of evidence use in recent curriculum guidelines, we don’t yet know as much as we should about how students understand and use evidence, especially when debating public policy issues.

These same issues have recently become a topic of national discussion. In the wake of the 2016 U.S. presidential election, trust and distrust of evidence (and of the candidates and their surrogates) played a big role in citizens’ attitudes and judgments. Journalists and pundits’ explanations of the election results drew on terms such as fake news, bubbles, post-truth, and social media echo chambers. These national concerns have direct relevance for classrooms today as teachers grapple with students’ complex and often competing understandings of evidence and evidence use.

Acquiring and processing evidence

Psychologists have found that when people consider evidence during the process of making up their minds about candidates, issues, and other questions they care about, they tend to engage in motivated reasoning (Clark & Avery, 2016). In other words, we process information in ways that reinforce our existing beliefs, values, and ideas. We spend more time critiquing information that challenges our views, and we seek out information that reinforces our views. These reasoning strategies are not unique to politics; the same dynamics affect a whole host of other judgments every day. When it comes to evidence, our “filters” play a big role. Many of us get news from social media, such as Facebook and Twitter, via newsfeeds customized to our personal likes and dislikes; decades ago, most of us watched the same three channels on TV or read the same regional or national newspapers.

Which brings us to the issue of real and fake news. When it comes to discriminating between the two, most people just aren’t very good at it. Shortly after the 2016 U.S. presidential election, Stanford University researcher Sam Wineburg and colleagues shared findings from a study of nearly 8,000 students, from middle school to college age, about their ability to distinguish between real news written by journalists and advertisements sponsored by other groups to mimic real news. Wineburg and McGrew (2016) reported that 80% to 90% of the teenagers they studied had difficulty distinguishing between real and fake news.

We believe that teaching young people how to thoughtfully engage in accessing, evaluating, and using sources and information — particularly in this era of information proliferation — is crucial in a democracy. This concern drove our study.

Analyzing students’ use of evidence: Our approach

Since 2015, we’ve examined students’ use of evidence in secondary social studies classrooms as students deliberate about contemporary public policy issues — a line of inquiry we think is relevant to the issue of real and fake evidence (Crocco et al., 2016a, 2016b). Working with students from three Michigan high schools, we sought to learn how students evaluated the relative trustworthiness and persuasiveness of various forms of legitimate evidence and how (if at all) they drew on various sources of evidence in public policy deliberations. We hypothesized that even when faced with genuine evidence, students might find some forms more or less compelling in shaping their thinking about a topic.

Research on how adults use evidence indicates that perceptions of the trustworthy and persuasive nature of evidence vary from person to person and from situation to situation. In other words, the acceptability and effectiveness of evidence depends on the context. Adults assess the credibility of evidence according to “the nature of the case, the type of audience, the prevailing ‘rules of evidence,’ and the persuasiveness of the analyst” (Majone, 1989, p. 48).

We suspect context matters even more for adolescents who stand at a pivotal point in their psychological and social development (Flanagan et al., 2010; Levine, 2013). Adolescents are developing their views on social and political matters. We were particularly interested in whether an adolescent’s social trust — or faith in others — might influence their views on evidence.

Seven types of evidence

Although policy makers and curriculum developers frequently refer to evidence, they are not always precise about what counts as evidence and how different types of evidence may be more or less trusted. Scholars identify at least seven types of evidence that are typically invoked during public policy discussions to support or refute viewpoints (Asen et al., 2013; McDonnell & Weatherford, 2013). These include statistical data, research, expert judgment, personal experience, anecdote/secondhand experience, example, and law/policy.

In our research, we selected 10 students in each school to represent a mix of sociocultural identities based on race, class, and gender. The students also represented a mix of profiles on social, personal, and political trust measures, which we determined by a survey we administered that drew on items from other surveys (Campbell, 2007, 2008; Flanagan, 2013; Jacobs, Cook, & Delli Carpini, 2009; Zukin et al., 2006). In our student interviews, we were interested in learning whether students with different sociocultural identities and trust profiles (that is, composite pictures based on their answers to the social, personal, and political trust questions) would evaluate evidence differently in terms of their trustworthiness and persuasiveness and whether their evaluations would change once the evidence was placed in the context of a real public policy issue.

For the first context, we focused on evidence associated with Brown v. Board of Education, which most high school students encounter in studying U.S. history. We asked students to rank the seven types of evidence by their trustworthiness and persuasiveness using both the abstract definitions below and contextualized examples from Brown:

  • Statistical data: Measurable information systematically collected to describe a set of conditions or trends, often presented in a numerical format.
    Brown example: A graph showing Educational Attainment by Race: Percent of Students Completing Four Years or More of High School, which shows disparities in graduation rates between blacks and whites.
  • Research: Findings resulting from systematic analysis of information, guided by purposeful research questions and method, conducted by a trained researcher.
    Brown example: Kenneth and Mamie Clark’s “doll tests,” which elicited children’s beliefs about race.
  • Expert judgment: The knowledge or perspective provided by someone with the education background, work experience, or credentials directly related to the issue.
    Brown example: A professor says segregated schools damage black children.
  • Personal experience: Firsthand knowledge, skills, or perspective derived from direct observation of or participation in events or activities.
    Brown example: Testimony from a teacher about the poor conditions and lack of access to up-to-date textbooks in the segregated school in which she taught.
  • Anecdote/secondhand experience: Retelling the perspective of someone else.
    Brown example: A student’s short description of the deplorable conditions in her segregated high school.
  • Example: A specific case or incident used to illustrate a point.
    Brown example: Photos of two high schools, one black and one white.
  • Law/policy: Rules and regulations that permit or prohibit particular actions, behaviors, or programs.
    Brown example: The U.S. Constitution.

We observed the students as they completed their rankings and asked them to think aloud as they made their choices. We then asked follow-up questions to better understand each student’s reasoning process.

Just as sociocultural identity influences how adolescents read, interpret, and respond to U.S. history (Epstein, 2008), we found that adolescents’ evaluation of evidence is also influenced by these characteristics but not as dramatically as we anticipated. What was less expected was that students shifted their evaluations of the trustworthiness and persuasiveness of evidence depending on whether they were making these assessments in the abstract or in the context of the Brown case. In fact, most students (19 out of 30 students) rank-ordered the types of evidence differently depending on whether the evidence was in the abstract or in context. For example, students ranked research and statistical data higher when considering the abstract descriptions. However, when ranking the contextual evidence (the Brown examples), students said emotional and personal resonance with different types of evidence, such as personal experience and anecdote, played a key role.

Although there was little indication of the effect of social trust in students’ engagement with evidence during these exercises, social trust became more salient during the public policy deliberations in which students exhibited greater faith in evidence drawn from social agents, groups, and the assertions of classmates. This was not surprising given that classroom interactions involve what has been called “situated cognition” (Wortham, 2001) and “epistemic agency” (Dotson, 2014) that is closely tied to student sociocultural identity and social trust.

Using evidence in policy deliberations

After the evidence-ranking activities, we then sought to determine if and how students would use the seven kinds of evidence in the second context of public policy deliberations. In collaboration with teachers, we selected two topics they felt would interest students — immigration reform and Internet privacy. Teachers spent two class sessions on each topic, involving students in discussion, debate, and deliberation. These three related forms of shared inquiry motivate students to dig deeper into subject matter and arrive at well-considered conclusions about complicated questions. Scholars have high regard for these approaches, which they consider “authentic intellectual work” (King, Newmann, & Carmichael, 2009), but research (Hess, 2004) also suggests that these practices are rare in social studies classrooms.

The Common Core’s emphasis on evidence, claims, and argumentation may increase the likelihood that these practices have more currency in K-12 classrooms. Assessments tied to the Common Core curriculum (for example, PARCC and Smarter Balanced) now include guidance for teachers on conducting classroom discussions.

Discussion, deliberation, debate: Defining the terms

The distinctions among these terms are straightforward. Discussions are sustained verbal student-to-student exchanges in response to open-ended questions typically based on a text (for example, what did Thomas Jefferson, a slaveholder, mean when he wrote “all men are created equal” in the Declaration of Independence?). Deliberation is a type of discourse in which participants pay attention to reasons, inclusivity, respect, public spiritedness, and finding common ground through opinion revision. Debates are structured differently, with either individuals or teams arguing for or against a certain proposition (for example, should physician-assisted suicide be legal?).

Deliberating a contemporary public policy issue gives students a unique opportunity to bring their values, opinions, and judgments to bear on topics they find interesting. Because public policy deliberations deal with current questions, the varieties of evidence that teachers have at their disposal are more extensive than what is typically available for discussing or debating historical topics.

Two deliberations: Immigration and Internet privacy

We modeled our approach to the deliberations on materials produced by the National Issues Forum (2011). Working with teachers, we created packets of age-appropriate evidence incorporating the seven types of evidence.

In the deliberation on immigration, the teachers posed the following question: Which of the following positions do you think U.S. policymakers should focus on regarding immigration to this country?

  • We should welcome anyone who wants to come into the country legally.
  • We should prevent undocumented immigrants from entering the country and deport all the undocumented immigrants already here.
  • We should allow only people with specific job-related skills to immigrate here.

In the deliberation on Internet privacy, students also were given an evidence packet but were asked to answer a question that was more open-ended: Should search engines like Google and social media sites like Facebook be permitted to monitor, track, and share users’ personal data (for example, searches, pages visited) with advertisers or does this violate personal privacy?

Even after teachers directed them to use evidence packets in building their arguments, students largely ignored the evidence.

The deliberation experience surprised both the researchers and teachers. Even after teachers directed them to use the evidence packets in building their arguments, students largely ignored the evidence we had developed. When students used evidence at all, they tended to bring in outside information. This was especially true for the Internet privacy deliberation, in which students gave little credence to evidence that suggested the potential dangers around privacy that stem from regular use of Google and Facebook. Students discounted examples of how personal information such as Facebook posts might be used against them when they were older and looking for a job or how pricing for products or services found online might be adjusted based on personal information found on the Internet. If students gave this evidence any weight at all, they seemed convinced that giving up their privacy was a small price to pay to gain access to social media and powerful search engines. Since no one in their classes could recount anything negative that had happened to them as a result of using Google or Facebook, they were satisfied that no threat existed, despite many examples of others’ negative experiences in the evidence provided.

Recommendations for teaching good evidence use

Our classroom observations, as well as student and teacher interviews, have led us to several conclusions. First, many teachers don’t have experience leading discussions and deliberations and need more training in these approaches. Second, many teachers are unfamiliar with various forms of evidence and may focus narrowly on a single type of evidence (for example, asking students to find three statistics to support an argument). Third and perhaps most challenging, adolescents don’t find all evidence equally trustworthy and persuasive, and their beliefs about the trustworthiness and persuasiveness of evidence shift depending on the topic. This third insight complicates the work of teaching with evidence.

In the context of education reforms promoting the use of evidence, making claims, and building arguments, we offer a set of recommendations.

Give students more opportunities to evaluate evidence. Students need more opportunities to examine the credibility of an author or a publisher, justify their evaluation, determine the intended audience, corroborate sources, and critically analyze what they read — not just in social studies but in all subject areas. We found these skills particularly important in public policy issue deliberations on topics already familiar to students. We also found that students needed more opportunities to consider opposing arguments. That’s why we support bringing more deliberation into classrooms since students will develop skills in using evidence only through regular practice.

People spend more time critiquing information that challenges their views, and they seek out information that reinforces their views.

Help students recognize the factors that influence how they evaluate evidence. Helping students understand how their background characteristics and political viewpoints might influence their position on public policy issues and the kinds of evidence they tend to find compelling is critical. By recognizing the subtle ways bias can creep into one’s perspectives, students may be more likely to consider opposing viewpoints and reconsider their own perspectives. By understanding their bias toward particular kinds of evidence, they may be more willing to consider different kinds of evidence, even if they decide these kinds of evidence are ultimately not compelling.

Be aware of the role of social trust in classrooms. Recognize that students, like adults, may find some forms of evidence and certain interpretations of evidence more compelling based on their social trust.

Go beyond “good” vs. “bad” evidence. Help students understand that the quality and credibility of evidence cannot be neatly separated into “good” and “bad.” Instead, students should understand that evidence can fall along a continuum of credibility and that sources can be more or less persuasive depending on the audience. What’s considered “good” depends on the context, audience, and sociocultural identities. For example, for some audiences, putting a personal face to an issue may be most persuasive. In other situations, using statistics and data may be better suited to demonstrating the need for a solution to a problem. Students should practice evaluating evidence along multiple dimensions, not just whether it is factual or relevant.

Introduce students to a variety of evidence types. Too often, students said assignments give them limited types of evidence. Classroom public policy issue deliberations should introduce students to a range of sources. For example, instead of (or in addition to) asking students to provide three statistics to support an argument, teachers might ask them to include an expert’s take on the issue or a specific case study to illustrate the point.

Enlist school and administrator support. Some schools, like one of those with which we worked, have made this issue a whole school effort, adopting common language and shared understandings of best practices in using evidence across school subjects. School and administrator support can help teachers across the disciplines develop coherent messages that will enhance student learning.

Getting smart about evidence use

The challenges in using evidence may be greatest in secondary social studies classrooms engaged in deliberating public policy issues. However, challenges also may arise in teaching historical topics, interpreting fiction and nonfiction texts in English language arts, and drawing conclusions about science, especially on topics such as global climate change. In each of these domains, how an individual reasons from evidence to claims to arguments is often influenced by both rational and emotional factors, elements of one’s identity, and values and belief systems. By keeping in mind the complicated ways in which evidence can be deployed, teachers can bring more higher-order thinking and student-centered learning into their schools and classrooms.

References

Asen, R., Gurke, D., Connors, P., Solomon, R., & Gumm, E. (2013). Research evidence and school board deliberations: Lessons from three Wisconsin school districts. Educational Policy, 27 (1), 33-63.

Campbell, D.E. (2007). Sticking together: Classroom diversity and civic education. American Politics Research, 35, 57-78.

Campbell, D.E. (2008). Voice in the classroom: How an open classroom climate fosters political engagement among adolescents. Political Behavior, 30 (4), 437-454.

Clark, C. & Avery, P. (2016). The psychology of controversial issues discussions: Challenges and opportunities in a polarized, post-9/11 world. In W. Journell, (Ed.) Reassessing the social studies curriculum in a polarized world (pp. 109-121). Lanham, MD: Rowman and Littlefield.

Crocco, M., Segall, A., Jacobsen, R., Halvorsen, A., Schmitt, A., & Frasier, A.S. (2016a, November). Facilitating evidence-based discussions in secondary social studies classrooms. Presentation at the Annual Meeting of the College and University Faculty Assembly of the National Council for the Social Studies, Washington, D.C.

Crocco, M., Segall, A., Jacobsen, R., & Halvorsen, A. (2016b, April). High school students’ use of social trust and evidence. Presentation at the Annual Meeting of the American Educational Research Association, Washington, D.C.

Dotson, K. (2014). Conceptualizing epistemic oppression. Social Epistemology, 28 (2), 115-128.

Epstein, T. (2008). Interpreting national history: Race, identity, and pedagogy in classrooms and communities. New York, NY: Routledge.

Flanagan, C. (2013). Teenage citizens: The political theories of the young. Cambridge, MA: Harvard University Press.

Flanagan, C., Stoppa, T., Syvertsen, A., & Stout, M. (2010). Schools and social trust. In L. Sherrod, J. Torney-Purta, & C. Flanagan (Eds.), Handbook of research on civic engagement in youth (pp. 307-331). Hoboken, NJ: John Wiley & Sons.

Hess, D. (2004). Discussion in social studies: Is it worth the trouble? Social Education, 68 (2), 151-154.

Jacobs, L.R., Cook, F.L., & Delli Carpini, M.X. (2009). Talking together: Public deliberation and political participation in American life. Chicago, IL: University of Chicago Press.

King, B., Newmann, F., & Carmichael. (2009). Authentic intellectual work: Common standards for teaching social studies. Social Education, 73 (1), 43-49.

Levine, P. (2013). All together now: Collaboration and innovation for youth engagement. Medford, MA: CIRCLE.

Majone, G. (1989). Evidence, argument, and persuasion in the policy process. New Haven, CT: Yale University Press.

McDonnell, L.M. & Weatherford, M.S. (2013). Evidence use and the Common Core State Standards movement: From problem definition to policy adoption. American Journal of Education, 120 (1), 1-25.

National Issues Forum. (2011). Immigration in America: How do we fix a system in crisis? Bloomington, IN: Author.

Wineburg, S. & McGrew, S. (2016, November 1). Why students can’t Google their way to the truth. Education Week. www.edweek.org/ew/articles/2016/11/02/why-students-cant-google-their-way-to.html

Wortham, S. (2001). Interactionally situated cognition: a classroom example. Cognitive Science, 25, 37-66.

Zukin, C., Keeter, S., Andolina, M., Jenkins, K., & Delli Carpini, M. (2006). A new engagement? Political participation, civic life, and the changing American citizen. Oxford, UK: Oxford University Press.

MARGARET CROCCO (croccom@msu.edu) is a professor and chairperson of the Department of Teacher Education, ANNE-LISE HALVORSEN (annelise@msu.edu) is an associate professor in the Department of Teacher Education, REBECCA JACOBSEN (rjacobs@msu.edu) is an associate professor in the Department of Educational Administration, and AVNER SEGALL (avner@msu.edu) is a professor in the Department of Teacher Education, all at Michigan State University, East Lansing, Mich.

Originally published in April 2017 Phi Delta Kappan 98 (7), 67-71.

© 2017 Phi Delta Kappa International. All rights reserved.