ProLiteracy has created a series of research briefs on adult literacy and education. Written by scholars who have demonstrated expertise on specific topics, ProLiteracy Research Briefs were developed to help adult literacy practitioners understand ideas that have emerged from research.
“Using Data in Practice: What Does It Look Like and What Does It Take” was authored by Dr. Stephanie Cronen, Managing Researcher, American Institutes for Research, and GeMar Neloms, Principal Technical Assistance Consultant, American Institutes for Research. All briefs are edited by Alisa Belzer of Rutgers University. An excerpt of the research brief is highlighted below.
Adult practitioners share a laudable goal—to help vulnerable adults learn and improve their lives. To determine whether their learners are making progress or have achieved a specific outcome, most practitioners rely on data generated by a variety of formal and informal assessments. Practitioners may use these measures to adapt instruction as needed, for example, to identify learners who are struggling and need more intensive or differentiated instruction (Supovitz & Klein, 2003; Wayman & Stringfield, 2006). Practitioners may also find data useful for evaluating and improving instructional practices (Halverson, Prichett, & Watson, 2007; Supovitz & Klein, 2003). Learners can use data on their own performance to inform their approach towards achieving an outcome (Hamilton et al., 2009; May & Robinson, 2007; National Research Council, 2012). At the administrator level, program-wide data can be used to assess whether curricula or special initiatives are having the desired effect, and it can inform a change in course when needed (Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Marsh, Pane, & Hamilton, 2006).
To understand the types of outcomes data that may be useful to practitioners and how they may be used effectively, we first review existing theory and research on using outcomes data in practice. We then suggest implications for practice but also highlight gaps in this research.
Existing frameworks for using outcomes data in education, referred to by a variety of labels such as data-driven or data-informed decision making, are not specific to adult education. Mandinach, Honey and Light (2006) developed a commonly cited framework which conceptualizes data use as a continuum that transforms data into the actionable knowledge and understanding needed to implement effective practices. This continuum begins with collecting and analyzing data, then summarizing findings in a way that creates usable information. For example, this could be compiling math sub-scores for a program that is trying to improve instruction. This information is then synthesized to form knowledge that is useful to guide decisions and action. More specifically, a comparison of math sub-scores before and after implementing a new curriculum could be used to evaluate its success in improving learners’ math outcomes. A key assumption of this model is that practitioners need both tools and knowledge—“data literacy”—to access and effectively analyze and make use of available data.