Research Report:
Evaluating the Predictive Value of Standards-Based Formative Assessments to Inform Online Instruction and Academic Program Management

Introduction

The ability to predict student performance on end-of-year assessments is of significant importance for schools and districts. When formative and summative assessments are deliberately aligned, the formative assessment results provide educators with “predictive value” for how students will likely perform on subsequent assessments (Ainsworth, 2007, p.80). Armed with this information, educators have the ability to differentiate instruction and improve student achievement.

The use of valid assessments is also crucial for implementing student support initiatives. Data-driven frameworks that promote positive student academic and behavioral outcomes such as Multi-tiered Systems of Support (MTSS) have become embedded in many schools. Recommendations for best practices in the planning and implementation of an MTSS include emphasis on data-driven decision making, timely interventions, and consistent monitoring of student progress to improve academic and behavioral outcomes. Adherence to these recommendations hinges on the ability to integrate formative assessment data into the day-to-day operations of the school.

The Study

Given the importance of establishing the predictive value of formative assessments, our goal was to understand how well scores on StrongMind course assessments predict student performance on a high-stakes, statewide achievement test. Because allowing students more than one attempt on assessments is a common teaching practice, we also wanted to examine the extent to which multiple attempts on the in-course formative assessments impacted the ability to predict end-of-year state testing performance.

We analyzed data from 985 high-school students attending an accredited public charter school offering online courses to Arizona students in grades 6-12. In order to conduct the analysis, each student’s course assessment results were matched to a corresponding statewide achievement test (AzMERIT) score in the appropriate tested subject. Utilizing regression analysis, we examined how students’ best performance scores on the formative assessments predicted performance on the AzMERIT. We then examined how well students’ assignment grades, based on their first attempted, predicted performance on the AzMERIT.

 

Key Findings

1. Student performance on StrongMind standards-based course assignments was found to be a significant predictor of end-of-year state testing performance.
   

   These findings confirm the predictive value of StrongMind’s course assessments to inform instruction and academic program management. The ability to predict how students will perform on subsequent assessments allows educators to evaluate learning needs and make instructional modifications, ensuring students make progress toward demonstrating proficiency on state academic standards. StrongMind’s formative assessments can serve as a valuable tool for administrators as well. Knowing how many and which students may be at-risk can inform decisions related to academic program management, such as how best to allocate resources, implement school or systemwide initiatives, or take other strategic action.

   Results from this study also confirm the value of StrongMind’s in-course assessments in facilitating data driven frameworks, such as MTSS, that promote positive student academic and behavioral outcomes. By providing valid assessments, StrongMind supports partner schools in implementing proactive approaches to improving student academic outcomes by facilitating timely interventions, and consistent monitoring of student progress.

2. Students’ first attempt scores on assessments were better predictors of performance on the end-of-year assessment than students’ best attempt scores.
   

   Educators utilizing StrongMind course assessments have the ability to view students’ first and best attempt scores on each item. Obtaining data using more than one method to collect information leads to a broader and deeper understanding of the topic of interest (Flick, 2009). One advantage of digital learning environments is the ability to provide students with immediate feedback on the correctness of assessment responses and allowance of multiple attempts to answer (Bergner, Colvin, & Pritchard, 2015). This affordance of multiple attempts to answer items also results in a series of scores, or data points, that can be useful to educators. First attempt scores can inform educators in providing scaffolded feedback while subsequent attempts allow teachers to make inferences about student perseverance and learning gains.

 

 

StrongMind Standards-Based Course Assessments

StrongMind courses include assessments intentionally crafted to align to state academic standards. Using Backward Design, we break down each standard into learning objectives for both instruction and assessment purposes. The assessments, embedded at multiple points within the online course, are designed to assess the instruction of discrete learning objectives in multiple ways throughout the online course. Providing both low-stakes practice and high-stakes testing evaluation, the assessments facilitate learning, practice, critical thinking, and student proficiency.

The assessments are technology enhanced, utilizing both teacher and system grading options. System scored assessments include workbook activities, checkpoints, unit exams, and final exams:

• Workbook assignments –use targeted questions for practice and to assess student mastery of activity objectives.
• Checkpoint assessments – designed to assess student mastery of lesson objectives and are lower-stakes assessments that can help students and teachers identify gaps in knowledge that need to be filled before unit and final exams.
• Unit exams – assess mastery of unit objectives.
• Final exams – designed to assess mastery of standards and key outcomes for the course.

Teacher graded assessments include discussion board posts as well as writing and project activities.

• Discussion board posts – give students the opportunity to critically evaluate and share thoughts and ideas about the topics covered in the lesson. They promote student engagement and interaction by prompting students with questions/topics that are open-ended.
• Writing and project activities – necessitate a greater level of critical thinking and reflection on course content. These assignments require students to apply their knowledge and writing skills to demonstrate mastery of the content presented in the lesson and across lessons. These assessments result in posters, infographics, presentations, essays, experiments, and other similar deliverables.

 

DOWNLOAD THE FULL WHITE PAPER

 

Author:
Tammy McKeown, Ph.D.

 

 

REFERENCES

Ainsworth, L. (2007). Common Formative Assessments: The Centerpiece of an Integrated Standards-Bases Assessment System. In D.B. Reeves (Ed.), Ahead of the curve: The power of assessment to transform teaching and learning. Bloomington, IN: Solution Tree.

Bergner, Y., Colvin, K., & Pritchard, D. (2015). Estimation of ability from homework items when there are missing and/or multiple attempts. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK ’15). Association for Computing Machinery, New York, NY, USA, 118–125. DOI:https://doi.org/10.1145/2723576.2723582

Flick, U. (2009). An introduction to qualitative research (4th ed.). Sage Publications Ltd.