Students using ASSISTments, developed by Worcester Polytechnic Institute, saw significant gains in standardized test scores compared to peers in control group.

MENLO PARK, Calif., Oct. 24, 2016 /PRNewswire/ — A new study from SRI Education, a division of SRI International, found that a web-based mathematics homework intervention called ASSISTments, developed by Worcester Polytechnic Institute (WPI), made a positive impact on students’ mathematics achievement. In schools where seventh-grade students and their teachers used ASSISTments, students achieved higher standardized mathematics test scores, compared with peers in schools that did not use ASSISTments.

The study was recently published in the journal AERA Open by the American Educational Research Association, and was supported by a $3.5 million grant from the Institute of Education Sciences (IES) of the U.S. Department of Education to SRI, WPI, and the University of Maine.

To evaluate the efficacy of ASSISTments for mathematics homework, SRI conducted a rigorous, randomized control trial in the state of Maine, which, since 2002, has provided every seventh-grade student with a laptop. SRI examined test scores of nearly 3,000 seventh graders at 43 public schools in Maine. Schools in the test group and a control group were matched in terms of demographics and socio-economic status.

Standardized test scores of students whose schools used ASSISTments were compared to those of students whose schools were not using the system and were found to be significantly higher. Students with low prior mathematics achievement, in particular, benefited the most. (This study used the TerraNova tests, which span kindergarten through 12th grade.)

In recent years, questions have been raised about the value of existing school practices for assigning, supporting and reviewing homework. This study demonstrates that schools can improve the value of homework by using an online tool for students and teachers, combined with teacher training on how to use the tool, to make homework a better learning experience.

ASSISTments provides immediate feedback to students as they work on assignments from their teachers; it then gives teachers insight into areas where students struggle, both individually and as a class. Teachers reported they found the system easy to implement, because it fits within common school policies for homework and doesn’t require instructional changes, such as new textbooks or altered pacing through topics. ASSISTments also enabled formative assessment practices for teachers, such as adapting their discussions of homework to fit students’ needs and to address common wrong answers, leading to more effective learning.

“This rigorous, large-scale evaluation found that ASSISTments increased student learning of mathematics,” said Jeremy Roschelle, Ph.D., co-director of the Center for Technology in Learning at SRI International, and principal investigator for the study. “Parents and educators alike want to know how students can learn more from their math homework—and now there is scientific proof that the right kind of intervention can provide a practical, targeted way to increase math learning.”

Key findings highlighted in the study:

Teachers changed how they conducted homework reviews in several ways, such as reviewing fewer problems but at a deeper level; targeting problems that were difficult for the students rather than reviewing them all; and paying special attention to addressing students’ mathematical errors.
The higher test scores for students using ASSISTments were statistically significant with an effect size of 0.18 standard deviation, which translates to a student measuring at the 50th percentile moving up to the 58th percentile. Students using ASSISTments scored an average of 8.84 points higher than those in the control group.
According to national data provided by the test’s publishers, TerraNova scores generally increase by an average of 11.66 points per year between 6th and 9th grades. Presuming the control group in Maine met the national average, the added 8.84 points gained by students using ASSISTments, as compared with the control group, is a large and meaningful effect.
The online math homework proved to be more beneficial for students who started the year with lower math scores, compared with those with high prior achievement, and thus can help to close achievement gaps.
“This study provides gold-standard proof that ASSISTments can provide solid gains in math learning,” said ASSISTments lead creator, Neil Heffernan, Ph.D., professor of computer science and director of Learning Sciences and Technologies at WPI. “Furthermore, we learned this type of intervention can help bridge the achievement gap between those students who start out weaker in math compared with those who start out with stronger math scores.” The WPI team is currently working to expand ASSISTments to include crowdsourcing as a tool to involve more teachers and researchers in the development process.

This study both shows how online tools can help teachers adopt effective formative assessment practices and also highlights how strong teamwork among researchers, computer scientists, and teachers can unlock innovative potential within existing school practices like homework.

Watch a brief video describing the SRI study.
Watch a brief animated video summarizing ASSISTments.

About SRI Education
SRI Education, a division of SRI International, is tackling the most complex issues in education to help students succeed. We work with federal and state agencies, school districts, major foundations, nonprofit organizations, and international and commercial clients to address risk factors that impede learning, assess learning gains, and use technology for educational innovation. SRI International, a research center headquartered in Menlo Park, California, creates world-changing solutions making people safer, healthier, and more productive.

Worcester Polytechnic Institute
Founded in 1865 in Worcester, Mass., WPI is one of the nation’s first engineering and technology universities. Its 14 academic departments offer more than 50 undergraduate and graduate degree programs in science, engineering, technology, business, the social sciences, and the humanities and arts, leading to bachelor’s, master’s and doctoral degrees. WPI’s talented faculty work with students on interdisciplinary research that seeks solutions to important and socially relevant problems in fields as diverse as the life sciences and bioengineering, energy, information security, materials processing, and robotics. Students also have the opportunity to make a difference to communities and organizations around the world through the university’s innovative Global Projects Program. There are more than 45 WPI project centers throughout the Americas, Africa, Asia-Pacific, and Europe. Visit www.wpi.edu.

This material is based upon work supported by the Institute of Educational Sciences of the U.S. Department of Education under Grant Number R305A120125. Any opinions, findings, and conclusions or recommendations expressed in this material belong to the authors and do not necessarily reflect the views of IES.