PEARSON SUCCESSMAKER MATH PILOT STUDY 12-1-09 No part of this report may be reproduced or used for any purpose without permission from Gatti Evaluation Inc. Guido G.Gatti Gatti Evaluation Inc. 162 Fairfax Rd.
Pittsburgh, PA 15221 (412) 371-9832 Manager Kate Giordano Gatti Evaluation Inc. Funded By For Information Please Contact: Marcy Baughman of Academic Research (724) 863-1621
OF CONTENTS EXECUTIVE SUMMARY 1-3 I.
INTRODUCTION 4-6 a.Instructional technology ; 4-5 b.Study Goals and Research Questions ...5-6 II.
Teaching Logs and Classroom Observations .. 17-19 III.RESULTS > a.Statistical Analysis of Outcome Measures 19-23 b.
Baseline Achievement and Attitudinal Measures 23-24 c.
SuccessMaker Math Achievement Gains > d.SuccessMaker Gains by Subpopulations 28-38 e.
SuccessMaker Gains by Implementation Style 38-39 f.Group Comparisons of Achievement Gains 39-43 g.
Group Comparisons by Subpopulations ..
44-48 h.Teacher and Student SuccessMaker Opinions 48-66 i.SuccessMaker math Program Usage .
DISCUSSION 69-70 A.1 Teacher Implementation Rating Results _________________________________ 72 A.2 GMADE Standard Score to Percentile Rank Correspondence ________________ 73 A.3 SuccessMaker Math vs.
Comparison Group Model Adjusted Grade Differences by Subpopulation Graphs _____________________ 74-90 SuccessMaker Math Pilot
Gatti Evaluation Inc. 12-1-09
- 1 - EXECUTIVE SUMMARY Pearson partnered with Gatti Evaluation to conduct rigorous research to support the assertion that the SuccessMaker Math computer based learning program effectively increases student mathematics achievement and attitudes.
The program was evaluated in forty-three diverse elementary grade classrooms from NJ) during the 2008-09 school year.
Students in SuccessMaker schools made regular use of the program while students in comparison schools received supplemental instruction from non-computerized district adopted supplemental mathematics programs.
The study schools come from public schools show considerable variation in ethnicity, students eligible for reduced priced lunch, as well as a wide range of ability with respect to mathematics and reading achievement.
The evaluation team sought out diversity in the study sample to ensure the program would be used by ity that is todays elementary classrooms.
The final study sample consisted of 408 3 grade (i.e., SuccessMaker = 230, comparison = 178)
A challenging assessment battery was group administered to studeschool year.
The assessment Group Mathematics Assessment and
(GMADE), and the mathematics attitude survey developed by the principal investigator where ststions regarding general math attitude, confidence, motivation, and self perceived aptitude.
Of the 879 students tested at baseline, 90% remained in the final study sample (i.e., SuccessMaker = 94%, comparison = 85%).
Quantitative data collected from site observations indicated that the studyteachers executed their mathematics instruction at similarly high levels with the vast majority to
scores were statistically
study groups, with the comparison group higher for the 3sample and the SuccessMaker group higher for the 5 grade sample.
In both cases the differences are 0.21 standard deviations, a small to moderate sized baseline difference.
RQ3.How did teachers and students react to the SuccessMaker Math program?
Focus group sessions were conducted at each school during site visits between mid November and early February and again between April and early June.
These evaluators with the following insights into teacher and student experiences with the program.
Teachers and students have become very comfortable very quickly with the SuccessMaker program and feel the program was a good educational investment.When inresponse to the program was overwhelmingly positive with 81% of the 475 recorded comments being positive in nature.
Teachers appreciated the reporting system in informing classroom for remediation, monitoring studerelaying student progress information to curriculum specialists and parents.
A majority of teachers felt the initial placement and adaptive motion of students through the program was effective, the learning activities were well differentiated and aligned to their current curricula and SuccessMaker Math Pilot
Gatti Evaluation Inc. 12-1-09state educational objectives, and that the program challenged both their lower and higher rare minor technical issues (ex., logging in, activities loading), most likely a result of
Teachers firmly believe that their students like using the program and feel that the program makes the learning process more fun.
When formally interviewed, teachers were overwhelmingly positive about their students interactions with the program.
Of the 108 recorded comments, 81% were positive in nature.
When surveyed, 90% of 3 liked the program.
Studentcharacters, animation, and found the learning activities engaging, especially the speed games.
Some students did find the characters immature and the animation sometimes excessive and nts appreciate the capacity of the program to allow them to rtual learning environment.
How was the SuccessMaker Math program utilized?
Proper implementation was supported by the teacher logs as 3 grade SuccessMaker teachers reported
Successmaker math efficacy report final
Principal Investigator Guido G.Gatti Gatti Evaluation Inc.
32 III.RESULTS > a.Baseline Group Equivalence.34 b.
Group Comparisons of Achievement Gains.
38 c.Group Comparisons by Subpopulation..43 d.Student Academic Attitudes 49 f.Teacher and Student SuccessMaker Opinions..52 IV.
DISCUSSION 58-59 A.1 Comparative Study Group Results by Program Usage ______________________ 39-43 Table 1
Gatti Evaluation SuccessMaker Math Study Site State Assessment Information 12 Table 2
SuccessMaker Math RCT Training Dates.16 Table 3
Gatti Evaluation SuccessMaker Math RCT Sample Demographic Information...29 Table 4
Third Grade Baseline GMADE Scores Study Group Comparisons..34 Table 5
Fifth Grade Baseline GMADE Scores Study Group Comparisons...35 Table 6
Seventh Grade Baseline GAMDE Scores Study Group Comparisons..35 Table 7
Baseline Math Academic Survey Score Comparisons..
35 Figure 1
GMADE Total.36 Figure 2
GMADE Concepts and Communication.36 Figure 3
GMADE Operations and Computation...
37 SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10 Figure 4
GMADE Process and Applications..
37 Figure 5
Math Academic Attitude Survey.50 Figure 6
Do You Like SuccessMaker Math?52 Figure 7
Do You Like It When the Characters Sing and Dance?....53 SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10- 1 - EXECUTIVE SUMMARY Pearson partnered with Gatti Evaluation to conduct rigorous research to support the assertion that the SuccessMaker Math computer based learning program effectively increases student mathematics achievement and attitudes.
The program was evaluated in sixty-three diverse elementary and middle grade classrooms from ten schools in seven different states i.e., AZ, AR, CA, IN, KS, NY, PA during the 2009-10 school year.
Students in classrooms randomly assigned to use SuccessMaker made regular use of the program while students in comparison classrooms received supplemental instruction from non-computerized supplemental mathematics programs.
Four widely-used classroom mathematics programs were utilized by the sites at 3rdand 5th grade, and three different programs were utilized at 7th grade.
The study schools come from public school districts located in large cities or suburbs of large cities.
The study schools show considerable variation in ethnicity, students eligible for reduced priced lunch, as well as a wide range of ability with respect to mathematics and reading achievement.
The evaluation team sought out diversity in the study sample to ensure the program would be used by learners of all abilities and backgrounds, thus reflecting the reality that is todays elementary classrooms.
Five schools began the study in the first month, three began in the third month, one in the fourth and the last in the fifth month of the school year.
The final study sample was large, consisting of 505 3rd grade i.e., SuccessMaker = 282, comparison = 223, 408 5th grade i.e., SuccessMaker = 224, comparison = 184 and 273 7th grade i.e., SuccessMaker = 136, comparison = 137 students.
A challenging assessment battery was group administered to students at baseline and again at the end of the school year.
The assessment battery consisted of the Group Mathematics Assessment and Diagnostic Evaluation GMADE, and the mathematics attitude survey developed by the principal investigator where students respond to self-report questions regarding general math attitude, confidence, motivation, and self-perceived aptitude.
Comparisons on assessment outcomes were made between study groups using model adjusted end-of-year raw score group mean differences.
Adjusted group mean differences are calculated holding the effects of confounding variables constant for both groups.
The equating of confounding variables and the maintaining of consistent implementation ensures the outcomes may more confidently be attributed to the study conditions randomly assigned to these groups.
Results were broken out and analyzed separately for each GMADE subtest i.e., Concepts and Communication, Operations and Computation, and Process and Applications.
Results were also broken out and analyzed for separate levels of five key student populations i.e., English proficiency, ethnicity, gender, meal status, math ability.
Further, the performance for the comparison group was compared to four blocks of program usage i.e., block 1 = 1 to 9 hours,
block 2 = 10 to 19 hours, block 3 = 20 to 29 hours, block 4 = 30 or more hours.
RQ: How did teachers and students react to the SuccessMaker Math program?
Focus groups were conducted at each school during site visits between April and early June.
These sessions provided the
Tags: successmaker math and reading,math and reading lesson plans,free math and reading games,math and reading worksheets,math and reading test,math acuity mcgraw hill,acuity math and reading,acuity math assessment,math acuity website