Evaluation Report: The Impact of Computer Assisted Instruction On 7th Grade Math Outcomes at Ardsley Middle School September, 1997

Prepared by George M. Holtz, Ph.D.

*There are many references to CCC in this paper. CCC was aquired by NCS Learn in 2001.*


Introduction

This report to the Ardsley Union Free School District has several sections. Section One reviews the question which generates the evaluation. Section Two describes the methodology used to provide the best possible answer to the question. Section Three contains the statistical results of the evaluation. In this section the reader will also find three graphs which constitute the first phase of the analysis and one table which contains statistical results. Section Four offers a brief discussion of the results.

Section One: Question

The Ardsley Union Free School District has made an investment in Computer Assisted Instruction (CAI) at the Middle School. This investment was made two years ago and the system had been used regularly for supplemental math instruction. Since full implementation has been achieved, the natural question arises: What is the impact of the CAI system on student outcome?

Section Two: Methodology

The methodology used for the Ardsley analysis has two phases. The first phase compares last springõs (1997) IOWA Test of Basic Skills math subtest scores for seventh graders to those scores achieved in 1992, 1993, 1995 and 1996. Scores are compared within I.Q. ranges (130-150, 109-129 and 108 and below). This longitudinal comparison is done to determine if there is a "bump" or an increase in scores over those achieved in prior years, before the implementation of CAI Program. I.Q. scores are controlled in order to determine how each learner group performed on the various subtests (Concepts, Problem Solving and Computation). In addition to the graphic presentation in phase one of the Ardsley analysis, a statistical procedure called regression analysis , is done to determine if the most recent scores are statically different from the overall historical pattern.

If an increase is observed for any learner group in the Spring of 1997, then a second phase of the analysis is performed.

In order to complete the answer to the Ardsley question (impact of CAI system on student outcomes in mathematics) continued care has to be taken to reduce the impact of cognitive ability, particularly in the manipulation of quantitative material.

Time on task (time on the CAI system) was the independent variable (suspected cause of variation in math outcomes). Phase one of the analysis is used to determine whether there is variation; Phase two is used to determine whether or not that variation can be attributed to time spent on the Ardsley CAI System. Math outcomes were measured using the three subtests on the IOWA TEST OF BASIC SKILLS: Concepts, Problems and Computation. Time on task was measured for each student beginning February 1 and running to May 5, 1997. The evaluation design followed this scheme.

High Users of CAI = Group 1 (4:01 hours or more)

Medium Range of CAI= Group 2 (3:01 ­ 4 hours)

Low Range Users of CAI= Group 3 (3 hours or less)



ASSESSMENT OF THE CCC PROGRAM ON TEACHING STYLE, STUDENT INTEREST AND ATTITUDES

The research indicates that the classroom environment affects student learning. There is a direct correlation between a student's interest in a subject and the manner in which the learning environment is organized. Therefore, the mathematics teachers were asked to complete a questionnaire to document their perceptions of how the CCC program helped them to create a student-centered environment and how this in turn, may have affected achievement. The following is a synthesis of the most common responses to each question.

1. CCC has changed the teaching from frontal instruction to facilitator dealing with many lessons at once.

   In what ways has the cluster of computers and the CCC Program changed your style of teaching?

• My classes have become more student-centered and there is more time for individualized instruction.
  
  
• Mini-lessons are followed by small group activities, the structure is not conducive to large group projects.
  
  
• The cluster of computers has transformed the mathematics classroom into a Math Workshop. After the mini-lesson, the students work independently while I circulate and "Conference" with students.
  
  
• CCC has changed the teaching from frontal instruction to facilitator dealing with many lessons at once.
  
  
• My teaching has become more versatile and more flexible.
  
  
• CCC allows me to change my style of teaching from a teacher-centered approach.. Instead of lecturing and drilling, I can be a facilitator of learning. Consequently the students have become active participants in their learning process.
  
  
• Less frontal teaching and more student participation.
  We used the computers regularly. The class was broken into groups. This created a major change in my teaching style.
  
  
• The large screen TV was excellent for introductions and to generate class discussion.
  
  

2. How often does each student use the CCC Program? What is the duration of a typical session?
   

• In general, students use the CCC program 2 to 3 times per week or every other day for approximately 15 minutes per session.

  Students seem to enjoy math and CCC because each one is working on his/her own level. They feel in charge and want to progress and succeed.

  
  

3. How has the CCC Program affected studentsõ attitudes toward mathematics?

• Students who are "grade oriented" enjoy tracking their progress.
  
  
• Students feel confident about certain strands because the CCC program reviews necessary material.
  
  
• Students seem to enjoy math and CCC because each one is working on his/her own level. They feel in charge and want to progress and succeed.
  
  
• Students are eager to go on CCC whether MCS or Investigations. They have made CCC an integral part of their learning. They enjoy the challenge, the variety of questions, and the fact that it provides an alternative to more traditional modes of instruction.
  
  
• Students in the accelerated class love the program. They are challenged by it and use it to the utmost. Students in the collaborative class have better attitudes towards math thanks to CCC.
  

4. In your assessment of student progress, how has the CCC program affected student achievement of course content?

• The main value in terms of student achievement is the constant review of topics previously taught and therefore better retention.
  
  
• We are able to spend less time on review and more time on new topics. I can definitely see a major change in the basic skill level of the students using the program. The type of mistakes made on tests are not the result of outlandish guesswork but rather near misses from not reading the question carefully or making minor mechanical errors.
  
  
• CCC has elevated students comfort level with computer/technology and it has helped students focus more on problem solving.
  
  
• This group did much better on classroom tests and on the final exam than the eighth grade the previous year. I would attribute some of that to CCC.
  

5. What courses in the CCC program have been most useful to you?
   

   CCC has elevated students comfort level with computer/technology and it has helped students focus more on problem solving.

   
   

• Math Concepts and Skills (MCS) is utilized most.
  
  
• Enrichment modules provide practice for specific units (geometry, statistics)
  
  
• Concepts and Skills, Math Investigations and Algebra topics have been used most.
  
  
• Math Concepts and Skills reinforces math skills..this allows me the privilege of spending more time on word problems and group projects in order to promote higher-level thinking.
  
  
• MCS has been most beneficial. Lately, students are almost exclusively on Math Investigations.
  

6. How does the CCC Courseware compare with the other software available for supporting lesson/unit outcomes?

• CCC is a good overall review. Many of the stand-alone software has proven to be very useful for specific topic.
  
  
• The stand-alone software is better for specific units of study.
  
  
• CCC is good for drill and practice. The stand-alone software has proven to be very useful for specific topics.
  
  
• The CCC Courseware offers a well organized curriculum that could not be matched by any stand-alone program. Through printed readings, we are able to analyze student growth and accomplishments.
  
  
• Individual software provides for specific unit reinforcement, CCC addresses overall needs by incorporating all types of questions.
  
  
• I have not seen better software for practice, skill development and concept development.
  

7. What are the strengths of the CCC program?
   

   This group did much better on classroom tests and on the final exam than the eighth grade the previous year. I would attribute some of that to CCC.

• The main advantages are the constant review and hopefully greater retention and the individualized nature of the instruction.
  
  
• Higher level students are exposed to new concepts.
  
  
• The individualized learning program allows students to learn at their own pace.
  
  
• The program is sequential and organized.
  
  
• Variation in questioning and differentiating instruction. If a student were to get a question wrong it takes them through the process step by step.
  
  
• CCC provides the students with a meaningful learning experience using computers to solve problems. Computers are an important link to success in their future and students should be comfortable using a variety of programs. The CCC programs are very beneficial in building confidence and enhancing the motivation level of students. If the studentõ confidence and motivation levels remain elevated the students will be more likely to go on to higher mathematics.
  
  
• CCC is an exciting and stimulating alternative. Students like the presentation, length of each segment and the fact that they learn without the pressure of grades.
  
  
• It has excellent demonstration potential especially in the area of coordinate geometry and probability. Math Investigations is a superior program that leaves students feeling accomplished and knowledgeable.
  
  

8. What are the weaknesses of the program?

• Many students would benefit from a longer lesson with guided practice and immediate independent practice before they go home to do homework.
  
  
• The time needed for CCC impedes projects which require full class participation.
  
  
• The only weakness is the hardware problem. This year we werenõt straightened out until January.
  
  
• There are not enough computers and the program freezes at times.
  
  
• We experienced problems with the server this year that resulted in loss of valuable time for the students.
  
  
• The hardware breaks down.
  

9. Some students in the accelerated fifth grade have progressed more than three years in seven months. ... These children are using CCC not only for practice but also they are learning from it.

   Other Comments

• I believe that having computers in the classroom, the CCC program and other math software are truly assets to the learning environment.
  
  
• I love the way the class can be working on what each member needs at the same time. I can help each student advance.
  
  
• The CCC programs allow the students to use tools such as calculators to assist them in solving problems.
  
  
• Most students view CCC as a worthwhile complement to regular math learning.
  
  
• Some students in the accelerated fifth grade have progressed more than three years in seven months. They are working on Math Investigations with extreme enthusiasm. These children are using CCC not only for practice but also they are learning from it. I feel that CCC has had a marked positive influence in our Math program.

COURSE USAGE SUMMARY

The following is a summary of the average time each student in a specific class and grade spent on the CCC Courseware for the period of approximately 50 days from February 1, 1997 to May 5, 1997.

*Regents Sequential Mathematics Course T.

top