Your Return on Investment

INVEST IN YOUTH
Two year Analysis of Tutor Effectiveness
In After School Tutoring at Four
Seattle Public Schools
Funded by the Washington Educational Research Association
Conducted by Jill Hearne, PhD. and Linda Elman, PhD., December, 2011

Invest in Youth Program Evolution:

1997 – Started with 1 school and 9 tutors from the investment community

2011 – Currently operating at 4 Seattle schools with 96 tutors
School sites- Bagley, Bailey Gatzert, Beacon Hill, Thurgood Marshall
Original group of volunteers evolved into a non-profit, volunteer board
A part-time Executive Director was hired in 2011 to provide program stability

In 2010 and 2011 Program Evaluations were funded by the Washington Educational Research Association

2010 Program Evaluation

– 37 student cases

– MAPS data from 2 schools

– 80% teacher survey return

– 68 student survey respondents

– No significant differences found on school measures

– Students made typical growth

2011 Program Evaluation expanded to include:

Reading: San Diego Quick Assessment
Math: Math facts count
Surveys: Students, Teachers, Tutors

Available data was doubled:

– 72 student cases

– MAPS data from 4 schools

– 100% teacher survey return

– 50 student survey respondents

– Significant differences found with internal measures and school measures

2011 Evaluation Results found our tutored students showed:

Significant improvement in Reading on internal program assessment
Students learn more than could be expected based on their pre-tutored progress.

2009-2010 Program Evaluation Report

October 15, 2010
by Jill Hearne, Ph.D. and Linda Elman, Ph.D.

The Invest in Youth(IIY) after school tutoring program has increased steadily in scope and proficiency since its inception in 1997. Beginning in October of the 2009-2010 school year, there were 89 students and tutors at four Seattle Public Schools. A half-time program director provided program stability and curriculum uniformity. Two independent assessments were selected to be given by tutors and school level assessments were tapped to provide a data base for student analysis.

Due to a number of factors, usable data was available for 34 students in May, 2010. Comparison of students’ actual growth compared to typical growth showed that students in Invest In Youth made typical growth relative to their classmates in 2009-2010. While in previous years they were falling further behind their peers, during their tutored year students made equivalent growth with their peers. When one considers that teachers selected these students precisely because data and observation indicated that they were not making adequate yearly growth, “typical growth” can be considered an improvement.

Students, teachers, and tutors were surveyed regarding their attitudes about their experience with Invest In Youth. As an after school program, IIY adds value to students‘ lives by increasing the number of responsible caring adults children can observe as models.

In May, 2010, students, teachers and tutors responded to surveys related to their experiences with IIY.

Tutors
Fifty (50) tutors responded to ten questions utilizing Survey Monkey:

The highest agreement (82% always, almost always) was with the item, “When I leave for the day, I feel good about the tutoring session.” 18% felt this was true “sometimes”.
The lowest agreement (40%) was with the item, “My student’s teacher communicates what I need to know.” Thirty percent (30%) responded “Never” or “Almost Never” to this item.
Tutors felt supported by the IIY tutoring staff; 88% reported positively that the tutoring staff is helpful in addressing problems that arise.
Half of responding tutors felt that their student was focused (50%), while 40% felt their student was focused “sometimes.”
The majority of tutors responded positively that their student came ready to work and the work was appropriate for their student’s level.
One item was poorly worded and will be reworded for next year’s survey. This item, “There is enough work; I do not have to supplement, ”held a positive and a negative stem so that the responses could not be interpreted.

Comments:
Themes in tutor comments included a desire for more communication and direction from the classroom teacher; “most effective when it deals with specific tasks,” “wish he had school work we could work on together”. Another theme was a desire for more structure and math/learning games.

Students
Sixty eight (68) students responded to the twelve-question survey.

Students were overwhelmingly positive (72%) about their tutor and their tutoring experience. Their favorite part of tutoring was the one to one relationship with their tutor, talking about homework and personal interests. Ninety-five percent (95%) of students believed they had improved in at least one area.

Comments:
The students, according to comments made by both teachers and students, value the time with tutors immensely and are hurt when tutors don’t come. Their comments are endearing regarding their relationship with their tutor. “I feel special,” “I get help,” ”I get my homework done,“ “I like being with my tutor” and “I get to make a new friend. ”

Teachers
Eighty per cent (80%) of the teachers returned their surveys. This was the highest return rate since surveys were started; a Starbuck’s bonus card proved to be motivational this year. Responses to the ten questions were positive overall.

Responding teachers (100%) agreed that student’s attitudes and behavior improved and that the program is beneficial for their school.

Ninety two percent (92%) of teachers reported progress on students’ academic work, homework completion, and motivation. Teachers responded positively that tutors worked effectively and respectfully with students.
The least agreement on items concerned communication. Teachers reported positively, seventy five percent (75%), that communication with the tutor was good. There was disagreement on “best way to communicate” with 5 choosing email and 6 preferring personal communication.

Comments:
Comments were uniformly positive. The teachers frequently commented about the importance of the relationships that students formed with tutors. They felt that the students benefited from the one-on-one time with a responsible, respectful adult. If they ever cited a problem, it was when a tutor failed to return after starting strongly at the beginning of the year.

Conclusion:
Although there is no evidence that the tutoring provided academic gains beyond those that would be expected in a year’s growth, it is clear that the adult attention to students living in poverty is valued by students and their teachers. Also, for these students, a year’s growth is positive since teachers selected the students after a review of classroom evidence and test scores indicated that these students were not making adequate yearly growth.

Survey Limitations:
A number of factors limited the available data for this year’s analysis. There was no one assessment common to all schools. The IIY assessments, while valid in design, were not administered with equal rigor in all 4 schools. Variation in tutor and administrative support at the schools contributed to uneven administration as well as poor return of assessment data. Consequently, we were limited at the end to use of MAP data at 2 schools. The other two schools chose not to use the MAP this year and had no pre-post data that could be utilized.

Improvements for the 2010-2011 Evaluation Program:
Next year MAP data will be available at all 4 schools and will be used. Additionally, the San Diego Quick Assessment and Math computation assessment will be given three times, October, January, and May, by the IIY director, who will do the assessments herself at each school. Closer monitoring of evaluation results and increased cooperation of school personnel should result in a larger data pool and increased ability to gather information for continued program improvement.

PROGRAM ENHANCEMENTS DUE TO GRANT FUNDING
In 2009 and again in 2010, Invest in Youth received capacity building grants from a national government program, the Compassion Capital Fund – Communities Empowering Youth which greatly bolstered our small budget. These grants affirmed our work for the past fourteen years and allowed us to make dramatic strides in our web presence, staffing, technology, evaluation tools, community outreach and administrative preparedness.

These funds allowed us to:

Update our website
Acquire a database
Obtain a 501(c)(3) designation from the IRS
Purchase equipment and supplies for our program director
Recruit new tutors
Provide our program director with continuing education
Develop a quantitative tool to independently evaluate student progress

These grants terminated in August 2010.

Invest in Youth is 501(c)(3) non-profit organization for community involvement in Seattle Public Schools, tax ID #27-2208137.