Renewal of After-School and Mentoring Programs

All after-school and mentoring programs that received funding beginning in July 2004 were eligible to apply for renewal of grant funds. The information below describes the renewal process, the overall results and findings, themes and observations, and a detailed summary of past performance and renewal request for each grantee.

Renewal Process

As you may recall, the following requirements were established for current grantees seeking renewal of funding for after-school and mentoring programs:

1. Eligibility: Renewal was available only for current grantees seeking funding for the same or similar activities.

1. Contract Compliance: Staff was charged with assessing and reporting grantees' compliance with contract terms including serving the number of children they agreed to serve, providing the program as stated in their scope of services, whether they were able to collect data on outcomes and if so, whether they met outcome targets, and complying with all other contract requirements.

1. Site Visits: All grantees were required to host a site visit during which staff conducted an extensive interview to assess program practices and implementation. Staff also completed program observations for all programs except for the one-on-one mentoring programs.

1. Financial Assessment: CHIF retained McDonald Jacobs, the firm that conducts the annual audit of CHIF, to review a quarterly expense report for each grantee to assure that the grantee could produce supporting records for all expenses claimed. A copy of the report is attached to this memorandum for your information.

Grantees were also required to submit an audit. Audits were reviewed using the audit tool designed by McDonald Jacobs to get a sense of the financial status of the organization applying for renewal. Individual results of the tool application to audits are in the Performance Assessment for each grantee.

1. Renewal Application: All grantees were required to submit a renewal application that included a program description, proposed program outcomes, program budget and budget narrative and any proposed program changes. Grantees were permitted to request annual funding at the same level as their current year 2 budgets with annual increases of up to 3% (compounded annually) to cover the costs of inflation.

Results and Findings

1. Contract Compliance: All grantees are generally complying with contract requirements and delivering the programs they agreed to deliver. Many grantees did not meet service goals in Year 1 of their contracts, primarily due to start-up issues such as hiring and training staff, and recruiting participants to new programs.

2. Site Visit Assessment: Staff performed site visits with all 20 grantees during January - March 2006. Site visits were approximately 2.5 to 3 hours long and included interviews with program staff, review of program files and documents, and observations of program activities. CHIF staff assessed program performance using a rubric defining best practice standards of program implementation in five areas:

• Client Recruitment and Outreach
• Client Intake/Enrollment

• Client Files and Client Data
• Program Effectiveness and Fidelity to Model
• Program Staffing.

Based on results of site visit, program performance was rated on a scale of poor, satisfactory, or excellent for each of the five areas and programs earned an overall assessment of failing to meet, meeting or exceeding standards. Ten after-school and mentoring programs exceeded program standards in three or more of the categories listed above, and the remaining ten programs met standards. No programs fail to meet standards.

3. Submission of Recent Audit: With the exception of Bridge Builders, all grantees submitted audits. Audits were reviewed using the audit tool and individual results are listed under the review of each program. Most organizations are in acceptable financial health. The most common issues of concern raised by the tool were low reserves, high debt to asset ratios, and low operating reliance which measures the ability to pay expenses with program revenue. These concerns are common in the non-profit world where organizational budgets may fluctuate significantly from year to year as grants come and go and public funds are cut.

4. Renewal Application: All current grantees submitted renewal applications. Staff reviewed the applications and relevant information regarding the applications is included in the individual program reviews appended to this memorandum. In cases where grantees requested funding above the ceiling established in the Renewal Application, staff has recommended a funding amount that meets this requirement.

5. Staff Recommendation: Based on the results of the analyses described above, staff recommends that the Committee renew all of its investments in after-school and mentoring programs in the amounts and on the terms stated in the individual Performance Assessments accompanying this memorandum.

Themes and Observations

1. Impact of Expansion and New Program Funding

Grantees' service and budget projections for the first year of operations for new or substantially expanded programs proved ambitious in many cases. Hiring and training staff, staff turnover, and successfully recruiting students at schools where a program was new proved challenging. Fifty-five percent of programs (11/20) served at least 90% of the children they projected to serve. Fifty percent of programs spent at least 90% of their Year 1 program budget. Interestingly, programs may have spent all budgeted funds, but not served the projected number of children due to lower than expected student participation and fixed staff costs to deliver a program. In the future, CHIF staff will encourage budgeting and service projection that takes into account a ramp-up period.

2. Data Collection and Outcome Information

All after-school and mentoring program contracts included outcome targets. Most grantees agreed to track outcomes for attendance, behavior and academics. Some grantees also agreed to track other outcomes including outcomes related to developmental assets. Five grantees were unable to provide data on any of the outcome targets set in their contracts, and seven grantees were only able to obtain data on one or some of the outcomes targets set in their contracts. In some cases, this was because they could not obtain the relevant data on enough of the participants in the program to generalize results. Sometimes grantees were simply unsuccessful in getting data from the schools or from parents despite working to do so. In one case, PPS refused to release the data to the individual program because there were too few participants in the program and confidentiality could be compromised.

Our new requirement that all grantees submit participant lists to the school districts in which they are working will assure that data can be gathered and outcomes analyzed on consistent data points across programs. We may need to work to devise alternate outcomes and data collection methods for programs that do not serve enough participants annually to allow districts to release data. Finally, staff will need to assure at the front end that grantees specify measurement methods and timeframes for any outcome targets they set for which the school districts will not collect data.

3. Organizational Development Needs

While all programs at least met program standards, many of the smaller organizations we fund could use resources to develop and/or solidify organizational processes and procedures for carrying out their work. The most universal need is for data systems and staff time to take information that is gathered on an individual client basis and aggregate it so that programs can use that data to inform program decisions.

4. Site Visit Interview Tool

CHIF staff may need to revise the site visit interview tool and the program implementation standards to better suit activity-based after-school programs. The tool includes some implementation standards that are more applicable to full spectrum social services rather than more limited interventions such as after-school music or art classes.

After School and Mentoring Program Investments: Performance Assessments and Renewal Requests by Program

1. Big Brothers Big Sisters: Community Based Mentoring Program

Contract Amount: $300,000

Contract Goals:

Services:

• Serve 137 children in year 1 and 162 children in Year 2 with an average of 12 hours per month of mentoring services for one year.

Outcomes:

• 70% of mentees will demonstrate improved school attendance or attendance rates of 85 % of school days during the school year.
• 75% of mentees will demonstrate improved classroom behavior as reported on the Program Outcome Evaluation.
• 90% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 130 children in Year 1 and at end of Q2 had 130 children in service. On track to meet service goal in Year 2.
• Outcome Goals: Data on outcome goals was not made available in terms of outcome targets i.e. the percentage of participants that achieved improvement or a certain status.
• Contract Spending: Spent 100% in Year 1, and 74% as of Q3 Y2; on track; on track to spend out.

Site Visit Assessment: Exceeds most program standards; new database should help grantee aggregate additional information currently in individual files.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Proposes to keep service level the same as contracted for Y2 of current contract, 162 children annually;
• Proposes to offer same services to children but to increase training available to mentors to improve retention rate;
• $340,602 over 2 years.

Staff Recommendation: Renew as requested.

2. Boys and Girls Aid Society: Committed Partners for Youth Mentoring Program

Contract Amount: $200,000

Contract Goals:

Services:

• Serve 72 children annually with 6 hours per month of one-on-one, team and group mentoring services.

Outcomes:

• 80% of mentees will demonstrate improved school attendance
• 80% of mentees will demonstrate decreased behavior referrals
• 80% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 56 children in Year 1 and at end of Q2 had 59 children in service. Changed mentor recruitment and training protocols after Year 1 to increase the number of children. Also increased recruitment of high risk mentees with parents involved in the criminal justice system. On track to meet service goal in Year 2.
• Outcome Goals: Made reasonable progress toward goals. At end of Year 1: 35% improved attendance; 66% attended at least 90% of school days; 75% had no behavior referrals; 33% decreased behavior referrals over the course of the school year; 71% showed significant gains in annual reading testing; 61% showed significant gains in annual math testing.
• Contract Spending: Spent 85% of funds in Year 1; has spent 74% as of Q3 Y2; are on track to spend out contract in Year 2.

Site Visit Assessment: Excellent; exceeds program standards in most categories.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Proposes to increase the number served to 95 children annually with same staff and same service level to each child;
• $208,329 over 2 years.

Staff Recommendation: Renew as requested.

3. Boys and Girls Clubs: Youth and Family Services After-School Program

Contract Amount: $360,000

Contract Goals:

Services:

• Serve 150 children annually with after-school program services made available 5 days per week for 5 hours per day (average participation is 2 days per week for each child).
• Provide 45 children and their families with counseling services through a subcontract with Trillium Family Services.
• 150 young people identified as high risk will be recruited into BGCP's programs over a one-year period. At least 60% of these youth will maintain their participation in the program during school year.
• 25% of participating youth will take part in a skill-building program (SMART Moves, SMART Girls,)

Outcomes:

• 75% of participating youth will have no criminal referrals after three months of program participation for at least one year.
• 50% of participating youth will have an increase in developmental assets as measured by survey instruments developed by Boys & Girls Clubs of America.
• 50% of the children recruited into the program will demonstrate academic achievement by improving academic performance either through improved grades, improved results on yearly assessments, meeting grade level standards, or meeting state benchmarks.

Contract Performance:

• Service Numbers: Served 124 children in Year 1 with after-school club services; as of Q2 Y2, serving 61 children with after-school club services. No staff at one club until late October and at a different club until December 2005 substantially decreased program attendance for first half of Year 2.

Trillium served 12 children/families with counseling services in Year 1; provided informal services for children during club hours and for BGC staff in Year 1.

Amended contract for Year 2 for Trillium to provide informal services during club hours to 30 children annually and formal counseling to 15 children/families annually. After Q2 of Y2, Trillium has provided informal services to 8 children and counseling to 5 children.

Overall, Year 2 numbers are down, primarily due to staff turnover. Y1 performance was close to contract goal.

• Outcome Goals: Most outcome goals met after Year 1. At end of Y1, more than 60% of children who entered the program maintained average participation of 2 days per week; 51% participated in a skill building program; no data on criminal referrals; more than 50% of participants increased developmental assets overall; 53% improved grades over the school year.
• Contract Spending: Spent 93% in Y1; have spent 65% as of Q2 Y2; staff vacancies in Y2 will likely push down spending for Y2.

Site Visit Assessment: Exceeds most program standards. Improvements in aggregating information to determine highest source of referrals and most successful methods of outreach, and in staff exit process would take program to next level.

Financial Assessment:

• Expense reporting spot check: Passed
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• No changes to level of service, number served or type of service proposed from current contract.
• $375,299 over two years.

Staff Recommendation:

• Authorize renewal up to requested amount but discuss whether subcontract with Trillium should be renewed at same level given relatively low usage by children and whether these funds should be redirected toward salary increases for line staff to decrease staff turnover.

4. Bridge Builders Mentoring Program

Contract Amount: $113,000

Contract Goals:

Services: Serve a total of 120 male and female high school youth with group mentoring services designed to prepare and assist students in continuing their education beyond high school. CHIF funds enabled creation of a girls program; previously delivered services only to boys.

Outcomes:

• 80% of mentees will demonstrate improved school attendance or attendance rates of 80% of school days during the school year
• 50% of mentees will demonstrate decreased behavior referrals or fewer than 3 referrals during the school year.
• 50% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.
• 90% of mentees who are seniors in high school will take a college entrance examination.
• 90% of mentees will enroll in a post-secondary educational institution.

Contract Performance:

• Service Numbers: Served 100 children in Year 1; currently serving 89 children; 74% of service goal
• Outcome Goals: Most outcome goals met. 98% of participants attended school at least 80% of school days; 87% had fewer than 3 behavior referrals; academic data from PPS was available for too few students to generalize performance results; 100% of seniors graduated from high school and was accepted to at least one post-secondary school; 83% of juniors took the SAT.
• Contract Spending: Spent 72% of funds in Y1; as of end of Q3 have spent 68% of funds; on track to spend out.

Site Visit Assessment: Meets program standards to the extent applicable. Grantee retains no paid staff to run the program; all functions are performed by volunteers. To improve program quality, grantee would need at least part time paid staff to better coordinate volunteers, maintain adequate records at a central location, and reach its service potential. Note: all program observation was for boys program; if grant is renewed, a review of the girls program which has been in operation only since receipt of CHIF funds will be conducted.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: No audit provided.

Renewal Request:

• Grantee proposes to serve fewer children annually (100 as opposed to 120 in its current contract). Given that grantee actually served 100 children the first year and is currently serving 85 children, this reduction is aligned with what appears to be their actual capacity.
• Grantee requested $133,600 over 2 years. The maximum grantee is eligible to apply for is $118,136. Most of the additional funds requested would provide higher stipends to the volunteers who deliver the program.

• Staff Recommendation*: Authorize renewal up to the maximum renewal amount ($118,136) but require that grantee retain some paid administrative staff whose duties would include some type of centralized record keeping, supervising grant reporting etc. Require independent financial review as part of contract renewal.

*Allocation Committee voted to make an exception to the 3% rule and fund this grant at the level requested and require that grantee allocate funds for administration of the grant.

5. Camp Fire USA: Campfire PALS After-School Program

Contract Amount: $435,621

Contract Goals:

Services: Serve 150 youth at three schools with 4 day per week after school program that includes academic enrichment, service learning, sports and recreation.

Outcomes:

• 90% of Camp Fire PALs youth will participate in community service as measured by a volunteer sign-in sheet.
• 90% of Camp Fire PALs youth will develop one to two personal/academic goals and, with support from Camp Fire PALs staff, 70% of those youth will have met their goal(s) by the end of the school year.
• 50% of Camp Fire PALs youth will report a positive change in their social and academic self-esteem as measured by a pre- and post-survey of participants.
• 50% of Camp Fire PALs youth will increase their rate of homework completion as measured by a pre- and post-survey of classroom teachers.

Contract Performance:

• Service Numbers: Served 113 children in Year 1 with after-school program services; as of Q2 Y2, serving 92 children. Close to meeting service goal in Y1; on track to meet service goal in Y2. Is modifying middle school curriculum to improve recruitment and attendance at this level.
• Outcome Goals: 100% participated in community service project; 100% developed personal goals but no data collected on how many met goals they set; 70% demonstrated increase in developmental assets related to social self esteem; no data on academic self esteem; no data on homework completion.
• Contract Spending: Spent 91% in Y1; have spent 75% as of Y2 Q3; on track to spend out.

Site Visit Assessment: Exceeds most program standards. Improvements in aggregating information to determine highest source of referrals and most successful methods of outreach would take program to next level.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool with the exception of a negative operations margin which looks at overall profitability (expected range for non-profits is -6% to 0% so a negative score is below a -6%).

Renewal Request:

• Proposes to deliver the same level of service to 175 children annually (increase of 25 children per year) through partnership with MFS at one elementary school. (Note: nothing in budget indicates financial partnership nor counts any additional staff into Campfire budget; follow up with staff re: how this p-ship will work such that it will allow them to serve more children). Will also launch adventure curriculum for middle school program to improve recruitment and attendance at this level.
• $467,765 over two years.

Staff Recommendation:

• Renew at funding level requested. Given lack of data on outcome measures specified in contract, work with grantee to set program outcome goals and baselines, and define measurement tools that will assure outcomes can be reported.

6. Ethos After-School Music Program

Contract Amount: $200,000

Contract Goals:

Services: Provide after-school music program services to 150 children annually.

Outcomes:

• 80% of the children receiving after-school music instruction will demonstrate improved school attendance or attendance rates of 95% of school days during the school year.
• 80%Students receiving after-school music instruction will demonstrate decreased referral rates at least 5% less than their school's overall discipline referral rate. of the children receiving after-school music instruction will demonstrate decreased behavior referrals or fewer than 3 referrals during the school year.
• 80% of the children receiving after-school music instruction will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 212 children in Y1; currently serving 154 as of Y2 Q2. Met service goals.
• Outcome Goals: 59% attended school 95% of days; 88% received fewer than 3 behavior referrals during school year; 84% met grade level standards in reading; 78% met grade level standards in math. Met or nearly met all outcome goals.
• Contract Spending: Spent 100% in Y1; has spent 69% as of Y2 Q3.

Site Visit Assessment: Meets program standards. To exceed program standards, program will need to formalize and regularize more of its policies and procedures, and develop the ability to track and analyze more data on program delivery.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool with the exception of no operating reserve.

Renewal Request:

• Proposes to deliver same level of service to same number of children annually (150).
• $209,090 over two years.

Staff Recommendation: Renew as requested.

7. Friends of the Children Mentoring Program

Contract Amount: $500,000

Contract Goals:

Services: Serve 280-300 children annually including at least 20 kindergartners.

Outcomes:

• 65% of mentees will demonstrate improved school attendance or attendance rates of 80% of school days during the school year.
• 65% of mentees will demonstrate decreased behavior referrals or fewer than 10 referrals during the school year.
• 50% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 299 in Y1; is currently serving 255 as of Y2 Q2. Served at least 20 kindergartners each year. Met service goal.
• Outcome Goals: 93% attended school at least 80% of school days; 97% received fewer than 10 behavior referrals during the school year; 59% met standards in reading; 50% met standards in math. Met all outcome goals.
• Contract Spending: Spent 100% in Y1; spent 75% as of Y2 Q3. On track to spend out.

Site Visit Assessment: Exceeds nearly all program standards.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool with the exception of low operating reserves.

Renewal Request:

• Proposes same level of service.
• $522,725 over two years

Staff Recommendation: Renew as requested.

8. IRCO ASPIRE After School Program

Contract Amount: $542,384

Contract Goals:

Services: Home visits for 250; after-school activities for 125; after-summer school activities for 120; monthly activity groups for 3 schools; adult education for 120 parents

Outcomes:

• 80% of program participants will demonstrate developmental asset increases as measured in pre and post surveys.
• 70% of program participants will demonstrate English language development skill gain measured through performance tests in Portland Public School ESL classes.
• 75% of the program participants will demonstrate improved school attendance, lessened behavior referrals or improved grades.
• 50% of the program participants who fail to meet state benchmarks at intake will demonstrate improvements on the next state test.
• 90% of parents will participate in adult education, groups or school activities.
• 30 parents will be recruited, trained, and placed as volunteers in after-school activities.

Contract Performance:

• Service Numbers: Y1: Served 94 children with after-school activities; served 154 parents with a combination of home visits, adult education, family nights and volunteer recruitment and development. No monthly activity groups; district cancelled summer school programs so no after-summer school activities. Failed to meet Y1 service goals due to several factors primarily related to difficulty in hiring and retaining bilingual and bicultural staff to deliver program.

Amended contract at end of Y1 to require the following:

• Social adaptation workshops for 80 families per year;
• Case management for 80 children per year;
• Family support and adult education for 45 adults per year;
• After-school activities for 75 children per year;
• Recruit 30 parent volunteers per year

Y2 Service Numbers: At midpoint of Y2 of contract, grantee had provided the following services:

• Provided 17 social adaptation workshops;
• Case management for 94 youth;
• Family support and education for 36;
• After school activities for 59;
• Recruited 14 volunteers, and averaged 200+ phone outreach contacts, and 30 home visits over last half year.

Generally on track to meet revised service goals with exception of social adaptation groups.

• Outcome Goals: After Y1: No data on developmental asset gains; no data on ESL skill gain; no data on attendance improvement; no data on decreased behavior referrals; no data on improved grades; no data on meeting state academic standards; 81% of parents participated in adult education, or other family activities; 22 volunteers were recruited to work in various aspects of the program but no parents.
• Contract Spending: Spent 50% in Y1; spent 53% as of Q3 Y2.

Site Visit Assessment: Meets program standards.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Proposes to increase number of youth receiving case management services to 90 annually (up from 80) and that at least 81 of these youth will also participate in after-school activities. Proposes to serve 50 parents in 5 family support workshops per year aimed at engaging parents in their children's education. Proposes to recruit 30 volunteers to assist in program delivery. Service intensity is the same as Y2.
• $543,393 over two years.

Staff Recommendation: Authorize renewal up to $407,545 (25% less than requested) on the grounds that grantee has consistently overestimated its expenses and will leave a substantial portion of its first grant unspent. Discuss staffing levels with grantee, particularly volunteer coordinator position since the position costs as much as it generates in in-kind support. Work with grantee to address issues raised by reconfiguration of the schools it is currently serving into K-8 to assure that program can adapt and continue to serve the same number of children with case management and after-school activities.

9. Lifeworks NW: After-School Program at Low Income Housing Developments

Contract Amount: $316,795

Contract Goals:

Services: Provide after-school program services two days per week at 5 sites for 100 children; provide case management services for 20-25 high need children.

Outcomes:

• 50% of the middle school children receiving after-school services will also participate in community service projects.
• 75% of the children receiving intensive after school services will demonstrate decreased behavior referrals as compared with a baseline assessment.
• 75% of the children receiving intensive after-school services will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 178 with after-school program services and 24 with intensive services in Y1; is currently serving 128 children with after-school program services and 22 with intensive services. Exceeded service goals.
• Outcome Goals: Outcome data is for 24 children who received intensive services, not for drop-in services: 79% improved overall academic performance; 13% stayed the same; 8% decreased. 83% decreased behavioral concerns; 8% stayed the same and 8% increased behavioral concerns.
• Contract Spending: Spent 90% of funds in Y1; has spent 69% at end of Y2 Q3; on track to spend out.

Site Visit Assessment: Meets program standards. Grantee does not track any outcomes other than participation on the bulk of the children it serves because they participate on a drop-in basis. Outcomes are tracked only for case managed kids; grantee would need to aggregate more information that is currently collected for individual clients, and would need to do more assessment and outcome tracking on drop-in participants to exceed most program standards.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool with the exception of low reserves.

Renewal Request:

• Proposes to increase level of service for drop-in services from 100 children annually to 120 children annually; number of children receiving intensive services is the same (24 annually); proposes to change curriculum used in after-school program in consultation with CHIF.
• $337,435 over 2 years.

Staff Recommendation: Renew as requested but require that all participants be tracked for outcomes, not just core youth.

10. Metropolitan Family Services: Experience Corp Mentoring Program

Contract Amount: $256,217

Contract Goals:

Services: Provide school-based mentoring services for 105 children in Y1 and 165 children in Y2.

• 75%of mentees will demonstrate decreased behavior referrals.
• 85% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 195 in Y1; currently serving 127 after Q2 of Y2; are currently at 77% of service goal for year. Exceeded goal in Y1; at 77% of goal after Q2 Y2.
• Outcome Goals: From teacher surveys: 73% of students improved reading performance and increased by advancing at least one grade level; 74% of students improved classroom behavior and 79% of students improved peer interactions based on mentor and teacher assessments. No usable data from PPS on reducing referrals and academic achievement because number of participants tracked for these outcomes was too small.
• Contract Spending: Spent 67% in Y1; have spent 65% as of Q3 Y2; are likely to come close to spending out.

Site Visit Assessment: Exceeds most program standards. Areas for improvement are aggregating information on outreach to and recruitment of mentors and getting parental consent for participation in this school day program.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Proposes to serve 240 children annually including an additional school site for service delivery (Davis Elementary). This service goal is a substantial increase over the service numbers projected in the contract but are in accordance with actual service levels over the contract period.
• $288,498 over two years.

Staff Recommendation: Renew as requested; assure that a significant number of participants are analyzed for outcomes by PPS so outcome data can be used.

11. Native American Youth Association: After-School Cultural Arts and Sports Program

Contract Amount: $220,000

Contract Goals:

Services: Y1: provide arts and/or sports programming for 200 Native American children. Y2: amended contract to change service goal to 115 children per year.

Outcomes:

• 50% of program participants will demonstrate improved school attendance or attendance rates of 80% of school days.
• 40% of program participants will demonstrate decreased behavior referrals or fewer referrals than the previous school year.
• 50% of program participants will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 159 in Y1 (although most did not participate regularly and were exited at the end of the year); are currently serving 68 children as of end of Q2 (60% of service goal).
• Outcome Goals: 95% of participants attended school at least 80% of the time; no usable data on decreased behavior referrals and on academic improvement because the number of participants for whom 2 years of data was available was too small. Exceeded attendance goal; no usable data on other outcome goals.
• Contract Spending: Spent 84% of funds in Y1; as of Y2 Q3 spent 64% of funds, will likely come close to spending out.

Site Visit Assessment: Exceeds some program standards and meets all others.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool with exception of low reserves.

Renewal Request:

• Proposes to offer the same services to the same number of children.
• $221,635 over two years.

Staff Recommendation: Authorize renewal up to requested amount but exclude expenses for evaluation of developmental asset outcomes; methodology used for evaluation after Y1 did not provide evidence of progress on asset outcomes.

12. Open Meadow: Step Up After-School Program

Contract Amount: $430,000

Contract Goals:

Services: Provide after-school academic support and arts enrichment for 40-48 middle school aged children at Open Meadow Alternative school and Portsmouth middle school; provide culturally specific family engagement services to 26-30 African immigrant and Latino families with children enrolled in Roosevelt cluster schools.

Outcomes:

• 75% of the children receiving program services will attend 75% of the school days during the school year.
• 65% of the children receiving program services will demonstrate decreased behavior referrals.
• 90% of the children attending the program 90% of the time will stay enrolled in school, attain a 4 point gain in RIT math and reading using statewide assessment tests, and receive passing grades in reading and math classes after the first semester of participation in services.
• 70% of the children attending the program 70% - 90% of the time will stay enrolled in school, attain a 4 point gain in RIT math and reading using statewide assessment tests, and receive passing grades in reading and math classes after the first semester of participation in services.

Contract Performance:

• Service Numbers: Served 94 children in Y1 and are currently serving 56 as of Q2 Y2; served 22 families with culturally specific services in Y1; are currently serving 5 families as of Q2 Y2. Exceeded service goal for children in both years; nearly met goal for families in Y1 but significantly below target for families for Y2.
• Outcome Goals: 91% of participants attended at least 80% of school days; no usable data on decline in behavior referrals; 61% of participants achieved at least a 4 point RIT score gain in reading; and 44% of participants achieved at least a 4 point score gain in math. Breaking participants down by level of participation did not yield a group large enough to generalize outcome data so data is reported for participants as a whole.
• Contract Spending: Spent 100% in Y1; has spent 60% as of Q3 Y2; on track to spend out.

Site Visit Assessment: Exceeds most program standards.

Financial Assessment:

• Expense reporting spot check: Passed; grantee noted billing error and explained that deduction would be made in next billing to account for over-billing.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Proposes to offer the same service to the same number of middle school aged children, but will provide services at the Portsmouth building and George building in addition to Open Meadow Alternative. This is due to the planned move to K-8 schools in this cluster.
• $449,543 over two years.

Staff Recommendation: Renew after-school program; leave open for discussion the renewal of a subcontract for family outreach due to low service in Y2 and the fact that this service is not necessarily provided to families of the children participating in the after-school program. Grantee has not really demonstrated why these services are needed and since the children of the families who are reached through home visits are not participating in the after-school program, there is no accountability for any outcomes for the service. This service is not tied to the delivery of the after-school program. In addition, work with grantee on budgeting and staffing specifics to assure that grantee can provide full service model in two PPS buildings and assure that schools are still supportive of the program as they reconfigure.

13. Oregon Council for Hispanic Advancement: OLI Mentoring Program

Contract Amount: $364,594

Contract Goals:

Services: Provide peer mentoring services to 140 middle school students and family involvement services to 70 of their parents annually.

Outcomes:

• 85%of mentees will demonstrate improved school attendance or attendance rates of 90% of school days during the school year
• 75% of mentees will demonstrate decreased behavior referrals or fewer than 4 referrals during the school year.
• 85% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 147 middle schoolers and 58 of their parents in Y1; currently serving 151 children and 71 of their parents as of Q2 Y2; exceeded service targets.
• Outcome Goals: 86% attended school at least 90% of the time. 86% of participants received fewer than 4 referrals. 54% of participants received at least a "C" grade in math; 62% received at least a "C" in reading .
• Contract Spending: Spent 78% of funds in Y1; spent 69% as of Y2Q3.

Site Visit Assessment: Exceeds most program standards; meets remaining program standards.

Financial Assessment:

• Expense reporting spot check: One exception noted on payroll documentation for one staff person during transition of accounting staff; supporting records were located and auditors noted that new payroll service tracks staff time electronically.
• Organizational Audit: Scored low on operating reliance which measures the ability to pay expenses from program revenues, and a high debt to net assets ratio; other indicators are within parameters of audit tool.

Renewal Request:

• Proposes to offer the same level of service but will include the high school mentors (in addition to the middle school mentees) as part of the population "served" by the program because the grantee spends significant resources locating, training and supporting the mentors to act as mentees. Grantee also proposes to consolidate service locations from 10 to 8 schools and will arrange transportation for affected students, further consolidation may be necessary as the schools in which grantee operates move to a K-8 model.
• $375,747 over 2 years.

Staff Recommendation: Renew at requested financial level and work with grantee to resolve how middle school aged students who will begin attending K-8 school can be reached and served effectively with proposed staffing model.

14. Oregon Health Career Center: AKA Science After-School Program

Contract Amount: $140,000

Contract Goals:

Services: Provide after-school science program to 1,455 youth in Y1, and 1,365 youth in Y2.

Outcomes:

• 75% of the children receiving after-school science instruction will demonstrate improved school attendance or attendance rates of 95% of school days during the school year.
• 50% of the children receiving after-school science instruction will demonstrate decreased behavior referrals or fewer than 2 referrals during the school year.
• 75% of the children receiving after-school science instruction will demonstrate academic achievement by improving academic performance in science either through improved grades, improved results on yearly assessments, meeting grade level standards in science, or meeting state benchmarks in science.

Contract Performance:

• Service Numbers: Served 673 unduplicated children in Y1; 846 duplicated kids (i.e. these are children who took more than one after-school science class during the school year); at 846 they made 58% of Y1 service goal. Served 411 unduplicated children as of Y2 Q2. If they enroll as many students during the winter and spring sessions of the program during the first half of 2006, they will be at 90% of their Y2 goal (using a duplicated count).
• Outcome Goals: Met or nearly met all outcome goals: 64% of participants attended school 95% of school days; 93% of participants had fewer than 2 behavior referrals; 71% of participants met state standards in science for the school year.
• Contract Spending: Spent 94% in Y1; has spent 72% as of Y2 Q3; on track to spend out.

Site Visit Assessment: Meets all program standards and exceeds some.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Low operating reserves, low on liquidity, and high on debt to assets ratio; independent auditors defined grantee as low risk however.

Renewal Request:

• Proposes to serve 1,365 children per year (current level of service for Y2 of contract). No changes to service proposed.
• $146,363 over two years.

Staff Recommendation: Renew as requested with monitoring to assure that they are making significant progress toward the service goal.

15. Portland Impact: Mentoring to Achieve Potential (MAP)

Contract Amount: $205,672

Contract Goals:

Services: Provide one-on-one mentoring services to 100 elementary school children in Year 1, and 123 children in Year 2.

Outcomes:

• 80% of mentees will demonstrate attendance rates at or above 80%.
• 70% of mentees will demonstrate decreased behavior referrals.
• 70% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• Service Numbers: Served 71 children in Year 1 (71% of goal); currently serving 64 children as of Y2 Q2 (52% of goal).
• Outcome Goals: 96% attended school at least 80% of the school days; no data available to measure decreases in behavior referrals; the number of participants for which academic information could be retrieved was too small to generalize any conclusions. The children for whom data was available (12) either improved or maintained standardized test scores. Data on non-contract outcomes showed that for children who participated at least 6 mos: 76% showed improved self confidence; 72% showed improved interactions with peers; 66% showed increased participation in class and motivation to learn; and 59% showed improved behavior and attentiveness.
• Contract Spending: Spent 82% of funds in Y1; have spent 71% of funds as of Y2 Q3; will likely come close to spending out in Y2.

Site Visit Assessment: Meets all program standards and exceeds some.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• Grantee anticipates serving 60 children currently receiving services during a given academic year, and recruiting 60 new children for service in a given academic year, bringing the annual total to 120. No other changes to program are proposed.
• $215,020 over two years.

Staff Recommendation: Renew for $214,279 (maximum allowable based on Y2 budget), but discuss whether it's likely that they will be able to serve the number of children they project given that they have not met service targets during the first contract period.

16. Portland Opportunities Industrialization Center After-School Tutoring Program

Contract Amount: $218,680

Contract Goals:

Services: Provides after-school tutoring services to 40 students at Ockley Green Middle School.

Outcomes:

• 80 % of program participants will demonstrate improved school attendance or attendance rates of 80% of school days.
• 80% of program participants will demonstrate decreased behavior referrals or fewer than 5 referrals during the school year.
• 80% of program participants will demonstrate improved academic achievement by increasing RIT scores by 2-4 points in reading and 2-4 points in math each semester.

Contract Performance:

• o Service Numbers: Year 1 intakes were 110 children, with 40 children regularly attending tutoring services. Grantee is currently serving 50 children; met contractual service targets.
• o Outcome Goals: No data on outcome targets: on average, participants increased RIT scores in reading by 4.88 points, and RIT scores in math by 3.89 points.
• o Contract Spending: Spent 74% of Y1 budget; has spent 55% as of Q3 Y2.

Site Visit Assessment: Meets all program standards and exceeds some.

Financial Assessment:

• o Expense reporting spot check: Passed.
• o Organizational Audit: Scored low on reserves, high debt to net assets ration and a negative operating margin.

Renewal Request:

• o Proposing to offer the same services to the same number of children.
• o $226,600 over two years.

Staff Recommendation: Renew at requested financial level even though Y2 spending percentage is low because Y2 budget included carry-over from Y1 and proposed budget for renewal is close to actual likely spending during second year of existing contract.

17. Portland Public Schools: SUN Schools Expansion

Contract Amount: $540,000

Contract Goals:

Services: Provide extended day services (before and after-school programs) to 675 children annually at Fernwood, Jackson and Sellwood middle schools; agreed that 50% of those served would be below state academic benchmarks.

Outcomes:

• PARTICIPATION: 40% of the program participants will demonstrate improved school attendance from one school year to the next as measured by a decrease in total number of days absentduring the school year.
• BEHAVIOR: 20% of program participants will demonstrate decreased behavior referrals from one school year to the next as measured by total number of behavior referrals during the school year.
• ACHIEVEMENT: 25% of program participants will demonstrate improved grades in math and 25% will demonstrate improved grades in English/language arts/reading between first and second semester grading periods.

Contract Performance:

• o Service Numbers: Served 616 in Y1 (91% of goal); currently serving 390 as of Q2 Y2; on track to meet service goal in Y2. Has not met goal of 50% of children served being below academic benchmarks mostly because there are not enough children in these three schools who fail to meet academic benchmarks. On average, 15% of the students at the 3 schools served are below academic benchmarks. During Y1, approximately 15% of the children served by grantee were below benchmark. Grantee agreed to extensively outreach at schools to recruit children not meeting academic benchmarks into the program. As of 2/6/06, approximately 21% of children served were below benchmark. Grantee has made progress but it is unlikely that it would ever reach the 50% goal.
• o Outcome Goals: Met achievement and attendance goals: 35% of participants improved school attendance from one school year to the next; the data on behavior referrals was not adequate to measure a decrease in referrals from one year to the next; 11% of participants improved grades from the beginning of the school year to the end; 58% of participants made RIT score gains in reading of 4 points or more; and 41% of participants made RIT score gains in math of 4 points or more.
• o Contract Spending: Spent 84% in Y1; have spent 65% as of Y2 Q2.

Site Visit Assessment: Exceeds most program standards and meet all other standards.

Financial Assessment:

• o Expense reporting spot check: Passed
• o Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• o Proposes to offer the same level of service to the same number of children at the same schools. Proposes that 50% of children served will be designated as "at-risk" for academic failure (as opposed to failing to meet state standards) which is defined as children who meet one or more of the following criteria: not meeting state or district academic standards, performing below grade level; having chronic attendance problems, English language learner, homeless, in foster care and/or referred by school staff.
• o $564,543 over two years.

Staff Recommendation: Renew as requested but discuss the likely number of children who will fit the "at-risk" definition at the schools served to determine whether 50% is an appropriate service target. Given that CHIF funds were used to start new SUN programs at these schools, and that 2 of the schools currently served are likely to be affected by the PPS re-configuration, staff thinks it is prudent for PPS to continue to administer this grant for another 2 years. However, should CHIF consider funding these SUN programs beyond the next two years, staff recommends that we contract directly with the providers who manage the sites.

18. Self Enhancement Inc. After-School Program

Contract Amount: $671,730

Contract Goals:

Services: Provide after-school program services and parent engagement services to 150 primarily African-American children annually.

Outcomes:

• 75% of the program participants will demonstrate improved school attendance or attendance rates of 90% of school days.
• 70% of the program participants will either have no behavioral referrals or will improve their behavior as measured by fewer referrals per quarter or an increase in positive teacher comments.
• 60% of the program participants will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• o Service Numbers: Served 180 children in Y1; currently serving 188 children; exceeded service goal.
• o Outcome Goals: 83% of participants attended 80% of school days; 69% had no behavior referrals; of those who had 1 or more behavior referrals, 23% decreased their referrals over the course of the school year; 91% met state standards in reading and 65% met state standards in math. Met all outcome goals.
• o Contract Spending: Spent 80% of funds in Y1; has spent 68% of funds as of Q3 Y2; on track to spend out this year.

Site Visit Assessment: Exceeds most program standards.

Financial Assessment:

• o Expense reporting spot check: Passed.
• o Organizational Audit: Indicators of financial health fall within the parameters set by the audit tool.

Renewal Request:

• o Proposes to serve the same number of children in its after-school program and will enhance its program with a new offering of advanced art classes for those participants who are interested.
• o $702,260 over two years.

Staff Recommendation: Renew at $678,260; $24,000 reduction in transportation budget because in Y1 they only spent 38% of their $88,800 transportation budget, and based on spending trends in Y2 are likely to spend only 75% of their $68,500 transportation budget. Renewal application calls for transportation budget of $65,000; reduce this line item to $53,000 annually.

19. Tears of Joy: Higher Stages After-School Arts Program

Contract Amount: $100,000

Contract Goals:

Services: Serve 280 children in five schools annually with the Higher Stages after-school arts program; offer 45 classes annually.

Outcomes:

• 75% of the program participants will demonstrate improved school attendance or attendance rates of 90% of school days.
• 70% of the program participants will either have no behavioral referrals or will improve their behavior as measured by fewer referrals per quarter or an increase in positive teacher comments.
• 60% of the program participants will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.

Contract Performance:

• o Service Numbers: Served 261 children in Y1 (93% of goal); are currently serving 102 as of Q2 Y2; on track to meet service goal in Y2.
• o Outcome Goals: Grantee was not able to collect any usable data on contract outcomes.
• o Contract Spending: Spent 100% in Y1; has spent 75% as of Q3 Y2; on track to spend out.

Site Visit Assessment: Meets program standards.

Financial Assessment:

• o Expense reporting spot check: Billing pro-rata quarterly shares, but auditors confirmed that quarterly expenses exceed bills to CHIF.
• o Organizational Audit: High debt to assets ratio and low reserves but making significant progress on debt.

Renewal Request:

• o Proposes serve 225-275 children annually at 5 newly configured K-8 schools in the Jefferson cluster (Beach, King, Vernon, Faubion and Ockley Green). Proposes to discontinue service at Tubman which is slated to become an all-girls school, and to add Faubion which is slated to become a K-8. Classes will be reconfigured to serve the relevant populations at each school.
• o $103,000 over two years.

Staff Recommendation: Renew as requested.

20. Trillium Family Services: Family of Friends Mentoring Program

Contract Amount: $100,000

Contract Goals:

Services: Facilitate families mentoring 30 at-risk children in Y1, and 60 children in Y2. Families volunteer to be mentors and receive training to do so.

Outcomes:

• 97% of mentees will demonstrate improved school attendance or attendance rates of 80 % of school days during the school year.
• 95% of mentees will demonstrate decreased behavior referrals or fewer than 3 referrals during the school year.
• 80% of mentees will demonstrate academic achievement either through improved grades, improved results on yearly assessments, meeting grade level standards in reading and math, or meeting state benchmarks in reading and math.
• 85 % of mentees will demonstrate increased positive self-concept.
• 97% of mentees will experience positive relationships with their mentor families.
• 90% of mentees will maintain or improve social functioning.
• 100% of mentees will participate in at least one civic service activity.
• 75% of mentees' relationships with his or her family will be maintained or improved.

Contract Performance:

• Service Numbers: Served 13 children in Y1; are currently serving 16 children in Y2 as of Q3; substantially overestimated the number of children the program could serve.
• Outcome Goals: 80% of mentees improved homework completion and 60% of mentees improved school attendance according to parents (i.e. based on parent surveys); PPS could not release data on school attendance, behavior and academics because the number served is too few. Depending on the total number served in Y2, it's possible that data may be available in future years.
• Contract Spending: Spent 100% in Y1; has spent 77% as of Y2 Q2; on track to spend out.

Site Visit Assessment: Meets program standards.

Financial Assessment:

• Expense reporting spot check: Passed.
• Organizational Audit: Low score in operating reliance which measures ability to pay total expenses from program revenues; and low current ratio which measures liquidity and ability to meet obligations as they come due.

Renewal Request:

• Proposes to serve 60 children per year with same services.
• $103,000 over 2 years[1]

Staff Recommendation: Renew at $103,000 for but work with grantee to set a reasonable service goal; based on past performance it is unlikely that they will serve 60 children per year.

Grantee proposes to measure the same outcomes, however was not able to report on these outcomes after Year 1. Pare down outcomes to be measured and consider alternative methods for measuring outcomes if numbers served remain too small for program to be included in PPS analysis.

[1] This amount is slightly higher than the amount grantee actually requested because grantee calculated the amount it was eligible to receive through renewal incorrectly.