Impact Study Evaluator

AmeriCorps Promise Fellows Motivational Interviewing Project 

Please see attached file for full RFP and additional information

Organizational Background

Minnesota Alliance With Youth's AmeriCorps Promise Fellow program focuses on student dropout prevention by supporting students in grades 6-12 most at risk of not graduating from high school. Our theory of change is that providing schools and organizations with the support and individuals to deliver a data-informed, evidence-based model based on the early warning indicators of high school dropout will result in increased school attendance and academic engagement for off-track students in grades 6-12. 

Scope and Purpose of Evaluation

A previous study observed statistically significant differences in attendance rates when comparing students served by a Promise Fellows to a matched comparison group of students not served by a Promise Fellow (Van Norman 2022). After establishing evidence of overall program impact, we conducted a study in 2025 to evaluate the impact of a specific intervention strategy called Motivational Interviewing (MI). This study showed that students served by Promise Fellows who used MI with fidelity saw improved attendance outcomes (Terra Luna 2025).

We now aim to conduct a follow-up study to determine which program modifications will lead to higher quality implementation of MI and in turn, improved student outcomes. MI is a fundamental component of member training and program delivery; however, we recognize that the quality of implementation could be improved by more regular training and observational support as the value of those ongoing supports has been demonstrated in previous empirical research. 

Evaluation Outcomes of Interest and Questions

Over this two year study, we intend to provide Promise Fellows with two different types of Motivational Interviewing training and support - monthly MI training refreshers in 2026-27 and access to an online training module in 2027-28.

The primary outcome of interest in the evaluation is student attendance, and the secondary outcome of interest is student engagement. In addition, we will measure member-level implementation of Motivational Interviewing, expressed as the average fidelity rating of MI implementation in two observations. 

The following research questions are of primary interest for this evaluation:

To what extent does student school attendance differ based on the method of MI training and support a Promise Fellow receives?

To what extent does student school engagement differ based on the method of MI training and support a Promise Fellow receives?

To what extent does the quality of MI implementation differ based on the method of MI training and support a Promise Fellow receives?

Deliverables 

• Engage with Minnesota Alliance With Youth staff throughout the evaluation timeline 
• Design an evaluation plan for the project outlining data collection methods, sampling technique, evaluation approach and instruments 
• Collect and analyze data measuring Motivational Interviewing and student attendance 
• Provide final written report on evaluation findings 
• Provide regular updates to Minnesota Alliance With Youth staff lead, as agreed upon by both parties 
• Provide Alliance with suggestions for improving the data collected
• Submit invoices for work performed quarterly 

Project Timeline

September 2026: Evaluator begins meeting with Alliance team to design and prepare for study.

September 2026 - June 2027: Promise Fellow Program Year 1

• October 2026: Fellows receive Intro to MI Training
• November 2026 - May 2027: Fellows receive monthly MI refresher trainings
• January and April 2027: Fellows are observed using MI and receive a BECCI fidelity score
• August 2027: De-identified data file shared with program evaluator
• Fellow training attendance
• Fellow MI fidelity scores from two observations
• Student outcomes (attendance and engagement)

September 2027 - June 2028: Promise Fellow Program Year 2

• October 2027: Fellows receive Intro to MI Training
• November 2027 - May 2028: Fellows receive access to online tool of MI resources and self paced training modules
• January and April 2028: Fellows are observed using MI and receive a BECCI fidelity score
• August 2028: De-identified data file shared with program evaluator
• Fellow training attendance
• Record of Fellow use of online tool
• Fellow MI fidelity scores from two observations
• Student outcomes (attendance and engagement)

August - September 2028: Evaluator analyzes data and prepares report

October 1, 2028: Preliminary draft of evaluation complete.

November 1, 2028: Final draft of evaluation complete.

Expected connections with Alliance throughout study:

• Sept-October 2026: 1-3 planning meetings to set study expectations and prepare for Program Year 1
  
  
• November 2026 - August 2027: brief occasional check-ins to discuss implementation and data collection, approx 30 min/month
   
• Sept-October 2027: 1-2 planning meetings to review data from Program Year 1 and prepare for Program Year 2
  
  
• November 2027 - July 2028: brief occasional check-ins to discuss implementation and data collection, approx 30 min/month
  
  
• August - November 2028: data analysis and report creation, meet monthly to discuss preliminary findings and drafts of report