This page is meant to serve as a reference for ECA applicants and current award recipients who are required to or interested in integrating the Monitoring Data for ECA (MODE) Framework into their programs. For those unfamiliar with the MODE Framework or who would like more information, please take time to read the full page, which provides in-depth explanations of the process and suggested resources. For those who only need a general understanding of the MODE Framework, please find the most relevant resources below:

To answer questions about the ongoing MODE implementation process and for one-on-one assistance for current award recipients, the MELI Unit is currently hosting office hours Tuesdays through Thursdays from 12:00 pm to 2:00 pm EST. Please email ECAEvaluation@state.gov to schedule an appointment during these times.

Implementing MODE
The MODE implementation process starts with the MELI Unit working with the ECA Program Offices to determine which indicators best fit the program.  The MODE Framework is then operationalized through pre-award documents (solicitations/negotiation letters).

Solicitation Process
Each solicitation includes a program-specific performance monitoring plan with relevant objectives, sub-objectives, and indicators so applicants understand the monitoring and evaluation (M&E) and reporting requirements.

The solicitation language also details the requirement for applicants to submit proposals that include:

  • Performance Monitoring Plan (PMP) that serves as the primary reference document for performance monitoring.  The PMP outlines how an applicant will track progress towards the proposed program’s goals and objectives through indicators and corresponding data collection questions. The PMP will be pre-populated with the standard MODE Framework indicators expected to be measured throughout the award and will be included as part of the solicitation package. Applicants are not required to use the ECA provided PMP, though they are strongly encouraged to do so as all content present in the PMP should be included in any alternative format the applicant submits. Applicants are also able to include any additional custom objectives and/or indicators they think are relevant to measuring program objectives and goals. If you have questions about how to read or fill out the PMP, please watch our webinar here.
  • An M&E Narrative section that outlines how an applicant intends to measure the indicators outlined in the PMP. The narrative should explain the resources, mechanisms, and data analysis and reporting procedures the applicant possesses to perform the required performance monitoring, while also demonstrating the applicant’s dedication to utilizing performance monitoring data through planned learning mechanisms and feedback loops.

MODE Setup
The Bureau's official knowledge management system, State Assistance Management System Domestic (SAMS-D), includes a Results Performance Monitoring (RPM) module, which is the system of record for reporting on MODE indicators. As part of the award process, award recipients are responsible for reviewing and approving the performance monitoring content in SAMS-D RPM. All indicators are entered and tracked through this module. The MELI Unit has created a training video to explain the SAMS-D RPM process.

Award recipients should plan to: review indicators, enter targets, and approve the monitoring plan through the SAMS-D RPM module.  The MELI Unit has recorded a webinar that explains how to create targets. For more information about how the SAMS-D RPM module is set up, please see the MODE Process Guide and FAQs for Award Recipients document.

Data Collection
Award recipients are responsible for administering a post-program survey to collect responses from participants. Each MODE indicator has corresponding data collection questions which are designed to ensure uniform data collection and data validity and reliability. Survey questions, including sample consent language and required demographic questions, can be found in the MODE Indicator Book as well as the PMP. Survey questions have also been translated into six languages, those translations can be found in this file.The Performance Indicator Reference Sheets (PIRS) provide a more detailed explanation of data collection.

Reporting
The SAMS-D RPM module builds on the functionality that Program Officers and award recipients are familiar with and includes the ability to explore and pull standardized reports on the available data. Therefore, award recipients are expected to report aggregate data into the RPM on any programs completed during the reporting period outlined in the award.  The MODE Performance Indicator Reference Sheets (PIRS) contain instructions on how to calculate the responses from survey questions and transform them into the numbers and percentages that should be reported.

In addition to reporting in the RPM, award recipients are required to provide ECA with the raw data for evaluation surveys. The MELI Unit currently offers two methods for data submission. Award recipients may use Qualtrics, the MELI Unit’s survey platform, or they can submit raw data semi-annually. Please see the Data Submission Instructions (forthcoming) for more information.

The MODE Framework includes the requirement for ECA award recipients to report their approximate spending for M&E activities in order to estimate the proportion of the Bureau’s overall budget. There will be no comparisons made between applicants on funding amounts during the proposal review process, nor does this information affect current awards or future funding decisions in any way. The sole purpose of this requirement is to gather the data necessary to establish and maintain an accurate estimate of the Bureau’s total spending on M&E that encompasses the activities of ECA award recipient staff, ECA program staff, and the MELI Unit. A list of defined M&E activities can be found in the M&E Budget Tracking Guidance.

Additional Resources
To develop the MODE Framework, the MELI Unit reviewed relevant literature, data collection instruments, and definitions for key terms. The resulting MODE Literature Review serves as an additional resource for award recipients who may be interested in collecting data beyond the requirements of the MODE Framework.

For award recipients looking to identify and code themes in their qualitative data (not required as part of the MODE Framework), the MELI Unit has developed a Qualitative Coding Guide  which identifies common themes that align with each objective of the MODE Framework.

Please see the list of the most relevant MODE resources below. A complete list of MODE Framework reference and guide documents can also be found here.

Please send questions and comments to ecaevaluation@state.gov.