This page is meant to serve as a reference for ECA applicants and current award recipients who are required to or interested in integrating the MODE Framework into their program. For those unfamiliar with the MODE Framework or who would like more information, please take your time and read the full page, which provides in-depth explanations of the process and suggested resources. For those of you who only need a general understanding of the MODE Framework, please find the most relevant resources below:
- ECA MODE Framework Implementation Overview PDF
- ECA MODE Framework Indicator Selection Webinar MP4
- ECA MODE Framework Indicator Book (PDF)
- ECA Evaluation Community of Practice MODE Framework Discussion YouTube
Introduction of the Monitoring Data for ECA Framework
In 2019, the Bureau of Educational and Cultural Affairs (ECA) Evaluation Division led an initiative to redesign the performance monitoring process across the Bureau. The development of the Monitoring Data for ECA (MODE) Framework was a collaborative process in which the Evaluation Division worked closely with ECA program teams, senior leadership, award recipients, and regional Bureaus and select Embassies to ensure the framework created is responsive to the Bureau’s data needs.
The MODE Framework includes a standardized Bureau-wide Results Framework with goals, objectives and indicators designed to track program performance and the direction, pace, and magnitude of change for ECA programs, leading to strengthened feedback mechanisms resulting in more effective programs. Each performance indicator has corresponding data collection questions defined to facilitate uniform data collection to ensure data validity and reliability. The objectives in the ECA Results Framework are mapped to the Functional Bureau Strategy (FBS) and the National Security Strategy through ECA’s Strategy Crosswalk (both the FBS and Strategy Crosswalk can be found here) to ensure that all indicators can be tied to each level of strategic plans.
The MODE Framework allows the Bureau to standardize and utilize its data in the following ways:
- Assess data and performance metrics to enhance program performance
- Inform strategic planning activities at the Bureau, division, and individual exchange program levels
- Supplement the information ECA Program Officers receive from their award recipients and exchange participants to provide a comprehensive view of programmatic activities
- Respond quickly and reliably to ad-hoc requests from Congress, the Office of Management and Budget (OMB), and internal Department of State stakeholders
Standardized Data Collection
To ensure programs across ECA are collecting performance monitoring data consistently, the MODE Framework consists of a series of standardized indicators with corresponding survey questions and data collection instructions. An overview of each indicator can be found in the ECA MODE Framework Indicator Book. The ECA MODE Framework Glossary defines key terms and concepts used or explored within the MODE Framework.
*To ECA Applicants: ECA is currently in the process of piloting the MODE Framework in its solicitations. If you do not see the information described below in a solicitation you are bidding on, then you are not required to provide any of this information. We plan to roll out the MODE Framework throughout the Bureau in waves; so please stay tuned.*
The MODE Framework is operationalized through the solicitation process. Each solicitation includes a program-specific results framework with relevant objectives, sub-objectives, and indicators so applicants understand the monitoring and evaluation (M&E) and reporting requirements. For the proposal and resulting cooperative agreement(s)/grant(s), the Bureau recommends the applicant include that results framework as the minimum requirement for measurement. The applicant is also able to propose objectives and indicators they feel should be added to sufficiently measure the relevant proposed program outputs, outcomes, and results over the course of the proposed period of performance. If more information is needed on how to develop objectives and indicators, the Evaluation Division has a number of internal and external resources on our Capacity-Building page.
The solicitation language also details the requirement for applicants to submit proposals that include:
- A Performance Monitoring Plan (PMP) establishes the monitoring plan for the award by outlining how an applicant will track progress towards the proposed program’s goals and objectives through indicators and corresponding data collection questions. The PMP will be pre-populated with the standard indicators expected to be measured throughout the award, and will be included as part of the solicitation package. Applicants are not required to use the ECA provided PMP, though it is strongly encouraged they do so as all content present in the PMP should be included in any alternative format the applicant submits. Applicants are also able to include any additional objectives and/or indicators they think are relevant to measuring program objectives and goals. If you have questions about how to read or fill out the PMP, please watch our webinar here.
- An M&E Narrative section outlines how an applicant intends to measure the indicators outlined in the PMP. The narrative should explain the resources, mechanisms, and data analysis and reporting procedures the applicant possesses to perform the required performance monitoring, while also demonstrating the applicant’s dedication to utilizing performance monitoring data through planned learning mechanisms and feedback loops.
The Bureau's official knowledge management system, State Assistance Management System Domestic (SAMS-D), includes a Results Performance Monitoring (RPM) module used by award recipients to report aggregate data on the MODE Framework. The RPM module builds on the functionality that Program Officers and award recipients are familiar with and includes the ability to explore and pull standardized reports on the available data. As part of the reporting requirements, you will also be required to provide ECA with the raw data for evaluation surveys in a CSV file. Please see the Data Submission Instructions and accompanying Example Data File for more information.
The MODE Framework includes the requirement for ECA award recipients to report their approximate spending for M&E activities in order to estimate the proportion of the Bureau’s overall budget. There will be no comparisons made between applicants on funding amounts during the proposal review process, nor does this information affect current awards or future funding decisions in any way. The sole purpose of this requirement is to gather the data necessary to establish and maintain an accurate estimate of the Bureau’s total spending on M&E that encompasses the activities of ECA award recipient staff, ECA program staff, and the Evaluation Division. A list of defined M&E activities can be found in the M&E Budget Tracking Guidance.
Post-Program Data Collection Efforts
As part of the MODE Framework, the Evaluation Division is responsible for surveying alumni, home, and host communities after a program has ended. Alumni are surveyed 1-, 3- and 5-years after their program participation, while host/home communities are surveyed one year after their hosting experience. Data collection after the program ends allows ECA to have a better understanding of the long term effects of programs on all affected stakeholders: alumni, host communities, as well as home communities.
All MODE Framework reference and guide documents included on this page can be found here.