2018
Invest in What Works Federal Standard of Excellence
(100 points possible)
Did the agency have a senior staff member(s) with the authority, staff, and budget to evaluate its major programs and inform policy decisions affecting them in FY18? (Example: Chief Evaluation Officer)
Did the agency have an evaluation policy, evaluation plan, and research/learning agenda(s) and did it publicly release the findings of all completed evaluations in FY18?
Did the agency invest at least 1% of program funds in evaluations in FY18? (Examples: Impact studies; implementation studies; rapid cycle evaluations; evaluation technical assistance, and capacity-building)
Did the agency implement a performance management system with clear and prioritized outcome-focused goals and aligned program objectives and measures, and did it frequently collect, analyze, and use data and evidence to improve outcomes, return on investment, and other dimensions of performance in FY18? (Example: Performance stat systems)
Did the agency collect, analyze, share, and use high-quality administrative and survey data - consistent with strong privacy protections - to improve (or help other entities improve) federal, state, and local programs in FY18? (Examples: Model data-sharing agreements or data-licensing agreements; data tagging and documentation; data standardization; open data policies)
Did the agency use a common evidence framework, guidelines, or standards to inform its research and funding decisions and did it disseminate and promote the use of evidence-based interventions through a user-friendly tool in FY18? (Example: What Works Clearinghouses)
Did the agency have staff, policies, and processes in place that encouraged innovation to improve the impact of its programs in FY18? (Examples: Prizes and challenges; behavioral science trials; innovation labs/accelerators; performance partnership pilots; demonstration projects or waivers with strong evaluation requirements)
Did the agency use evidence of effectiveness when allocating funds from its five largest competitive grant programs in FY18? (Examples: Tiered-evidence frameworks; evidence-based funding set-asides; priority preference points or other preference scoring; Pay for Success provisions)
Did the agency use evidence of effectiveness when allocating funds from its five largest non-competitive grant programs in FY18? (Examples: Evidence-based funding set-asides; requirements to invest funds in evidence-based activities; Pay for Success provisions)
In FY18, did the agency shift funds away from or within any practice, program, or policy that consistently failed to achieve desired outcomes? (Examples: Requiring low-performing grantees to re-compete for funding; removing ineffective interventions from allowable use of grant funds; proposing the elimination of ineffective programs through annual budget requests)
**Meeting this criteria requires both federal agency and congressional action.
¹Results for America gave SAMHSA several opportunities to review and edit the information in this document, but it declined to do so. Results for America’s 2018 Invest in What Works Federal Standard of Excellence, therefore, includes information from the 2017 Standard, which SAMHSA helped develop, as well as new information posted on the SAMHSA website between October 2017 and September 2018.
²Results for America was unable to determine the amount of resources SAMHSA invested in evaluations in FY18 for criterion #3. Therefore, to tally a final score, Results for America scored criterion #3 a 0 and reduced the denominator from 100 to 90 points.
³MCC only administered competitive grant programs (and no non-competitive grant programs) in FY18. Therefore, to tally a final score, Results for America doubled the score for criterion #8 (9×2=18) and awarded 0 points for criterion #9.
⁴USAID only administered competitive grant programs (and no non-competitive grant programs) in FY18. Therefore, to tally a final score, Results for America doubled the score for criterion #8 (8×2=16) and awarded 0 points for criterion #9.
Results for America’s 2018 Invest in What Works Federal Standard of Excellence highlights the extent to which the Administration for Children and Families (within HHS); Administration for Community Living (within HHS); Corporation for National and Community Service; Millennium Challenge Corporation; Substance Abuse and Mental Health Services Administration (within HHS); U.S. Agency for International Development; U.S. Department of Education; U.S. Department of Housing and Urban Development; and U.S. Department of Labor have built the infrastructure necessary to be able to use data, evidence, and evaluation in budget, policy, and management decisions.
It is important to note that:
- Results for America developed the standard’s criteria and scoring structure in close consultation with more than 75 current and former federal government officials and key stakeholders from all across the country.
- The purpose of the standard is to educate members of the general public as well as public, private, and nonprofit sector leaders on how federal departments and agencies are currently using data, evidence, and evaluation to invest taxpayer dollars in what works.
- Results for America gave the federal departments and agencies included in the standard multiple opportunities to review and comment on the content and presentation of the information included in it. Results for America greatly appreciates their willingness to help develop this document and their continued commitment to making the federal government as effective and efficient as possible. Since Results for America recognizes that it is very difficult to distill complex practices, policies, and programs into a single cross-agency scorecard, Results for America exercised its best judgment and relied on the deep expertise of leaders both within and outside of the federal government during the development of the standard.
- Results for America released six previous versions of the Invest in What Works Federal Standard of Excellence, formerly entitled as the Invest in What Works Index, in June 2013, September 2013, May 2014, March 2015, April 2016, and October 2017.
- Read the press release for highlights of the 2018 Invest in What Works Federal Standard of Excellence.