Harnessing Impact 评价s for All Levels of Policymaking
As America continues to grapple with the COVID-19 p和emic, public leaders will need to decide what initiatives to incorporate in efforts to rebuild the economy. Successful initiatives will deliver broad benefits; impact evaluations conducted by economists 和 social scientists can guide local, 状态, 和 national decisionmakers to those choices.
首先,也是最重要的, using evaluations to get the maximum impact from national initiatives will be essential. This requires measuring how much a given initiative contributes to the country’s rising economic tide 在全国代表性的基础上 (rather than in a smattering of places most convenient for investigation) since federal funds will be spent 和 results will be needed for the entire nation.
This principle—assess the effectiveness of government interventions in a representative set of sites to get the most reliable national policy guidance—applies in many contexts. In a recent presentation to officials at the U.S. 劳工部, 2 趣赢平台 researchers studying how to make evaluations stronger 和 more meaningful reinforced this principle.
“If a rigorous impact evaluation lacks a well-defined target population of individuals whom the policy seeks to assist 和 neglects to have a representative sample from that population, findings will be misleading to policymakers,韦斯特的斯蒂芬·贝尔解释道, Ph.D.和罗布·奥尔森博士.D.
完善国家政策指导
他专注于劳动力政策研究. 贝尔, a 趣赢平台 Vice President 和 Senior Statistical Fellow, 和 Dr. 奥尔森, 教育研究部副主任, discussed ways to increase “external validity” to make findings more generalizable 和, 因此, improve the reliability of findings informing national policy. “This will require evaluators to select a nationally representative sample of sites 和 to incentivize site participation,博士说. 奥尔森.
在过去的10年里. 贝尔和奥尔森与dr. Elizabeth Stuart 和 Larry Orr of Johns Hopkins University, have examined multisite impact evaluations to interpret how well their findings represent the nation or individual 状态s 和 localities seeking policy guidance. Their pioneering research—including recommendations 和 innovative tools for making findings more representative of the target population—has been published in such outlets as the 政策分析杂志 和管理 和 教育评价与政策分析.
迈向更一般化发现的步骤
为了支持全国推广，dr. 贝尔 和 奥尔森 recommend 4 steps that impact evaluations can take to better inform national policy decisions:
- Identify the population which the studied intervention hopes to affect.
- Select a representative sample of sites where that population lives.
- Use creative study design features 和 financial incentives to encour年龄 site participation.
- Choose replacement sites wisely when some sites opt out.
These steps play a crucial role for evaluations of labor policy 和 in other policy areas such as education. 见,例如, 教育 Impact 评价s: Finding Ways to Enhance Reliability.
在当地推断影响证据
状态s 和 localities also need sound evidence about program effectiveness. “Few social program evaluations in the United 状态s are specific to a single 状态 or metropolitan area,博士说. 贝尔. “即使是在大规模的国家评估中, sample sizes for individual political subunits are rarely large enough for separate analysis, so the push for ‘evidence-based’ policymaking doesn’t work well for, 说, the 状态 of Ohio or the City of San Antonio.”
“To draw localized measures of impact from multisite evaluations—for workforce programs 和 for social policy generally—statistical adjustments need to be made,博士说. 奥尔森. “We are striving to learn how well such adjustments can work.”
未来会怎样?
“The team at 趣赢平台 和 Johns Hopkins intends to be the catalyst for ensuring that external validity becomes as high a priority as internal validity in social policy impact evaluations,博士说. 奥尔森. “We believe that already existing methods for improving external validity should 和 can be comprehensively adopted over the next 5 to 10 years.“但这还不够。. 贝尔. “Our impact evaluation team pushes innovative approaches designed from the get-go to deliver reliable policy guidance to all levels of government.”
