A well-informed, knowledgeable bureaucracy is a necessary precondition for effective policy design and implementation. Unfortunately, real-word bureaucracies fall well-short of this ideal due to limited information, monitoring, and accountability of bureaucrats. Subnational performance assessments (SPAs) are increasingly used by international organizations, policy groups, and governments to combat poor bureaucratic performance and improve the performance of local governments around the world [1]. By creating objective rankings of subnational administration on outcomes such as democracy, economic governance, transparency, and corruption control, SPAs should, in principle, inspire constructive regional competition, enable voters to hold leaders accountable, identify best practices for policy diffusion, and inform flows of labour and capital to the most productive locations.
Despite such lofty goals, rigorous empirical tests have disappointingly failed to deliver clear-cut evidence of the benefits of SPAs on their stated objectives — including, most importantly, sustained improvements in governance. The growing consensus that bureaucratic performance is a primary determinant of state effectiveness renders the streamlining of SPA interventions all the more essential.
Why are these theoretically grounded interventions not working in practice? By analysing twenty development studies that use randomised information interventions to test the impacts of SPAs, we uncovered two explanatory trends for their lack of success: (i) the targeting of SPAs to constituents or voters rather than bureaucrats themselves, and (ii) the dominance of unidirectional delivery such as flyers, infographic scores cards, and videos, instead of bidirectional delivery that encourages feedback and understanding of the measures. To empirically address the validity of these trends, we conducted a randomised experiment where the dissemination of a nationwide SPA was delivered through carefully designed facilitation workshops that worked directly with bureaucrats in Myanmar townships. These facilitation workshops addressed what we saw as the two major shortcomings of existing SPAs by (i) engaging with bureaucrats directly, and (ii) allowing participant bureaucrats the opportunity to ask questions about the SPA results, speak to other bureaucrats, and work together to formulate policy responses.
Experimental Design
Leveraging insights from the literature, we cooperated with The Asia Foundation (TAF) office in Myanmar to employ facilitation workshops in between the dissemination of their Myanmar Business Environment Index (MBEI). TAF’s MBEI project combined panel survey data from 4,874 small and medium size enterprises (SMEs) with administrative and observational data to rank them along ten dimensions of economic governance—for example Entry Costs, Transparency, and Infrastructure—in all fourteen Myanmar states and regions (S/Rs). The MBEI focuses on indicators under the jurisdiction of bureaucrats and thus could be altered by improvements in bureaucratic performance. The 2018 wave of the index collected data in 60 randomly drawn townships — the unit primarily responsible for business government interactions in the country. Prior to the collection of data for the 2020 wave, TAF organised facilitation workshops to disseminate results of the 2018 MBEI to township leaders and bureaucrats.
The workshops were conducted by substantive experts who trained the officials on reading and understanding their MBEI rankings, offered opportunities for cross-departmental deliberation and feedback, worked to generate agreement on concrete policy solutions for problems that were identified, and set measurable goals for improvement. As such, the treatment went beyond unidirectional SPA provision and focused on internalising and coordinating information across teams, leading to improved governance.
Our treatment was whether the township had a facilitation workshop or not. Twenty townships from the 60 townships in the 2018 wave of the MBEI were randomly assigned to receive the workshops. To mirror the traditional delivery of SPAs, all S/R leaderships received written copies of the 140-page technical report to share with township leaders, a brief pamphlet on key indicators, and access to an online widget that allowed them to compare the 10 subindices to other townships in their S/R and Myanmar more generally. The treatment group received these same reports, but with the additional benefit of intensive facilitation workshops.
Main Results: Facilitation Workshops Work
Upon analysing changes in the MBEI index between the 2018 and 2020 waves, we find that firms in control townships reported improvements of about four points on a 100-point index (see figure below). On the other hand, firms in treatment townships reported governance improvements of nearly eight points, indicating that facilitation workshops more than doubled the recorded reform achievements. These findings are especially impressive when we consider that the bureaucrats had less than a year to implement their reforms after the workshop. This means that the improvements from the facilitation workshops were observed very quickly!
The Role of Decentralization in SPA Effectiveness
We also explore the role of decentralization in the relationship between facilitation, accountability, and improved governance. Taking advantage of Myanmar’s unique model of incomplete decentralisation at the time of the experiment, we observed the differences in MBEI improvements between the decentralised Development Affairs Office (DAO) and the highly centralised General Administrative Department (GAD). Decentralisation scholarship suggests that high levels of decentralisation support more bottom-up learning and greater embeddedness of local bureaucrats. We therefore test the idea the positive relationship between facilitation workshops and improved governance would be greater among administrative units with higher levels of decentralisation.
In line with decentralization theory, we find a significant and positive relationship of facilitation workshops of about half a point on an index consisting of indicators solely under the control of the decentralized DAO. These included reductions in regulatory burdens and inspections and local infrastructure, such as lighting and local roads. Conversely, there is no significant improvement on a parallel GAD index, which includes indicators under the jurisdiction of the heavily centralised GAD. This reveals that facilitation is conditioned by decentralization, as local officials are more embedded in local populations, possess greater local knowledge, are more accountable to citizens through direct elections, and have greater ability to coordinate among different units to deliver effective results.
Conclusion
Facilitation workshops not only provide valuable information on which reforms best improve economic outcomes, but also act as a productive opportunity for bureaucrats to coordinate their reform agenda, which can lead to large and immediate improvements in overall bureaucratic performance. These workshops may substantially enhance the effectiveness of SPAs on policy making, even in contexts where accountability channels are muddled and perhaps especially when administrative responsibilities are not clearly delineated. Furthermore, the effects of facilitation workshops on SPA effectiveness are moderated by the level of autonomy and embeddedness of local officials. This finding contributes to important political economy work on the benefits of decentralization for economic development and breaks new ground in the use of SPAs by development practitioners to significantly improve bureaucratic performance.
[1] Examples of well-known subnational performance assessments are the World Bank’s subnational doing business studies and the Provincial Competitiveness Index in countries Vietnam, Malaysia, Cambodia, Bangladesh and Myanmar.
Image: Christian Schnettelker
This blog piece is based on the forthcoming Journal of Politics article “Facilitating Development: Evidence from a National-Level Experiment on Improving Bureaucratic Performance in Myanmar” by Edmund Malesky and Dean Dulay.
The empirical analysis has been successfully replicated by the JOP and the replication files are available in the JOP Dataverse.
About the Authors
Dean Dulay is an Assistant Professor of Political Science at Singapore Management University. I completed my PhD in Political Science (with a focus on Political Economy) at Duke University. His research interest centers on Historical Political Economy and the Political Economy of Development. Examples of his research include the impacts of political dynasties on economic policy, the long-run effects of missionary state-building, and how facilitation workshops improve local bureaucratic performance with respect to businesses. He also has a regional interest in Southeast Asia, especially the Philippines. His work was published or forthcoming in outlets such as The Journal of Politics, Comparative Political Studies, The Journal of Public Economics, and the Journal of East Asian Studies. You can find additional information of his website. Twitter: @deandulay
Edmund Malesky is a Professor of Political Economy in the Political Science Department at Duke University and a noted specialist in economic development, authoritarian institutions, and comparative political economy in Vietnam. In 2012, he received a state medal from the Government of Vietnam for his role in promoting economic development for USAID’s Vietnam Provincial Competitiveness Index and, in 2013, he was appointed by President Obama to serve on the board of the Vietnam Education Foundation. He has published extensively in leading political science and economic journals and has received several academic awards including the Harvard Academy Fellowship (2004-2005; 2007-2008) and the Rockefeller Bellagio Residency Fellowship (2014). For more information visit his website.
