Audits strengthen electoral accountability in South Africa

Does information about government performance and corruption shape voter behaviour? Canonical theories of retrospective accountability suggest that it should, as voters reward their politicians for good performance and punish them for bad performance. Audits of financial management in government are an increasingly prominent source of such information in countries around the world, particularly when conducted by credible, independent official bodies. Many governments appear to fear auditors. For example, in Sierra Leone, the President recently suspended the widely-respected Auditor General shortly before her planned publication of annual audit findings.

But many recent studies, often based on randomized interventions by researchers and NGOs delivering audit or similar information to individual voters through flyers or text messages, have found mixed or null effects. These contrast with most “natural-experimental” studies of real-world audits that have found substantial effects on voter behaviour. One possible explanation is publication bias – maybe audits don’t matter in most cases, but only significant results with observational data get published. With field experiments based on pre-analysis plans, null results might be less likely to “disappear”.

But there is an alternative explanation – these two sets of studies capture rather different processes. Researcher or NGO-led information campaigns that reuse audit findings are very different from the large-scale and sustained audit processes that many constitutions require. The impact of information depends on who produces information, what it covers, and how it reaches voters.

First of all, the constitutional status of most supreme audit institutions – auditors general and audit courts – is impossible to replicate by researchers or NGOs. But credibility is not the only difference. We also argue that scale is important for the electoral salience of information. This includes the scale of production – the coverage of information about government performance over space, time, and substance – and the scale of transmission – the extent to which this information is disseminated in ways that are both public as well as socially and politically embedded. Macro-level political information production and transmission can have impacts that may not be replicable in smaller-scale interventions.

However, these macro-level processes are difficult to study in ways that enable researchers to differentiate the effects of information itself from other characteristics of the same context. In practice, this has led studies to focus on particular settings featuring natural or quasi-experimental variation, such as the now well-known randomised municipal audits in Brazil; but much less – though thankfully growing – scholarly or practical knowledge outside these settings.

Our research extends this work to a new setting, and one particularly well-suited to studying audit information at scale. In South Africa, the constitutionally-independent Auditor General audits every single municipal government every year and publishes the results. From the perspective of rigorous evaluation, this leaves no obvious “control group” of municipalities with no information produced. Our solution is to instead focus on variation in the transmission of information to voters across time. Every year, the Auditor General of South Africa releases a consolidated report of municipal audit outcomes on a particular day, which is routinely followed by intense spikes of media attention. Politicians and political parties also use this information in their campaign activities. This exemplifies information transmission “at scale” – reaching individuals through familiar channels via which they are used to encountering political information.

In order to compare electoral settings with and without this information, we focus on by-elections held to replace local councillors who have died, resigned, or otherwise left office. Since resignations and many other forms of leaving office could be politically motivated, we focus only on by-elections triggered by the deaths of councillors in office. We compare voting behaviour in these by-elections falling in the three months following each year’s audit report being released – when this information is recent and salient – with voting behaviour in by-elections falling outside of this window – when this information has fallen out of the news and is more easily forgotten or disregarded by voters. We further disaggregate these comparisons based on whether a municipal audit demonstrates an improvement in performance relative to the previous year, a decline, or a stagnation.

We find that voters do indeed punish stagnation or deteriorations, and reward improvements. The vote share of the responsible mayor’s party rises or falls by roughly five percentage points, comparing by-elections held inside of the post-audit release date window to those outside of it. We also find compatible results in a separate analysis using individual-level survey responses from a nationwide survey that was in the field both before and after the release date of the 2016-17 municipal audit report. Assessing audit declines and improvements separately, we find stronger effects of improvements on how citizens evaluate the performance of their mayor.

These findings are important for several reasons. First of all, we document effects consistent with functioning retrospective accountability, in a setting where both the production and transmission of information are fully at scale. This finding stands in contrast with many recent more pessimistic studies, typically involving information disseminated by researchers or NGOs in settings where either its production, transmission, or both, were more limited in scale. Second, while prior work has examined the accountability effects of audits with directly-elected politicians, such as mayors in Brazil or Mexico, we extend this literature to a party-centered system. Third, many studies focus on voters punishing governments for poor performance, whereas we find they also reward improvements. Demonstrating potential electoral benefits from good governance can help make the case for reforms in this area.

More broadly, while constitutions around the world require auditors general or audit courts to scrutinize and report on how governments use public resources, we still have limited knowledge about the effectiveness of holistic accountability systems, as opposed to individual initiatives. This gap is surprising given that $88 million was spent on building capacity of audit institutions in 2020 alone, and three-quarters of a billion in total over the preceeding decade. We hope our findings will prompt further work that evaluates official accountability processes and the public attention they generate.