What future for the World Bank’s Doing Business project?

The World Bank’s Doing Business (DB) project captures the nature of basic laws and regulations that allow domestic small and medium-sized businesses to operate, invest, contract, trade, and compete on the basis of the rule of law outside the informal sector. It is arguably the most effective existing tool to stimulate country-based reform efforts that promise to improve these institutional underpinnings of productivity and wealth creation.

On August 27, the World Bank announced that it will pause publication of its Doing Business report and conduct an internal audit of data integrity. This was in response to findings of “irregularities” in the treatment of the basic data in the reports published in 2017 and 2019. Allegations of willful tampering with data for four countries have been reported in the press. Where does this lead? Will problems be fixed, and the project continued? Or will it be allowed to wither away?

In 2003, when the report came out for the first time, I was the World Bank Group Vice President overseeing the project, which I had helped initiate (I left the World Bank in 2009). As soon as the report came out, some countries questioned its approach. The earliest public debate after publication was held in France. At a panel discussion, a panoply of methodological critiques was raised. The debate was intense.

At the very end of the contentious event, a distinguished gentleman stood up. He declared that he was a principal in one of the French law firms that provided data for the report. He told the audience that the very data DB relied on were mistaken, that the staff collecting the data did not know what they were doing. At that moment, my heart sank fearing the untimely death of the project. Method can be debated. Data integrity cannot. It is required even to have meaningful debates on methodology.

Staff rechecked. Basic data had been recorded based on written responses to detailed questionnaires submitted pro bono. Partners in the law firm had signed off on the data. The data were resubmitted to the law firm to learn where there had been mistakes. It turned out that there were no mistakes in this case. In retrospect it seemed, possibly, as if the lawyer had felt under some pressure to cast doubt on the project in public.

Data discipline helped in this case. Its basic components: access to the laws and regulations that DB converts into numerical measures; detailed questionnaires and written answers; detailed methods to code and aggregate underlying qualitative information. At the same time, it is clear that mistakes are possible. If after publication mistakes were found, they were to be acknowledged and corrected. In reviews of DB, it was noted that its data disciplines compare well with those of other data systems. For almost 20 years the project has managed data well in the face of intense debate with governments.

It appears that the current data irregularities were reported to senior World Bank management by DB staff who could see that the rules in producing data were not followed. The basic disciplines have allowed problems to emerge.

Can something be done to further strengthen data governance? Today, it might be possible to expose the data management process to further scrutiny, for example, by using tools available in modern cloud-based data platforms and collaborative spaces such as GitHub or GitLab. This would allow data management to be checked and history of data adjustments to be traced–all in public.

The data are not confidential. DB measures what governments make–rules and regulations. It then plays them back to governments in a way that allows comparison with how other countries solve the same problem, for example, how to register a firm. Hence it should be possible to move to a more public platform.

No doubt, the current plight of DB gives renewed opportunity to critics asking not just for better data governance but fundamental methodological change or even discontinuing the project. That brings back the debates that have ebbed and flowed since 2003. Here are the most basic features of debates:

Is the world more complex than what Doing Business measures? Yes. Is DB an investor attractiveness index that captures all relevant features of the business environment? No.

DB focuses on regulations that matter and that are not captured in other measures. To use a rough analogy: DB measures “economic cholesterol”—what clogs the arteries of commerce. Having a bad cholesterol score is a risk factor but is not all that matters for health.

When discussing reform, DB data need to be interpreted in the context of other country-specific data and combined with judgement. Other data include DB’s companion project, the World Bank’s enterprise surveys (ES), which I initiated before DB. The surveys capture what firms experience not what the law says.

A detailed study comparing DB and ES data found that firms are more likely to follow official rules when these are less complex. The law matters, and precisely because of this not all practice follows the law. For example: In the early days of DB, registering a firm in Brazil took 150 days when following the rules of the largest commercial city, São Paolo. Brazilians rightly pointed out that with the help of “facilitators” it was possible in practice to reduce the time substantially. That goes to the point of Doing Business. Rules are needed but they should be as easy as possible to use without compromising the purpose of the rule. That way all businesses have a chance rather than just those who have special connections or means.

The fundamental question for the World Bank Group is whether it can manage sensible use of data. This is a challenge for World Bank program and project design in general. It has to be a core competence of the Bank to do so, whatever the data domain.

The Doing Business-specific question is whether there is anything intrinsic in the data as currently presented that makes this harder than usual. This leads to a renewed debate about rankings. The aggregate rankings help promote reform effort. They also provide incentives to “look good” in the rankings and concomitant political pressure.

The upshot? Publication of Doing Business has sensibly been paused while data issues are checked. At the same, the World Bank should make it clear that the project is fundamentally sound and will continue, while the future of aggregate rankings needs to be debated again. Country-specific judgment with useful data beats judgment without data.