Meeting the iChallenge: Measuring public management systems

By Claudia Buentjen on Mon, 09 June 2014

ADB, World Bank and several others have launched the iChallenge in an effort to crowd source ideas for indicators that can measure government capacity and performance.

With the iChallenge, our goals are to raise awareness of the need for performance measures which complement and expand those already being regularly collected across countries, to start a public conversation on what these measurements could look like, and to identify several indicators which could potentially be piloted and/or scaled up.
 
But why is it so important to measure public management systems and what is their connection with successful development? The easy answer is that a country that has a well-functioning public management system can deliver more successful development outcomes with the same amount of taxpayer money than a country with a less well performing system.
 
Included within the umbrella of public management systems are public financial management, procurement, public administration and civil service, tax administration and accountability mechanisms. While there is “no one size fits all,” ADB is providing support to many countries who want to reform their public sector management systems and good quality indicators are needed to monitor and support the reforms.

To assess public management systems, ADB uses a range of indicators.
•    For broader country assessments we usually monitor the World Bank Worldwide Governance Indicators. These combine measures from 31 different data sources. Their main benefit is that they cover 215 economies over the period 1996-2012.  
•    To assess public financial management systems, we use the public expenditure and financial accountability (PEFA) performance measurement framework that exists for many countries. PEFA assessments are often prepared on a joint-donor basis, with a view to aligning development partner support to government-owned reforms.
•    For national procurement systems, a country procurement assessment report (CPAR) is the main instrument for assessing procurement policies, organizations, and procedures. ADB and other development partners regularly participate in the CPAR exercise. The Organization for Economic Co-operation and Development—Development Assistance Committee (OECD-DAC) Methodology for Assessing Procurement Systems (MAPS) provides another common tool for countries and development partners to assess the quality and effectiveness of procurement systems.
•    Corruption risks and the effectiveness of national anticorruption arrangements are assessed in many countries by development partners, national and international nongovernment organizations, think tanks, and other agencies. Reports cover a wide range of issues, including political and legislative systems, legal and judicial systems, law enforcement, civil service structures, state relations with civil society, the media, private sector, anticorruption agencies, and public accountability bodies such as the national audit authorities. Transparency International’s National Integrity Systems assessment provides a comprehensive report of the country’s efforts to curb corruption. Transparency International’s Corruption Perception Index and Global Corruption Barometer also help to provide an overall picture of corruption in the country.
      
In order to get a better handle on the myriad of indicators available to measure the core functions of government, ADB has joined the World Bank and other partners in an effort called Indicators of the Strength of Public Management Systems (ISPMS). The initiative is looking for indicators that are:
1)    action-worthy (indicator measures an aspect of PSM performance that makes a difference for development outcomes and/or is intrinsically valuable in itself);
2)    actionable (indicator is specific enough to point to policy actions that can be taken to make improvements (e.g. not a composite indicator);
3)    behavioural (indicator captures the functioning or performance of institutions, rather than specific legal, organizational or institutional forms);
4)    replicable (indicator is generated transparently and if the measurement was undertaken by someone else, they would reach similar conclusions); and
5)    feasible (indicator could be collected on a regular basis for at least 20 countries and had been collected at least twice, but not necessarily regularly).

Out of the more than 1,000 indicators that are available, around 100 meet these criteria.  Of the indicator sets listed above, many PEFA indicators meet these criteria, as well as some MAPS and Transparency International metrics.

The Worldwide Governance Indicators “fail” the actionability criterion, given that they are composites of multiple individual indicators and it can be difficult to identify what exactly needs to be done in order to improve scores.
 
By far the most difficult criterion to meet is “behavioral” —a vast amount of existing indicators measure whether certain laws or institutional structures are in place, not whether they have led to any impact or behavior change.

While a legal basis is undoubtedly important, we have seen too many times where laws, procedures and guidelines aren’t followed and nothing on the ground actually changes.

While 100 indicators sounds like a lot, the public management sector is broad and diverse. There is room for more good quality indicators in some areas, such as public administration and civil service. It is with this in mind, as part of ISPMS, that the iChallenge was launched.
 
With the iChallenge, our goals are to raise awareness of the need for performance measures which complement and expand those already being regularly collected across countries, to start a public conversation on what these measurements could look like, and to identify several indicators that could potentially be piloted and/or scaled up.

Do you have any ideas for indicators that can be used to measure the strength of a country's public financial management, tax administration, procurement, public information, or public administration and civil service systems? We want to hear from you! Join the conversation at www.worldbank.org/ichallenge or in the comments below.