5 Apr 2016
KPI framework termed “ineffective”

The Secretariat has not yet established an effective mechanism to routinely monitor and assess The Global Fund’s performance against strategy. The current key performance indicator framework is ineffective and does not enable full understanding of the challenges posed by the Fund’s 2012-2016 strategy. Nor does it make timely corrective action possible.

These are some of the key findings of an audit on The Global Fund’s strategic planning processes conducted by the Office of the Inspector General. A report on the audit was released on 8 March.

In the audit, the OIG sought to answer three questions:

  1. Does The Global Fund have effective processes to plan and develop its 2017-2022 Strategy?
  2. Are the Global Fund processes to implement the 2012-2016 strategy effective?
  3. Are the Global Fund processes to monitor the progress and results of its 2012-2016 Strategy effective?

The balance of this article deals with the monitoring of the 2012-2016 Strategy. See separate articles on the planning processes for the 2017-2022 Strategy (here), and on the implementation of the 2012-2016 Strategy (here).

When the 2012-2016 strategy was implemented, the OIG said, The Global Fund did not have the right tools to gauge whether the organization was achieving its aims. Even when a set of KPIs was adopted halfway through the process, it said, monitoring of key initiatives, corporate priorities, and operational performance remained “weak and inconsistent.”

The OIG found that reports prepared by the Board, the Strategy Investment and Impact Committee,  and the Secretariat since the start of the current Strategy focused primarily on the progress of disbursements and the roll-out of the new funding model. There was little in the reports on results, such as what impact the Fund was having in the countries it supports, or what was being achieved through activities related to human rights, health systems, and community systems strengthening.

The OIG said that the Board and the Financial and Operational Performance Committee performed  significant oversight of The Fund’s operating expenses, which represents about $300 million annually, about 9% of total expenditures. But monitoring the other 91% – i.e. the funds tied to grant implementation – was “limited, inconsistent and not systematic [enough] to effectively evaluate the Fund’s performance against Strategy.”

The audit found that in the first two-and-a-half years of the 2012-2016 strategy, none of the documents from the nine Board meetings held during that time show any indication that the performance of The Global Fund was being monitored. Of the 189  Board documents reviewed by the OIG, only one measured the Fund’s performance against its strategic objectives, and just eight included country and epidemiological data.

The KPIs were intended to be the main tool for monitoring the Strategy, but the OIG said the KPIs were poorly designed. The OIG noted that the Board did not get around to approving the KPI framework until November 2014, halfway through the 2012-2016 strategy. In addition, the OIG said, the Fund acknowledges that there are shortcomings in some of the KPIs, including significant gaps in reporting achievement against targets, and significant limitations concerning available data. In this respect, the OIG specifically mentioned the KPIs on health systems strengthening and human rights.

The OIG said that it recognizes the complexities around measuring the impact of the The Global Fund’s activities, and the time needed to be able to measure impact effectively.

However, the OIG said that the KPI framework is just one tool, and that it needs to be supported by adequate and effective processes to regularly evaluate the Fund’s performance at the Board, committee, and Secretariat level. “The organization needs to devote sufficient time to strategy monitoring,” the OIG said. “An equal, if not larger, weight should be applied to understanding the key drivers to progress and discussions around root causes and course correction for the challenges experienced.”

The OIG audit found that although there have been successes in the implementation of The Global Fund’s 2012-2016 strategy, “the absence of effective monitoring processes … has led to a number of key areas supporting the achievement of strategy trailing behind without a timely and clear approach to course correction.” As a result, the OIG said, it was not until after June 2015 (three-and-a-half years into the strategy) that the Fund developed actions plans to address these shortcomings.

In its report, the OIG said that the Secretariat has made improvements with respect to monitoring the Fund’s performance. In response to the audit findings, the Secretariat said that the Fund will develop and implement procedures for monitoring its Strategy on a regular basis. This will include a clear delineation between strategic monitoring performed at the Board and SIIC levels, and operational monitoring performed at the Secretariat level. All monitoring and oversight of organizational performance will be formalized and followed up with relevant actions for the Board, SIIC and Secretariat.

The Global Fund is currently defining a new set of KPIs for the upcoming 2017-2022 Strategy.

Editor’s note: (1) This is not the first time that doubts have been expressed about the value of The Global Fund’s KPIs. In an article in November 2015, GFO reported on a study commissioned by the Technical Evaluation Reference Group which questioned the value of the KPIs for monitoring Secretariat performance. The authors pointed out that the value of KPIs lies in understanding changes in an indicator, which requires considerable understanding of the context. “When corporate KPIs are intended for use by the Board as much as by management, as in the Global Fund,” they said, “problems arise when Board members lack the same level of contextual understanding.” The article also described concerns raised by others. (2) On 29 March 2016, the OIG released a report on a separate audit that it conducted on the Fund’s KPIs. The findings of this audit are similar to the KPI-related findings in the audit that is the subject of this article. 

Leave a comment