Detailed assessments of how well the Global Fund has performed against 13 key performance indicators will be presented to the Fund’s Board at its 34th meeting on 16-17 November. The Board will also review the 2016 performance targets for 10 of the indicators, and one additional sub-indicator that measures the efficiency of the Fund’s investment decisions.
But Andrew Kennedy, the Global Fund’s Manager of Strategic Controlling, told GFO that the first KPI results under the current system were only presented last November, so there has been relatively little time to rectify any problems that have been identified. “KPIs are most useful where you have data over a number of years,” he said. He also noted that the November 2014 and March 2015 reports didn't identify many major performance problems.
The Global Fund began using KPIs in 2004, when 14 indicators were developed, so that “a clearly-defined and rigorous process” could be put in place to objectively appraise performance at the Fund.
Results are reported quarterly to the Fund’s Management Executive Committee, and twice a year to the Board and Board committees.
Kennedy said the MEC currently uses the KPIs to drive the Secretariat to achieve the five objectives set out in the 2012-2016 Strategy. “The Board and committees use them as a means of maintaining oversight of the Secretariat and the wider global fund partnership, and ensuring that action is taken to resolve any problems that are identified,” he said.
However, since the Fund started using KPIs ten years ago, they have been modified several times, sometimes by adjusting how they are measured, sometimes to better align with changes in the Fund’s corporate priorities, and other times by completely abandoning earlier metrics and instituting new ones.
According to Fund documents, the original set of KPIs were revised in 2006, and again in 2008, such that by 2009 they had become a set of 24 indicators.
Whereas the 2006 set tracked the development of the Fund’s strategy and business model, the scale-up of interventions, grant performance, the impact of investments, and progress towards putting in place efficient internal systems, the 2009 framework examined operational performance, grant performance, system effects, and impact.
A year after this set was put in place, the KPIs were substantially changed once more when the Policy and Strategy Committee asked for an independent review to assess “the continued appropriateness of the framework for monitoring grant and Secretariat performance.”
That time, it was decided the KPIs should align with the 2012-2016 Global Fund Strategy, and should monitor that plan’s goals, targets, and objectives.
It was acknowledged that some of the old KPIs could be retained, but also that new ones were “required to measure progress in prioritizing and targeting resources to countries, most affected populations, and most cost effective interventions,” Kennedy said.
Five years on, the creation of a revised set of KPIs is still a work in progress.
In early 2013, the Board Coordinating Group asked the chairs of the three Board committees and the chair of the Technical Evaluation Reference Group to work with the Secretariat to put together another revised KPI framework.
A working group drove this process, consulting with senior management and seeking input from partners, including the the World Health Organization, UNAIDS, PEPFAR, the (U.S.) President’s Malaria Initiative, the (U.S.) Centers for Disease Control, and the World Bank.
The Audit and Ethics Committee, the Strategy, Investment and Impact Committee, and the implementer and donor blocs of the Board also had input into the final set of 16 KPIs, which were approved by the Board last November.
Kennedy pointed out that KPIs are intended to monitor implementation of an organization’s strategies and priorities, and therefore must evolve as the organization develops. “The Fund is still a relatively young organization and we’re only now reaching maturity where we might see some of these measures continue over a number of years,” he said.
A number of the current KPIs were carried over from previous frameworks, but the precise indicators may “shift in definition over time to reflect changes in the business model,” Kennedy said.
Changing the definition of indicators or altering them altogether can reduce their effectiveness, making it harder for an organization to gauge progress over time, but Kennedy said this is not always the case.
At the 25th Board meeting in November 2011, reporting on the mid-year progress of the 2011 KPIs, the Policy and Strategic Committee noted “the deteriorating performance of four grant-related indicators, and the negative impact of this deterioration on the achievement of the mission of the Global Fund.”
Although those KPIs were soon supplanted by the current set, Kennedy said the indicators in question were retained in one way or another, and the issues identified were rectified to a large extent. They were addressed through some of the initiatives put in place to transform the Fund, including efforts to fix “stuck” grants, an overhaul of the Fund’s financial systems, and the new funding model.
Among the KPIs that have remained fairly consistent over various iterations, Kennedy said the ones that have measured service delivery and grant financial data have been particularly useful.
According to Kennedy, some indicators in the current set have already delivered enough information to identify and rectify problems. He said the data collected have prompted the Global Fund to change the way it does risk assessments on human rights, and have contributed to priority setting and budget allocations for the 2016 Secretariat work plan.
The current set of KPIs is expected to remain in place until the end of the next year, when a new strategy for the coming years will be implemented. Kennedy said it is likely that many of the indicators will be carried over into subsequent frameworks because there is expected to be “considerable continuity” between the 2012-2016 strategy and the one that will follow.
Kennedy said that he expected that some indicators will be dropped, however, because they are not useful enough, and that others will be delegated to the Secretariat for monitoring because the new strategy will have altered priorities.