Audit snapshot

Why did we do this audit?

  • Approximately 97 per cent of aid delivered on behalf of the Australian Government through facility contractors was delivered by the top four contractors.
  • Effective and efficient delivery of aid by managing contractors is critical to achieving Australia’s aid policy and broader national interest objectives.
  • Transparency about the purpose, results and costs of the aid program helps to maintain public confidence.

Key facts

  • DFAT’s largest facility by value is the Papua New Guinea–Australia Governance Partnership.
  • For the facilities in this audit, 14 to 21 per cent of expenditure is on administration.

What did we find?

  • DFAT’s achievement of value for money in the delivery of aid through facilities is largely effective, with a need for greater focus on the collection, monitoring and analysis of aid administration costs.
    • Facility design and contractor procurement processes are well structured.
    • Partnering with managing contractors is effective, but DFAT does not appropriately monitor the ratio of administration costs to aid delivered in facilities.
    • Suitable frameworks have been established for evaluating the performance of aid investments.

What did we recommend?

  • The Auditor-General made three recommendations to DFAT. They relate to: planning processes; value for money assessments; and visibility of costs.
  • DFAT agreed with the recommendations.

19%

of total aid delivered by top four suppliers

$750m

the value of DFAT’s largest facility

14–21%

of expenditure is on administration

353

Aid Quality Checks completed in 2018

Summary and recommendations

Background

1. Australia’s Official Development Assistance (ODA) budget in 2019–20 is $4.044 billion. In 2018–19, expenditure on aid by the Department of Foreign Affairs and Trade (DFAT) made up 90.8 per cent of Australia’s total ODA spend (valued at $3.976 billion).1 DFAT’s ODA expenditure represented 65 per cent of the department’s total expenditure.

2. There has been an increase in the value of aid delivered through DFAT’s major suppliers. The value of aid delivered through DFAT’s top four suppliers over the period 2008–09 to 2018–19 moved from $236.4 million to $751.1 million, representing an increase in value from 7.8 per cent to 19 per cent of DFAT’s total aid spend.2

3. DFAT uses contracting arrangements, termed ‘facilities’, to deliver programs of work aimed at achieving broad developmental outcomes.3 A key capability expected of a managing contractor is the ability to develop and implement aid activities.

4. As at January 2020, DFAT managed 20 commercial facilities with a total approved value of $2.791 billion.4 Seventeen of these facilities were delivered by the top four providers and valued at $2.716 billion, representing approximately 97 per cent of the total approved value of facility contracts.

5. The Kemitraan Indonesia Australia untuk Infrastruktur (KIAT) facility in Indonesia and the Papua New Guinea–Australia Governance Facility (PAGP) are two of the largest facilities managed by DFAT.

  • The KIAT infrastructure facility commenced in 2017, with an expected expenditure of approximately $300 million over 10 years. KIAT aims to assist the Government of Indonesia to improve social and economic infrastructure.
  • The PAGP was established in April 2016. The facility is valued at approximately $750 million over six years. It delivers the majority of Australia’s bilateral governance initiatives in Papua New Guinea.

Rationale for undertaking the audit

6. Facility contractors are delivering $2.8 billion in aid on behalf of Australia over the life of 20 major investments. Effective implementation of Australia’s aid program supports the Australian Government’s long-term policy objectives. Transparency about the purpose, results and costs of these investments, and value for money achieved, helps to maintain confidence that they represent a proper use of public resources.

Audit objective and criteria

7. The audit objective was to examine DFAT’s achievement of value for money objectives in the delivery of Official Development Assistance (aid) through facility arrangements. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

  • Does DFAT’s design of frameworks for the delivery of aid through facilities support value for money objectives?
  • Does DFAT’s implementation of facility arrangements support efficiency and effectiveness in the delivery of aid?
  • Does DFAT appropriately evaluate and report on value for money achieved through facility arrangements?

Conclusion

8. DFAT’s achievement of value for money in the delivery of aid through facilities is largely effective, with a need for greater focus on the collection, monitoring and analysis of aid administration costs.

9. DFAT’s processes for the design and procurement of facilities are largely effective. Improvements have been made to arrangements for reviewing facility investment designs since the establishment of the KIAT and PAGP facilities in the period 2014–2016, reflecting increased contestability and risk awareness. Design processes do not, however, include appropriate consideration of facility administration costs as part of the formal assessment of value for money. Procurement processes for KIAT and PAGP were conducted in accordance with Commonwealth Procurement Rules and were effective in establishing fit-for-purpose contractual arrangements.

10. DFAT’s implementation of the KIAT and PAGP facility arrangements are partially effective in supporting value for money in the delivery of aid. Arrangements for collaborative partnering and high-level decision-making have been established, but supply chain risks are not being appropriately monitored in all instances. DFAT does not effectively analyse facility financial data at an aggregate level to determine whether administration costs are proportionate to the value of aid delivered and is therefore unable to determine whether the KIAT and PGF are realising overall expected efficiencies.

11. DFAT has effective frameworks for evaluating and reporting whether aid investments are achieving their intended purpose. There are suitable frameworks and processes in place for assessing progress in the implementation of investments and contractor performance, but there is scope to improve the transparency of evaluation and reporting on the performance of facility arrangements.

Supporting findings

Design

12. DFAT’s design of aid delivery through facilities largely supports the achievement of value for money objectives. Design processes followed the department’s investment planning requirements, but did not include appropriate consideration of cost baselines and savings projections, or information about the proportion of aid funding to be allocated to administration costs. Since 2017, improvements to the oversight of investment proposals have contributed to increased contestability and awareness of risk.

13. DFAT’s procurement of managing contractors for the KIAT and PAGP facilities was consistent with policy requirements and supported the achievement of value for money. Approaches to market maximised competition, tender evaluation processes appropriately considered provider capacity and price, and fit-for-purpose contractual arrangements were established.

Implementation

14. Governance arrangements for the KIAT facility and PAGP are largely effective in supporting the achievement of value for money. DFAT has established appropriate structures for decision-making and coordination with the respective managing contractors. There is inconsistent evidence that value for money assessments are being conducted for individual aid activities developed under the PAGP, and inconsistent compliance with requirements for the monitoring of risk in facility supply chains.

15. DFAT’s implementation of processes for monitoring and analysing facility costs are partly effective. DFAT does not routinely monitor administration costs for the purposes of analysing costs across facilities, and is therefore unable to determine whether the KIAT and PAGP are realising expected efficiencies. Audit analysis indicates that the costs associated with establishing and operating each of the two facilities are tracking higher than budgeted as a proportion of expenditure on aid activity.

Evaluation

16. DFAT has suitable frameworks for evaluating the performance of aid investments such as facilities. These are effective in supporting internal and public reporting on DFAT’s performance in achieving the results expected by government. There are opportunities for improvement in relation to the collection of data against a standard set of quantitative efficiency indicators.

17. DFAT’s reporting on the achievement of value for money in the delivery of aid through facility arrangements is largely transparent in the detail of its reporting on individual aid investments. Performance reporting on contractors occurs only at the aggregate level. Transparency would be enhanced through increased reporting on the Government’s spend on the non-aid component of facility contracts.

Recommendations

Recommendation no.1

Paragraph 2.30

That DFAT planning processes for the design of facilities include consideration of facility administration costs as part of the formal assessment of value for money.

Department of Foreign Affairs and Trade response: Agreed.

Recommendation no.2

Paragraph 3.30

That DFAT ensures that the assessment of individual aid activities under a facility are subject to appropriate value for money assessment and approvals processes.

Department of Foreign Affairs and Trade response: Agreed.

Recommendation no.3

Paragraph 3.72

That DFAT establishes processes to enable visibility, monitoring and analysis of facility administration and supply chain costs.

Department of Foreign Affairs and Trade response: Agreed.

Summary of the Department of Foreign Affairs and Trade response

18. DFAT’s summary response is provided below. The department’s full response can be found at Appendix 1.

The Department of Foreign Affairs and Trade (DFAT) welcomes the report, which followed an internal DFAT review of the operation of facilities conducted in 2017–18. We welcome the findings that Facility design and contractor procurement processes are well structured, partnering with managing contractors is effective and suitable frameworks have been established for evaluating the performance of aid investments.

DFAT is committed to continuous improvement as part of our strategy to deliver an effective and efficient development assistance program. We accept the audit report observations and recommendations regarding a broader consideration of costs during planning, improved value for money approval processes within facility structures and managing risks associated with supply chain management.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Policy and program design

Procurement

Performance and impact measurement

1. Background

Introduction

1.1 Australia’s Official Development Assistance (ODA) budget in 2019–20 is $4.044 billion.5 In 2018–19 expenditure on aid by the Department of Foreign Affairs and Trade (DFAT) was valued at $3.976 billion and made up 90.8 per cent of Australia’s total ODA spend. DFAT’s ODA expenditure represented 65 per cent of the department’s total expenditure.

1.2 The objective of the Australian aid program is to ‘promote Australia’s national interests through contributing to sustainable economic growth and poverty reduction.’6 DFAT’s annual Performance of Australian Aid report assesses the department’s management of the aid program against a performance framework, which includes 10 strategic targets.7 The eighth target requires a high standard of value for money to be achieved in at least 85 per cent of aid investments.8

1.3 DFAT and the Australian Agency for International Development (AusAid) merged on 1 November 2013.9 Responsibility for managing the aid program within DFAT now lies with policy, administration and contracting areas across the department. DFAT’s overseas missions (referred to as ‘posts’) have primary responsibility for implementing aid program activities.10

Definitions of value for money

1.4 DFAT is a non-corporate Commonwealth entity under the Public Governance, Performance and Accountability Act 2013 (PGPA Act). Under the Act, the accountable authority of a Commonwealth entity must promote the proper use and management of public resources. Proper means efficient, effective, economical and ethical use of resources.11 The Commonwealth Procurement Rules (CPRs) set out requirements for achieving value for money in the purchasing of goods and services by Commonwealth officials.12

1.5 In 2014, DFAT introduced eight principles to make explicit how value for money should be understood in the context of the aid program (see Table 1.1).13 These are referenced in DFAT guidance for managing the aid program, including for: strategic planning; procurement; program administration; and monitoring, evaluation and reporting.

Table 1.1: DFAT’s value for money principles

Economy

Efficiency

Effectiveness

Ethics

Cost consciousness

Evidence based decision-making

Performance and risk management

Accountability and transparency

Encouraging competition

Proportionality

Results focus

 

 

 

Experimentation and innovation

 

       

Source: DFAT value for money principles.14

Delivery of aid through contractors

1.6 Over the decade to 2019, the Australian Government has sought to consolidate the number of individual investments under the aid program.15 Consistent with Australia’s commitment to implementing the 2008 OECD Accra Agenda for Action, consolidation is aimed at improving value for money by reducing fragmentation in funding and improving policy coherence.16 There are now fewer, larger programs of activity of higher value — between 2013 and 2016 the number of aid investments managed by DFAT fell by 23 per cent, and the average value of investments increased by 26 per cent.

1.7 There has been an increase in DFAT’s use of major suppliers for the delivery of aid over the past decade, and in the value of aid delivered through these suppliers. The value of aid delivered through DFAT’s top four suppliers over the period 2008–09 to 2018–19 moved from $236.4 million to $751.5 million, representing an increase in value from 7.8 per cent to 19 per cent of DFAT’s total aid spend.17 In 2018–19, the top 10 suppliers delivered 95.5 per cent of DFAT’s contracted work by total approved value.

Facilities

1.8 DFAT uses facilities, managed by an external contractor, to deliver programs of work aimed at achieving broad developmental outcomes.18 A key capability expected of a managing contractor is the ability to develop and implement aid activities.

1.9 The general structure of a facility comprises a services platform and program implementing arms (see Figure 1.1). The platform supports the operation of the facility through corporate services19, with program lead areas managing the implementation of aid activities. Program areas are responsible for the development of aid proposals and engagement with DFAT and partner government stakeholders on these. This may involve establishing and managing a network of technical advisers embedded in partner government entities.

Figure 1.1: General facility structure

This figure illustrates the organisational units generally involved in the management of a facility. It shows the relationships between the Australian Government, the partner government and the managing contractor.

Note: There are some differences in the structure of the PNG–Australia Governance Partnership. See Appendix 6.

Source: ANAO analysis of data provided by DFAT.

1.10 A facility allows aid activities to be developed on an ongoing basis during implementation, with funding allocated annually. Facility operations can be scaled in response to policy and funding decisions made by the Australian or partner government. Annual work plans are used to determine a program of activities and associated budgets. DFAT describes the main benefits of the facility model as: greater policy coherence and coordination in implementation; and increased operating efficiency arising from economies of scale.

1.11 As at January 2020, DFAT managed 20 commercial facilities with a total approved value of $2.791 billion. Seventeen of these facilities were delivered by the top four providers and valued at $2.716 billion, representing approximately 97 per cent of the total approved value of facility contracts.

1.12 This audit has examined DFAT’s achievement of value for money through the use of two development facilities, the Kemitraan Indonesia Australia untuk Infrastruktur (KIAT) facility in Indonesia and the Papua New Guinea–Australia Governance Partnership (PAGP).

KIAT facility

1.13 The KIAT infrastructure facility commenced in 2017.20 It was expected that annual aid expenditure would be around $30 million on average (the total possible value under the agreement is $300 million over 10 years).21 The KIAT facility also manages four grants programs funded separately to the KIAT aid program budget.22 These grants are valued at up to $76 million over the life of the facility. Actual total spend on the facility as at 30 June 2019 was $39.7 million (approximately $20 million per year), plus an additional $18.7 million on the four grant programs. A timeline of the development and implementation of the facility is provided at Appendix 2.

1.14 KIAT aims to assist the Government of Indonesia to support sustainable and inclusive economic growth through improved access to infrastructure. It has three intended outcomes:

  1. Improved policies and regulations for infrastructure development.
  2. High-quality projects prepared for financing by the Government of Indonesia, multilateral development banks or the private sector.
  3. High-quality infrastructure delivery, management and maintenance by the Government of Indonesia.

    Detail about KIAT objectives and work streams is provided at Appendix 3.

    1.15 As of 31 July 2019, the KIAT facility had 48 contracted staff. Of these, 43 positions are located in the facility and five are attached to Government of Indonesia agencies. Development assistance activities include: the provision of technical assistance; the design of activities; feasibility studies; project management; engineering designs; capacity building; and training.

    1.16 The managing contractor for KIAT is Cardno Emerging Markets (CEMA, Australia), which is a wholly owned subsidiary of Cardno Limited.23 As at January 2020, Cardno’s 33 contracts with DFAT had a total approved value of approximately $1.3 billion. Almost one third of its programs are delivered in Indonesia.24

    Papua New Guinea–Australia Governance Partnership

    1.17 The PAGP was established in April 2016.25 A timeline of the development and implementation of the facility is at Appendix 4.

    1.18 The facility delivers the majority of Australia’s bilateral governance initiatives in PNG. Key areas of cooperation are delivered through six streams of work:

    1. Economic Governance and Inclusive Growth;
    2. Public Sector Leadership and Reform;
    3. Decentralisation and Citizen Participation;
    4. development assistance to the Autonomous Region of Bougainville;
    5. the Kokoda Initiative; and
    6. the PNG Partnership Fund — health and education grants.

      Detail about the activities of the PAGP is provided at Appendix 3.

      1.19 The implementation of PAGP activities is supported by a corporate services unit, which provides: contracting; human resource and financial management; legal; and other services. Just over a quarter of PAGP staff (91 of 332 contracted positions) are located in this unit.

      1.20 The facility commenced in 2016 with funding of up to $390 million for an initial four years, with an option to extend the contract for an additional four years to 2024. Funding was increased to $480 million in 2017 to enable the delivery of health and education grants.26 Following a two-year contract extension to 2022, the upper limit of spending over the life of the facility is $750 million.27

      1.21 At 30 June 2019, the PAGP had expended $342 million since its establishment. Expenditure under the PAGP during 2018–19 was $125.5 million, which represented approximately 22 per cent of Australia’s total bilateral aid expenditure to PNG for that year.28

      1.22 The managing contractor for the PAGP is Abt Associates Australia (Abt), a company specialising in health, social and environmental policy and international development. As at January 2020, Abt held 52 contracts with DFAT with a total approved value of around $1.6 billion. Seventy per cent of programs delivered by Abt are in PNG.29

      1.23 Timelines showing key stages in the development and implementation of the KIAT and PAGP facilities are at Appendices 2 and 4.

      Rationale for undertaking the audit

      1.24 Facility contractors are delivering $2.8 billion in aid on behalf of Australia over the life of 20 major investments. Effective implementation of Australia’s aid program supports the Australian Government’s long-term policy objectives. Transparency about the purpose, results and costs of these investments, and value for money achieved, helps to maintain confidence that they represent a proper use of public resources.

      Audit objective and criteria

      1.25 The audit objective was to examine DFAT’s achievement of value for money objectives in the delivery of Official Development Assistance (aid) through facility arrangements. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria.

      • Does DFAT’s design of frameworks for the delivery of aid through facilities support value for money objectives?
      • Does DFAT’s implementation of facility arrangements support efficiency and effectiveness in the delivery of aid?
      • Does DFAT appropriately evaluate and report on value for money achieved through facility arrangements?

      Audit methodology

      1.26 This audit applied the ANAO’s methodology for auditing value for money, consistent with the PGPA Act. This involves relating principles of effectiveness, efficiency and economy in an overall assessment of value for money, which has regard to the Government’s purpose. The principle of effectiveness relates to the extent to which the intended objectives of public expenditure have been achieved. Efficiency has been assessed with reference to whether DFAT optimises its use of resources in delivering the intended quantity, quality and timing of outputs. Economy is understood as the principle of minimising costs, with regard to requirements of timeliness and the quantity and quality of inputs.30

      1.27 The audit team undertook the following activities:

      • selected two facilities as case studies by identifying the highest value facility in PNG and Indonesia respectively, taking into consideration sector (governance and infrastructure) and suppliers (first and second ranked in terms of total approved value of contracted work)31;
      • examined documentation and data held by the department relating to the facilities and the broader aid program and reviewed public submissions to the ANAO; and
      • conducted field visits to DFAT posts in Jakarta and Port Moresby to interview departmental and managing contractor staff and to review documentation and data held at post. The audit team also interviewed departmental staff and other stakeholders in Canberra.

      1.28 The audit was conducted in accordance with the ANAO auditing standards at a cost to the ANAO of $612,126.

      1.29 The team members for this audit were Judy Lachele, Nathan Callaway, Yvonne Buresch, Dr Cristiana Linthwaite-Gibbins and Paul Bryant.

      2. Design of facilities

      Areas examined

      This chapter examines whether the department’s design and procurement of facilities supported value for money objectives, using the KIAT facility and the Papua New Guinea–Australia Governance Partnership (PAGP) as case studies.

      Conclusion

      DFAT’s processes for the design and procurement of facilities are largely effective. Improvements have been made to arrangements for reviewing facility investment designs since the establishment of the KIAT and PAGP facilities in the period 2014–2016, reflecting increased contestability and risk awareness. Design processes do not, however, include appropriate consideration of facility administration costs as part of the formal assessment of value for money. Procurement processes for KIAT and PAGP were conducted in accordance with Commonwealth Procurement Rules and were effective in establishing fit-for-purpose contractual arrangements.

      Area for improvement

      The ANAO made one recommendation that the design of facilities include consideration of facility operating costs as part of the formal assessment of value for money.

      2.1 An effective design process clarifies the purpose and expected results of an investment and determines how value for money can be best achieved and measured over its lifespan. In order to examine this criterion, the audit reviewed DFAT’s frameworks for establishing facility investments, with reference to the design and procurement of the KIAT and the PAGP facilities. This encompassed the review of:

      • processes for the design of aid delivered through facilities — including DFAT’s consideration of the Government’s priorities; use of cost and risk analyses; and the operation of oversight mechanisms; and
      • processes and practices for procuring facility managing contractors — as the selection of an effective delivery partner has a major influence on the achievement of value for money over the life of an investment.

      Does DFAT’s design of aid delivery through facilities support the achievement of value for money objectives?

      DFAT’s design of aid delivery through facilities largely supports the achievement of value for money objectives. Design processes followed the department’s investment planning requirements, but did not include appropriate consideration of cost baselines and savings projections, or information about the proportion of aid funding to be allocated to administration costs. Since 2017, improvements to the oversight of investment proposals have contributed to increased contestability and awareness of risk.

      Planning of aid delivery

      2.2 DFAT’s Aid Programming Guide governs the process for the development of new aid investments. Investment designs are required to set out the policy rationale and intended outcomes of a proposal, as well as broad options for implementation. These form the basis of consultation with partner governments, approvals for the commitment of funds, and subsequent procurement.

      2.3 For investments over $10 million, the design process involves the development of an initial, high-level proposal (Investment Concept Note). Following approval by the delegate, a detailed design (Investment Design Document) is developed.32 This informs the development of a Request for Tender (RFT).

      2.4 Criteria for the design of proposals are set out in DFAT guidance. Key ‘quality criteria’ are:

      • relevance to government policy priorities;
      • effectiveness — how the strategy will address priorities and deliver measurable and achievable outcomes; and
      • efficiency — whether the proposal will make appropriate use of resources and time. Efficiency is defined in terms of costed inputs and outputs, with alternative models of delivery evaluated.33

      2.5 The alignment of a proposal with government priorities is informed by four-year Aid Investment Plans (AIPs), which are agreed with partner governments and set out high-level objectives for country and regional programs.34 The plans aim to link aid objectives, aid programming and intended results.

      2.6 The design of high-value or high risk (or a combination of both) proposals involves external and peer review processes.35 Proposals are provided to an oversight body — the Aid Governance Board (AGB) — for endorsement at the Investment Concept Design stage.

      Design of the KIAT and PAGP facilities

      2.7 Activities for the design of the KIAT were undertaken from April 2014 to February 2017. The PAGP was designed between March 2012 and July 2015. Appendices 2 and 4 provide timelines for these design processes. Both proposals were reviewed and endorsed by the Aid Investment Committee (AIC), the principal DFAT governance body responsible for overseeing funding proposals at the time. The AIC was replaced by the AGB in December 2017 (discussed at paragraphs 2.18 to 2.34 below).

      KIAT

      2.8 The KIAT facility succeeded an established facility, the Indonesia Infrastructure Initiative (IndII)36, largely maintaining its focus on infrastructure development and the facility mode of delivery. The Investment Concept Note finalised in November 2015 outlined Australia’s bilateral policy objectives, which aligned with the AIP for Indonesia published two months earlier. The AIP signalled a shift in emphasis from the direct funding of infrastructure to influencing policy and regulation, recognising Indonesia’s increased capacity to fund capital works.

      2.9 Agreed bilateral priorities were reflected in the KIAT design through the following changes from the existing IndII facility:

      • a shift in focus from grants-based aid to targeted technical assistance;
      • a reduction in funding from a planned $130 million to $30 million per annum, with an increased emphasis on leveraging Indonesian public and private resources37; and
      • linking aid investments more closely with Australia’s trade objectives.

      2.10 DFAT’s decision to use the facility model for KIAT was informed by evaluations undertaken of the previous infrastructure program and feedback on the existing facility from the Indonesian Government.38 A facility was considered to provide greater transactional efficiency and program coherence compared to a number of stand-alone investments. DFAT’s design and associated approval process did not, however, include an explicit cost and benefit comparison of alternate delivery mechanisms.39

      2.11 Some process efficiencies identified in an ANAO audit and evaluations of the IndII facility40 were addressed in the final design. This included streamlining processes for the approval of aid activities developed under the facility to reduce administration and costs to DFAT and the managing contractor. KIAT’s activity development process is discussed further at paragraphs 3.24 to 3.27.

      PAGP

      2.12 In October 2014, the AIC endorsed the PAGP Investment Concept Note. The PAGP design process established the policy rationale and business case for the use of the facility mode of delivery. The proposal was aimed at delivering greater development impact than had been achieved through a number of extant governance programs due to end in December 2015.

      2.13 The PAGP design reflected the Government’s policy of reducing direct service delivery through the aid program and supporting economic development. It aligned with the intent of a ministerial agreement concluded in December 2013 to expand Australia’s support for governance in PNG through activities directed at:

      • expanding support for good governance (delivered primarily via the Strongim Gavman Program)41;
      • strengthening anti-corruption and security efforts;
      • professionalising the public service;
      • improving accountability and leadership;
      • undertaking critical economic reforms; and
      • meeting the basic needs of the population.42

      2.14 A facility model and two alternative models were considered:

      • partial consolidation — integration of individual projects into a smaller number (8–10); or
      • consolidation into three mini-facilities, each with 8–10 smaller initiatives, designed and contracted separately.

      2.15 The facility model was assessed as the most appropriate implementation mechanism for ensuring aid activities were strategically aligned, coordinated and responsive to changing priorities. It was anticipated that a flexible and iterative design function would enable lessons learnt to be applied throughout implementation. The facility model was determined to offer the greatest scope for achieving administrative efficiency. An anticipated reduction in staffing at post (estimated at 20 per cent), and hence capacity to manage multiple stand-alone investments, was a key consideration.43

      2.16 DFAT estimated that the facility model would lead to overall savings of approximately 25 per cent compared with existing arrangements. This included one-off savings (for example, through the consolidation of office costs) and on-going savings (for example, as a result of reduced design, tendering and contracting costs). Estimates of potential savings were not prepared for the other two options.

      2.17 Cost savings identified for the facility model were not assessed with reference to costs associated with operating a facility. Without an understanding of these costs, the net saving likely to be achieved cannot be determined. This reduced the capacity of decision-makers to assess the proposal’s overall proposition for achieving value for money.44

      Oversight arrangements

      2.18 An internal 2017 review, (the ‘Aid Operations Health Check’) concluded that the AIC’s oversight of aid program risks was not effective. The main reasons for this were:

      • active avoidance of AIC scrutiny by proponents of proposed investments by undervaluing investments or understating risk;
      • large and unstable committee membership and infrequent meetings;
      • a lack of portfolio view; and
      • oversight not extending to the implementation of investments.

      2.19 The AGB replaced the AIC in December 2017. The AGB is chaired by a Deputy Secretary and members include the Chief Economist (Development), the Chief Financial Officer and Senior Executives of policy and contracting divisions. Unlike the AIC, the AGB has an independent member. Members are expected to apply a whole of department perspective in exercising their responsibilities rather than acting as representatives of their Group.

      2.20 The AGB’s primary purpose is to act as an advisory body to the Secretary; the departmental executive; and delegates under the PGPA Act. Its stated aims are to increase strategic clarity and strengthen governance, including risk management of the aid program.

      2.21 In addition to broader functions associated with providing advice on the aid program, the AGB advises delegates on designs for investments that have a value of $100 million or more, are rated ‘high risk’, or are a facility. The AGB can also elect to consider investments developed under time pressure or those that have highly ambitious or innovative objectives. It may request to review investments during implementation.

      2.22 A technical unit, the Quality and Risk Advisory Unit (QRAU) overseen by the First Assistant Secretary of Contracting and Aid Management Division, appraises all proposals submitted to the AGB. Its primary role is to advise the AGB and delegates on the quality of concepts and designs. QRAU appraisals aim to assess the extent to which proposals:

      • have clear objectives and align with strategic priorities and are likely to bring substantial benefits and impacts;
      • are informed by evidence of the country, regional, and sector context, as well as relevant DFAT and international expertise;
      • have clear and effective management and governance arrangements, and feasible implementation plans;
      • have arrangements to manage risk; and
      • have clear and effective arrangements for monitoring and evaluating progress.

      2.23 This audit reviewed documentation relating to new facility proposals considered by the AGB since its inception (refer to Table 2.1) to assess whether intended improvements in the higher-level oversight of facilities are evident.45

      Table 2.1: Facility proposals reviewed by the Aid Governance Board from April 2017 to May 2019

      Facility proposal

      Date reviewed by AGB

      Indonesia Governance for Growth (KOMPAK — contract extension)

      11 April 2017

      Pacific Labour Mobility Facility

      22 June 2018

      Australia–PNG Health Program Managing Contractor (PNG-AUS Transition to Health (PATH) Program)

      23 August 2018

      PNG–Australia Program for Improved Performance in Education

      4 October 2018

      Southeast Asia Economic Governance and Infrastructure Facility (Phase 1)

      6 December 2018

      PNG–Australia Governance Partnership (next steps — contract extension)

      20 April 2019

      Australia Infrastructure Financing Facility for the Pacific

      31 May 2019

         

      Source: ANAO analysis of data provided by DFAT.

      Objectives

      2.24 The design process generally established a clear line of sight between proposed activities, delivery mechanisms and the intended outcomes of the investment. Clarification of objectives and intended impact was requested by either the AGB or QRAU for four of the seven proposals, and further information on how proposed activities would result in the expected development impact was requested for two proposals. DFAT advised in March 2020 that actions relating to these investments have been completed.

      Costs

      2.25 DFAT guidance states that delegates must understand the costs and impacts of their spending decisions, as well as the risks involved. The funding for an individual proposal is set through the department’s annual budget processes. In order to assess value for money, the delegate requires information about how the proposed scope of an activity and its delivery mechanism represents the best use of resources.

      2.26 The investment documents for each proposal reviewed did not clearly link the expected quantity and quality of aid delivery to the funding available. Value for money was systematically assessed for one of the seven proposals examined; the PNG–Australia Governance Partnership (next steps — contract extension). The need to achieve value for money was referenced in a further three proposals.

      2.27 Only limited information about anticipated indirect delivery costs as a component of the overall budget was presented in proposals.46 Delivery options and associated costs are not examined as a standard component of QRAU assessments. Such costs are relevant to understanding the amount of program funding likely to be available for direct aid activities.

      2.28 Proposals generally did not assess DFAT staffing requirements. The QRAU has noted a continuing tendency across the department to select the facility modality as the most efficient mechanism for delivering programs given staffing constraints, without sufficient consideration of potential consequences for effectiveness.

      2.29 The Commonwealth Procurement Rules (CPRs) state that when a business requirement arises, it is important to take into consideration the market’s capacity to competitively respond to a procurement.47 This perspective was evident in two of the seven proposals.

      Recommendation no.1

      2.30 That DFAT planning processes for the design of facilities include consideration of facility administration costs as part of a formal assessment of value for money.

      Department of Foreign Affairs and Trade response: Agreed, noting that at design stage facility administration cost will only be an estimate, with costs finalised through the tender process and contract negotiations.

      Risks

      2.31 The involvement of independent members48 and the use of technical assessments have contributed to increased visibility and testing of risks associated with individual proposals. More broadly, the AGB is overseeing work to strengthen management of aid program risks.

      2.32 Reporting to the AGB in June 2019 stated that DFAT has ‘no system for capturing and reporting on aid program risks in a consolidated form’. Risk information contained in AidWorks, the department’s system for managing aid investments, is incomplete. The system cannot be used therefore to identify high-risk investments.49

      2.33 Proposed work to address weaknesses in DFAT’s management of risk identified by the 2017 Aid Operations Health Check included:

      • risk planning and oversight of the top 15 aid investments by value;
      • development of risk appetite statements for the aid program;
      • aid risk updates to be provided every six months; and
      • a broader assurance framework and systems capability to address non-compliance with DFAT policies and procedures.

      2.34 In late 2019, DFAT identified 15 aid investments for oversight by the AGB on the basis of key risk criteria. These criteria included: either being a facility and/or of high value; delivered in high-risk countries; having a high safeguard risk; and having been assessed as an investment requiring improvement.50 The first risk report was to be provided to the AGB in March 2020.

      Do DFAT’s processes and practices for procuring facility managing contractors support the achievement of value for money in the development of aid programs?

      DFAT’s procurement of managing contractors for the KIAT and PAGP facilities was consistent with policy requirements and supported the achievement of value for money. Approaches to market maximised competition, tender evaluation processes appropriately considered provider capacity and price, and fit-for-purpose contractual arrangements were established.

      2.35 The CPRs require procurement activities to deliver a value for money outcome. This is achieved when procurements:

      • encourage competition and are non-discriminatory;
      • use public resources in an efficient, effective, economical and ethical manner;
      • facilitate accountable and transparent decision-making;
      • encourage appropriate engagement with risk; and
      • are commensurate with the scale and scope of the business requirement.51

      Procurement approach

      KIAT

      2.36 The procurement process for the KIAT facility commenced with an open tender in February 2016.52 The tender sought the provision of both design and implementation services for the facility.53 This approach differed from DFAT’s more commonly used procurement strategy of tendering for design and implementation services separately and sequentially. Engagement for the implementation phase was not guaranteed to the provider of design services — it was to be at DFAT’s discretion whether to exercise the implementation option following the design phase.

      2.37 The procurement approach was aimed at minimising the time between the completion of the design and the start of implementation, as DFAT would not be required to undertake a separate procurement for implementation services.54 DFAT also saw benefit in the contractor responsible for consulting with key stakeholders in the design of the facility carrying these relationships forward into the implementation phase. However, the scheduled timing of the KIAT procurement would not have allowed DFAT to return to market for the implementation phase without disrupting activities that were to be continued from the IndII facility.55 This reduced the scope for benefits of an option to not proceed to implementation.

      2.38 DFAT ensured that the market received appropriate information about its objectives for the procurement.56 Of the 11 companies that attended the industry briefing, four submitted bids.

      PAGP

      2.39 The open tender process to engage a managing contractor for the PAGP commenced in August 2015 with the release of the PAGP Design Document and RFT to the market.

      2.40 The PAGP design detailed DFAT’s expectations that the managing contractor would be responsible for the development and implementation of most of Australia’s governance related program activities in Papua New Guinea.57 The draft deed of standing offer outlined a requirement to establish four broad programs of work. The transition of a large number of lapsing agreements and their alignment with a yet to be developed governance strategy was to be determined during a six month inception period.

      2.41 DFAT recognised a key risk was that the procurement would not secure the most able supplier for delivering the substantial scope of work, particularly as DFAT would be locked into an arrangement for several years. To mitigate the risk, DFAT engaged with industry at an early stage, providing comprehensive information.58 Fourteen organisations attended DFAT’s industry briefing resulting in four tender submissions. All were consortia ranging from two through to eight organisations, led by four of DFAT’s top five suppliers in 2015–16.

      Tender evaluation

      KIAT

      2.42 An Evaluation Committee (EC) was established in March 2016 to assess the tender submissions for the KIAT facility. The EC’s Evaluation Plan set out four key criteria for evaluating value for money: technical; operations; financial and commercial; and risk.

      • The technical assessment was based on three areas: the bidding organisation59 (weighting 20 per cent); the bidder’s proposed approach to delivering the tender requirements (40 per cent); and proposed personnel (40 per cent). Each proposal was given a technical score out of 10. Cardno was ranked first on this assessment.
      • The financial and commercial assessment was conducted on each bidder’s proposed costs over the life of the facility. Cardno was initially ranked last on this criterion. However, the EC identified that Cardno’s higher cost was due to its proposed high number of technical advisers. As these advisers would be paid for directly by DFAT, the Committee determined that removing these costs would allow for Cardno’s pricing to be better compared to the other tenderers. Taking this into account, the bid was determined to be more competitive compared to the other bids.60
      • The EC’s evaluation of value for money did not initially consider financial viability assessments as these were not yet available.61 However, when these reports were completed, the EC assessed this information and presented it to the delegate as part of its recommendation for decision. While Cardno was assessed as satisfactory in terms of its financial viability, the EC recommended that DFAT obtain additional security to mitigate risk in the form of a performance guarantee from the parent company, Cardno Limited.62 During contract negotiations, DFAT agreed to waive the requirement of a performance guarantee.63 The delegate was advised that the inclusion of clauses in the contract (relating to Definitions, Early Notification and Termination for Breach) would address concerns raised in the financial viability assessment. However, advice did not provide detail on how the financial viability risk would be mitigated by this approach.
      • Each of the proposals were risk assessed by the EC on technical risk and commercial and financial risk. Cardno was rated as low risk for both risk types.

      2.43 The EC provided an overall value for money ranking for each of the bidders, recommending Cardno as the preferred supplier. In June 2016, the delegate approved the tender evaluation outcome.64

      2.44 While the delegate was apprised of adjustments to Cardno’s price and of its assessed financial viability in the evaluation outcome minute, there would have been merit in providing the delegate with a revised overall value for money assessment as:

      • Cardno’s financial proposal would have reflected a more comparable price with other tenderers; and
      • the financial viability assessment forms part of the systematic consideration of risk.

      While the overall ranking based on the value for money assessment may not have changed in light of these factors, it would have provided a more complete basis for, and record of the value for money evaluation.

      PAGP

      2.45 The tender period for the procurement of the PAGP closed on 6 October 2015. Four submissions were received. An Evaluation Board (EB) was established to provide a recommendation on which proposal represented the best overall value for money.

      2.46 The assessment of value for money under the EB’s Evaluation Plan was undertaken in four areas: technical; operations; commercial; and risk. These assessments were conducted by separate committees reporting to the Board.65

      • The technical evaluation was conducted by the Technical Evaluation Committee. Proposals were assessed out of 100 on their technical ability. The committee assessed Abt as the second ranked tenderer on its technical proposal, although the difference between the first and second tenderers was relatively small (4.5 points).
      • The operations evaluation was conducted by the Operation Evaluation Committee to examine the bidder’s approach to supporting the delivery of the facility services and the proposed organisational structures for managing the PAGP. The committee assessed Abt as the strongest tenderer and a clear first.
      • The commercial evaluation was conducted to assess if the tenderer’s proposal was cost effective, competitive and provided for the full delivery of the proposed technical and operational activities. Abt was assessed as the best commercial tender.
      • The risk evaluation was conducted with reference to the assessed financial viability of the tenderer and the compliance of the proposal with the terms of the RFT. Abt’s proposal was considered to be the lowest risk proposal.

      2.47 Abt was ranked first across each of the evaluation areas, except for technical ability where it came a close second. When the operations score was taken into account, Abt remained the preferred bidder.

      2.48 The EB also tested value for money against five aid program expenditure scenarios (ranging from $200 million to $400 million) using a value for money index.66 Abt was assessed as representing the best value for money for three of the scenarios.67 The Evaluation Board recommended Abt as the preferred tenderer, as its proposal was assessed as representing best value for money overall.

      2.49 This recommendation was approved by the delegate68 on 19 December 2015.

      Contracting — negotiation and finalisation

      KIAT

      2.50 DFAT conducted formal negotiations with Cardno between July and August 2016, entering into a contract in September 2016. Negotiations clarified adviser support costs for the design and implementation phases. Agreement was reached to reduce the number and cost of technical advisers, achieving an 18.5 per cent reduction in adviser-related costs.69 Cardno completed the Final Design Document for the KIAT Facility in February 2017.

      2.51 The commissioning of Cardno to also deliver implementation services was contingent on DFAT enacting an amendment to the original contract. The structure of the design–implement contract thereby provided a mechanism for re-testing value for money before proceeding to the implementation stage. DFAT negotiated with Cardno to improve the terms of the contract. This resulted in more favourable performance and management fee arrangements.70 However, DFAT did not conduct an overall assessment of value for money linking deliverables and price, despite the relatively high value of the implementation phase ($140 million)71 and a revised statement of requirements.72

      PAGP

      2.52 For PAGP, the second preferred tender was close to the first. While the tenderer achieved a higher ranking on technical ability, it had also submitted a higher price. In light of this, DFAT could have considered adopting a ‘best and final offer’ process, involving a formal request that tenderers indicate if they are able to improve their proposals.

      2.53 DFAT commenced negotiations with Abt in December 2015. As the draft deed set out only a high-level program of work and did not specify the activities to be transferred to the PAGP, there was a need to more clearly define the scope of work to be undertaken under the contract.73 DFAT also negotiated on three issues related to performance: pricing and the proportion of payments linked to performance; the flexibility of the facility platform in scaling up as PAGP activities increased; and establishing a process for adding service orders to the facility. DFAT and Abt entered into a deed of standing offer on 4 April 2016.

      General negotiation planning

      2.54 Formal plans were not developed to support either the KIAT or PAGP contract negotiation processes.74 DFAT has since developed an optional template to support negotiation planning.75 DFAT’s standard practice is to conduct negotiations only with the first ranked tenderer. Negotiations with the next preferred tenderer are commenced if agreement cannot be reached with the preferred tenderer. For both KIAT and PAGP, DFAT retained the option of opening negotiations with the next preferred tenderer.

      Contracting — payment structure

      2.55 A contract should provide incentives for the contractor to deliver value for money throughout the life of the investment. The contract for a development facility sets out the fee structure for operating the facility and general arrangements for implementing aid activities. All personnel, operational and program costs are paid on a reimbursable basis.

      KIAT

      2.56 Fees paid to Cardno comprise:

      • fixed monthly payments (60 per cent of total) — not dependent on performance;
      • milestone payments (20 per cent) — paid upon the satisfactory completion of milestones; and
      • performance payments (20 per cent) — paid on the results of annual DFAT performance assessments. The payment reduces for each performance indicator against which Cardno is rated as less than ‘adequate’.

      2.57 Milestone payments are for outputs such as: facility plans; policies and procedures; and reporting. These items are needed to support the functioning of the facility itself. Some milestones relate more directly to the implementation of aid activities (for example, the design of a performance-based grants system; the development of specific aid proposals; or policy briefs).

      2.58 The KIAT contract includes a cost control regime. This is intended to reinforce cost consciousness by requiring contractors to: use competitive tender processes when engaging sub-contractors; report expenditure against contract limits; and implement resource-sharing arrangements.

      PAGP

      2.59 The PAGP was established as a deed of standing offer arrangement with no initial value. The deed sets out the overall scope of the PAGP and allows DFAT to enter into service orders with the managing contractor, as required, to deliver activities against the objectives of the facility. Service orders function as standalone contracts, which specify funding amounts for the activities. As at November 2019, there were six service orders in place under the PAGP.76 These ranged in value from $21 million to $261 million.

      2.60 The deed of standing offer arrangement reduces the risk of a single point of failure in the event of contractor underperformance by allowing individual service orders to be terminated and re-tendered.77 The use of service orders therefore supports flexibility in programming and the alignment of resources with specific outputs.

      2.61 Unlike other DFAT facilities, the PAGP contract has both a management fee and a procurement fee.78 The management fee is paid in relation to services provided by the facility platform (Service Order 1). The procurement fee is based on the total expenditure (operational and program costs) of service orders for the delivery of aid program activities. A procurement fee was used because of budget uncertainty in Australia and PNG at the time of tendering. The fee is scaled in line with changing levels of funding.

      2.62 The management fee is capped at $16.5 million over the life of the facility.79 The procurement fee is set at 5.3 per cent of total reimbursable expenditure.

      3. Implementation

      Areas examined

      This chapter examines whether the department’s arrangements for the implementation of facilities support value for money in the delivery of aid, using the KIAT and the Papua New Guinea–Australia Governance Partnership (PAGP) as case studies.

      Conclusion

      DFAT’s implementation of the KIAT and PAGP facility arrangements are partially effective in supporting value for money in the delivery of aid. Arrangements for collaborative partnering and high-level decision-making have been established, but supply chain risks are not being appropriately monitored in all instances. DFAT does not effectively analyse facility financial data at an aggregate level to determine whether administration costs are proportionate to the value of aid delivered and is therefore unable to determine whether the KIAT and PGF are realising overall expected efficiencies.

      Areas for improvement

      The ANAO made two recommendations aimed at strengthening value for money assessments and increasing the visibility, monitoring and analysis of administration costs.

      3.1 For this criterion, the audit examined whether DFAT has demonstrated value for money in the implementation of KIAT and PAGP. This involved an assessment of:

      • facility governance arrangements — the establishment of facility functions; governance structures; coordination processes with managing contractors; development of aid program activities; and risk management and assurance processes to ensure facilities have the appropriate foundations for achieving value for money; and
      • cost oversight — structures for the ongoing monitoring and analysis of facility administration costs in order to ensure that expected efficiencies associated with the facility model are being realised.

      Do governance arrangements for the delivery of aid through facilities, including for the management of risk, support the achievement of value for money?

      Governance arrangements for the KIAT facility and PAGP are largely effective in supporting the achievement of value for money. DFAT has established appropriate structures for decision-making and coordination with the respective managing contractors. There is inconsistent evidence that value for money assessments are being conducted for individual aid activities developed under the PAGP, and inconsistent compliance with requirements for the monitoring of risk in facility supply chains.

      Establishment of facility functions

      3.2 Delays in the start-up of both the KIAT and PAGP meant that approximately one year was required to put in place a strategic framework; organisational arrangements; and a pipeline of work aligned with the objectives of the design for each facility respectively. The KIAT and PAGP organisational structures are shown at Appendices 5 and 6.

      3.3 The audit examined DFAT’s establishment of core governance functions and processes for the respective facilities.

      KIAT

      3.4 The final design of the KIAT facility provided the basis for its mobilisation.80 The facility was slower than expected in establishing new streams of work and did not fully expend its aid program funding in the first 18 months.81 This was due to the effort required to transition six substantial activities to the KIAT facility. DFAT guidance issued in July 2019 states that mobilisation plans should specify sufficient human resource allocations for set up and staff recruitment.

      PAGP

      3.5 DFAT engaged a consultant in December 2015 to assist with the transition of existing contracts to the facility. As the contract with Abt was not finalised until April 201682, the scope to undertake transition activities was limited. While hand-over briefs were prepared for a number of transitioning programs, there was no overarching implementation or risk plan.

      3.6 DFAT assessments of the inception of the PAGP indicate that during this period the facility’s effectiveness was impacted by:

      • the large number of complex contracts that were required to be novated83;
      • unrealistic assumptions about the time, effort and funding requirements for the development of new programs of activity during the first 18 months;
      • delays in recruiting qualified international and local facility staff; and
      • a lack of fit-for-purpose operational systems to manage budgets and to monitor and report on activities.

      Facility governance structures

      3.7 Good governance helps a facility to maintain a focus on the purpose of an investment and establish organisational delivery structures aligned with value for money principles. Aid policy objectives are set by the Australian Government in consultation with partner governments, with facility arrangements providing a flexible mechanism to deliver on these through aid programs such as technical assistance and grants. A facility is expected to quickly design and tender for new programs in response to changes in country priorities, while delivering on longer-term development impact goals.

      3.8 For both KIAT and PAGP, strategic decision-making authority is vested in a senior management committee co-chaired by the Australian and respective partner government. Each committee’s role is to direct and oversee the activities of the facility to ensure progress toward the achievement of the facility’s strategic objectives. Six-monthly meetings are held to review progress in the implementation of the work program and financial expenditure. These processes are also intended to enable DFAT to respond to changing priorities.

      3.9 Each facility’s funding is allocated on an annual basis. A proportion of funding is uncommitted, and the management committee may direct these resources to new priorities.

      KIAT

      3.10 A number of technical committees have been established to support the senior management committee. These generally meet on a quarterly basis to review the implementation of specific programs of work and to approve new funding. If technical committees cannot reach consensus agreement, decisions are elevated to the senior management committee. Day-to-day management is supported by facility operations manuals and standard operating procedures.

      3.11 A range of reporting is produced by the facility to support the review of progress by joint committees. Reporting for compliance and accountability purposes is provided by the managing contractor to the Jakarta post in relation to performance against contract milestones and Key Performance Indicators.

      PAGP

      3.12 Several changes have been made to the governance structures of the PAGP since its establishment.

      3.13 In the early phase of the PAGP, bilateral arrangements for reaching agreement on facility program priorities were not considered effective. Concern centred on stakeholder perceptions that the facility had assumed a decision-making role that properly sat with the Australian and PNG governments. In October 2017, the Australian and PNG governments agreed to name the facility the Papua New Guinea—Australia Governance Partnership. Key changes to operational arrangements were made to give effect to government direction. These included:

      • a shift in the facility’s role from the design of activities to delivering activities determined by the Australian and PNG governments;
      • the devolution of decision-making from the senior management committee to individual work streams overseen by relevant DFAT and PNG officials; and
      • restructuring the corporate unit to ensure it would deliver required support for work stream activity, while appropriately assessing and reporting on facility-level outcomes.84

      3.14 The changes, which were reflected in the updated contractual arrangements for the PAGP, were approved by the Australian High Commissioner to PNG. A subsequent internal review in 2019 noted that the new arrangements had addressed stakeholder concerns and reduced some inefficiencies relating to centralised decision-making.

      3.15 The new arrangements reduced the managing contractor’s responsibilities for supporting centralised strategy setting. However, management fees under the deed were not adjusted to reflect this.

      3.16 Funding for the operation of the PAGP and its activities is derived from six separate allocations. These funding allocations are managed by three senior executives. Recognising a risk that these funding allocations and associated individual work streams could become siloed, DFAT has assigned a Senior Responsible Officer (SRO) to the PAGP.85 The SRO is responsible for ensuring the facility meets overall outcomes and for managing strategic risks.

      The DFAT and managing contractor relationship

      3.17 Effective collaboration between DFAT and managing contractor personnel in the implementation of aid can: promote common understanding of objectives; increase flexibility and responsiveness to changes in the operating environment; and lead to more effective management of disputes.

      3.18 DFAT guidance emphasises the importance of DFAT retaining responsibility for strategic engagement with partner governments, while recognising that facility personnel will need to interact on a day-to-day basis with partner government officials and the community to be effective in their role.

      3.19 The KIAT facility has established effective partnering and working arrangements through clear governance and business processes. These recognise the differing roles of DFAT and the contractor in the design and delivery of aid. Charters set out protocols of engagement between the three main parties of DFAT, the Government of Indonesia and the facility contractor. Processes for the development of aid activities are underpinned by this understanding.

      3.20 The design of the PAGP envisioned the facility contractor having a strong capacity to design aid program activities, with post only overseeing those initiatives considered high risk. Over the course of implementation, the respective roles of DFAT and the facility contractor in the development of aid program activities have changed. The facility contractor’s role is now largely defined as an implementing arm of post. This approach risks not making full use of the expertise and experience offered by the managing contractor, potentially diminishing value for money outcomes.

      Development of aid activities under the facility structure

      3.21 Facilities are a mechanism for developing a pipeline of aid activities over the life of an investment. A key driver of activity development is the need for each investment to meet annual and overall investment expenditure targets.

      3.22 Similar to a standing offer arrangement, DFAT makes use of aid design and implementation services, as needed. The process of activity development involves:

      • determining requirements — proposals may be generated by DFAT, the partner government or via facility personnel or contracted advisers;
      • the preparation of proposals, including an estimated budget — involving consultation between DFAT, stakeholders and the facility at working level;
      • DFAT approval based on consultation with the partner government — this may involve consideration by a management or technical committee; and
      • implementation — monitored by the facility which reports to DFAT and partner government.

      3.23 Value for money must still be considered each time services are procured.86 The audit examined KIAT and PAGP records to determine how this requirement had been applied in the development and approval of proposals.87

      KIAT

      3.24 The KIAT facility uses an aid activity development process, which has defined approval ‘gates’ (see Figure 3.1).

      Figure 3.1: KIAT aid activity design and approval process

      This figure illustrates the process for the design and approval of aid activities under the KIAT facility. The process differs based on the type of activity. There are minor activities valued at less than $500,000 and major activities valued at $500,000 or more. Both types of activities include a staged design and approval process, but there is a more in-depth process for the major activities.

      Source: ANAO analysis of data provided by DFAT.

      Minor activities

      3.25 Gate points for DFAT approvals are clearly defined. Activities with an estimated value of less than $500,000 are classified as minor activities and do not require the development of concept notes or designs (phases 3 and 4). KIAT undertakes competitive tender processes as a requirement under its contract, reflecting requirements of the Commonwealth Procurement Rules (CPRs). As at August 2019, half of all activities managed by the facility had been classified as minor, with an average value of $196,000 (comprising eight per cent of the total value of activities).

      Major activities

      3.26 There is evidence that DFAT staff actively contribute to the development of major activities in KIAT, providing varying levels of feedback. Comments and questions in documents reviewed centred on matters of effectiveness. As at August 2019, 21 major activities above $500,000 had been approved by DFAT with an average value of $2.5 million per activity.

      3.27 DFAT relies on the managing contractor to maintain formal records of how, when and who provided funding approvals. The terms of the KIAT and PAGP contracts stipulate that DFAT retains legal ownership of all records created for the provision of the goods and services by the managing contractor. Contractors are required to transfer records to DFAT upon termination or conclusion of the contract. This supports DFAT to meet Commonwealth record-keeping obligations.88 DFAT should also ensure records of proposal approvals can be readily accessed and verified by DFAT personnel throughout the implementation of a facility.

      PAGP

      3.28 The PAGP does not have an equivalent documented activity development process. Under the deed of standing offer, DFAT approves work for activities through an Annual Work Plan process. Each work stream and the platform develops a high-level annual work plan and budget that outlines areas of activity and the maximum amount available for those activities for the financial year. These are approved by senior personnel at post and confirmed at a senior management meeting held between post and the managing contractor.

      3.29 Briefs for individual activities are developed for high risk and high value proposals. DFAT advised that activity development processes reflect the specific needs of each work stream. In reviewing a sample of activity briefs, the ANAO found the extent to which these linked objectives, deliverables and costs, as part of a value for money assessment, varied considerably, and there were inconsistencies in the extent to which approvals had been recorded.89

      Recommendation no.2

      3.30 That DFAT ensures that the assessment of individual aid activities under a facility are subject to appropriate value for money assessment and approvals processes.

      Department of Foreign Affairs and Trade response: Agreed.

      Facility risk management and assurance processes

      3.31 The aid program often operates in environments where there is a heightened risk of financial misappropriation or where there may be different standards of community health and safety. While the facility contractor manages sub-contractors and non-government organisations, DFAT remains accountable for aid investments achieving objectives in compliance with Australian Government policies and standards, including for how well the contractor manages delivery partners.

      Risk management

      3.32 DFAT has developed guidance to assist delegates to meet requirements under the Public Governance, Performance and Accountability Act 2013 (PGPA Act), including to appropriately manage risk.90 Delegates are to ensure managing contractors have systems in place for managing program implementation risks.

      3.33 The KIAT facility has a risk plan, which sets out DFAT and managing contractor roles and responsibilities and processes for monitoring and managing risk. A comprehensive risk register for each work program, and the facility as a whole, has been developed. DFAT requires risk registers to be reviewed at least quarterly to respond to changes in the risk environment and to ensure controls remain effective. Jakarta post and the managing contractor comply with this requirement.

      3.34 For the PAGP, risk registers have been developed for each work stream and are reviewed on a six-monthly basis. A ‘risks, threats and opportunities’ report exists at the facility level and is reviewed monthly. While the report provides a high-level summary of risks, it does not rate levels of risk or assign responsibility for managing risks. As at February 2020, an organisation-wide risk management system was in the process of being established. It is intended that this will incorporate facility level risk registers.91

      Fraud

      3.35 Under PGPA Rule 1092 an entity must take all reasonable measures to prevent, detect and correct fraud.93 A 2017 review of DFAT’s management of fraud identified risks arising from the growth of facility sub-contracting arrangements. DFAT does not specifically monitor the frequency of fraud associated with facility arrangements, however it advised the ANAO on instances of potential fraud reported by Cardno and Abt over a two-year period for all investments (including programs other than KIAT and PAGP).94

      • Cardno — 29 detected by managing contractor and six by sub-contractor (of these, 15 cases were confirmed as fraud); and
      • Abt — 41 detected by managing contractor and seven by sub-contractor (of these, 35 cases were confirmed as fraud).

      3.36 The efficacy of DFAT’s fraud control system is subject to a separate ANAO performance audit and therefore has not been examined further in this audit.95

      Sub-contractor due diligence

      3.37 DFAT policy requires managing contractors to conduct due diligence checks of delivery partners (sub-contractors and non-government organisations) before awarding contracts. Checks are aimed at providing assurance about policies, processes and controls for mitigating key risks.96 DFAT’s Environmental and Social Safeguard Policy for the aid program sets out mandatory risk and safeguard processes that apply to working with partners. The policy requires early identification of risks during planning and throughout the life of the investment.97

      KIAT

      3.38 Cardno has standard policies and procedures for ensuring compliance with DFAT’s assurance requirements throughout its delivery chain. As at July 2019, due diligence checks were finalised for 10 of 38 active sub-contracts, representing five different sub-contracted entities, with remaining checks partially finalised. Senior management of Cardno had become aware in December 2018 of the company’s non-conformance with DFAT and Cardno policy, but did not inform DFAT until this audit requested evidence of sub-contractor checks in June 2019. A majority of checks were then completed by 9 August 2019, with all checks completed by December 2019.

      3.39 KIAT has established a web-based management information system (MIS).98 The MIS stores due diligence and other reports completed by the facility and can be accessed by both managing contractor and DFAT personnel. In 2019 Jakarta post introduced an assurance plan and reporting system for DFAT’s aid program in Indonesia.

      PAGP

      3.40 Port Moresby post does not have an overarching formal assurance plan and reporting system for DFAT’s aid program in-country. PAGP processes for ensuring sub-contractor and NGO compliance are thorough and records are well maintained. Processes include a set of legal and compliance checks each time a contract is let; proactive field education about DFAT requirements; and regular site visits.

      Has DFAT implemented processes to effectively monitor and analyse facility costs?

      DFAT’s implementation of processes for monitoring and analysing facility costs are partly effective. DFAT does not routinely monitor administration costs for the purposes of analysing costs across facilities, and is therefore unable to determine whether the KIAT and PAGP are realising expected efficiencies. Audit analysis indicates that the costs associated with establishing and operating each of the two facilities are tracking higher than budgeted as a proportion of expenditure on aid activity.

      3.41 Facility costs comprise:

      • direct expenditure on aid programs, projects and activities; and
      • costs that are administrative in nature and result from processes of aid delivery (management fees and operational costs, including contracted personnel costs).99

      3.42 Efficiency in facility operations depends on the costs involved in establishing and operating a facility being proportionate to the benefits delivered. Understanding and monitoring the ratio of administration costs (inputs) to aid expenditure (outputs) is necessary to effectively guide resource allocation, planning and financial management. A range of factors relating to the specific operating context of delivery need to be taken into account when determining an appropriate ratio. This should occur at the investment design stage as part of establishing a baseline for monitoring costs over the life of the investment.

      Monitoring facility administration costs

      3.43 Staff at the Jakarta and Port Moresby posts generate and review financial data for the purposes of managing invoicing and payments under contracts for the respective facilities and to support financial reporting.

      3.44 The KIAT contract requires the managing contractor to seek opportunities for price reductions and savings throughout the life of the facility. There is evidence that post monitors compliance with elements of the contract’s cost control regime, including by addressing sub-contracting costs. The contract does not identify a specific cost savings target to be achieved.

      3.45 DFAT expected that the PAGP would deliver significant efficiencies compared with separately tendered programs — a total of $29.4 million in operating cost savings over a five-year period.100 Post has reported some savings in corporate and staff costs (cumulative totals of $4.5 million as at 2017–18 and $9.1 million as at 2018–19), although this has not been verified.101 There have been on-going efforts to improve the operating efficiency of the PAGP, including through an organisational review in 2018 that recommended eliminating duplication in enabling services.

      3.46 In April 2019, the managing contractor of the PAGP initiated the development of a ‘value for money framework’ aimed at enabling facility operations to be assessed against DFAT value for money principles.102

      Analysing facility administration costs

      3.47 While Jakarta and Port Moresby posts are able to monitor elements of expenditure associated with the KIAT and PAGP facilities respectively, DFAT has a limited capacity to analyse costs across its portfolio of facilities.

      3.48 AidWorks is DFAT’s IT system for administering the Australian aid program. It supports program planning; budget and financial management; procurement; and implementation of agreements, including performance management.

      3.49 DFAT has identified a number of limitations with AidWorks, which hamper its ability to systematically analyse data:

      • contracts have differing data reporting requirements;
      • financial data is not collected against a chart of accounts;
      • personnel information is incomplete;
      • reports are in a format which means they are unable to be effectively analysed; and
      • there is a reliance on managing contractor systems to generate data.

      3.50 DFAT is reliant on incomplete data from AidWorks and data from managing contractors to undertake analyses of facility costs.103 This reduces its ability to analyse costs across its portfolio of facilities and to demonstrate the efficiency benefits of the facility model. A review commissioned by DFAT in 2019 observed that cost data is not collated or used to develop a strategic view of the efficiency of individual facilities, or the contracting model more broadly.

      Audit analysis of KIAT and PAGP costs

      3.51 Table 3.1 shows the ratio of administration costs to expenditure on aid activity based on cost data for the KIAT facility and the PAGP. Noting the limitations of AidWorks, the table was developed by drawing on total source data for the KIAT and PAGP facilities contained in the AidWorks system. The data was extracted for the purposes of this audit and is reported against categories agreed with DFAT. AidWorks does not automatically and routinely produce information in this format.

      Table 3.1: KIAT and PAGP expenditure, as at 30 June 2019

      Expense type

      KIAT (% of total facility expenditure)

      PAGP (% of total facility expenditure)

      Management fees (capped amount)

      $4,145,744 (7%)

      $7,909,801 (2%)

      Procurement fees (percentage based)

      Not applicableb

      $12,003,630 (4%)

      Operational costs (reimbursed)

      $1,726,271 (3%)

      $25,945,770 (7%)

      Contracted personnel costs (25% of total)a (reimbursed)

      $2,439,029 (4%)

      $27,039,941 (7%)

      Total administration costs

      $8,311,044 (14%)

      $72,899,142 (21%)d

      Activity / program costs

      $42,801,872 (73%)c

      $188,400,653 (55%)

      Contracted personnel costs (75% of total)a (reimbursed)

      $7,317,087 (13%)

      $81,119,822 (24%)

      Total expenditure on aid activity

      $50,118,959 (86%)

      $269,520,475 (79%)

      Total facility expenditure

      $58,430,003

      $342,419,616

           

      Note a: Expenditure does not include DFAT costs such as those related to DFAT staff and overheads. DFAT costs are discussed below.

      Note b: The KIAT facility contract does not include procurement fees.

      Note c: The KIAT facility supports the delivery of four grant programs. As grant funding is transferred directly to the Government of Indonesia it is not reflected in the KIAT aid program budget. Expenditure relating to these programs over the life of the KIAT facility has been included in the activity / program costs ($18,679,401).

      Note d: Percentage totals may differ due to rounding.

      Source: ANAO analysis of data provided by DFAT.

      3.52 Table 3.1 classifies the expense types for each facility as follows:

      • management fees — covers management related expenses such as financial management costs, administrative and head office staff, insurance and costs of reporting. The management fee also includes the managing contractor’s profit margin. The management fee is limited to a maximum amount in the contract. Payment of this depends on satisfactory performance in line with the agreed performance framework;
      • procurement fees — a fee charged for the administrative costs associated with procurement activity, based on the annual value of aid programs delivered through the service orders. The amount paid will therefore vary;104
      • operational costs — costs paid on a reimbursable basis for a range of overheads such as rent for offices and personnel housing for the operating platform, IT-related costs and travel;
      • contracted personnel costs — salaries and fees for personnel that are contracted by the managing contractor to implement the facility. Personnel may be either administrative or technical staff. As DFAT does not routinely capture and analyse aid to administration costs there is no standard ratio to support the attribution of these costs. Based on information provided by Jakarta post, the analysis assumes that 75 per cent of personnel costs can be considered aid activity and 25 per cent are attributable to administration. This ratio has also been applied to the PAGP for the purposes of comparing the two facilities105; and
      • activity/program costs — expenditure on activities delivered by the facility that directly supports the achievement of the investment’s aid objectives. Sub-contractor costs (excluding contracted personnel costs) are also categorised as activity program costs.

      3.53 Table 3.1 indicates that the ratio of administration costs to expenditure on aid activity (administration to aid ratio) can vary across facilities. The administration to aid ratio can also vary over the life of the investment.

      3.54 As the management fee is a fixed dollar amount for both facilities, a lower than anticipated level of aid activity in the early years of a facility is likely to result in a proportionately higher administration to aid program cost ratio. Payments for the procurement fee are proportional to the value of aid expenditure. Increases in the expenditure for aid activities will therefore result in an overall higher level of payment for administration.

      3.55 Table 3.2 compares the expected (budgeted) administration to aid ratios for each facility compared to the actual ratios based on expenditure as at 30 June 2019. Administration costs have been calculated in accordance with the categories detailed in Table 3.1.

      Table 3.2: KIAT and PAGP administration to aid ratios — expected versus actual

      Facility

      Expected (based on contracted amounts)

      Actual (expenditure as at 30 June 2019)

      KIAT

      10:90

      14:86

      PAGP

      19:81

      21:79

           

      Source: ANAO analysis of data provided by DFAT.

      3.56 Table 3.2 reflects that, for both facilities, administrative costs are tracking higher than budgeted as a proportion of aid. This difference is larger for the KIAT facility. As the management fee agreed for KIAT is a fixed rather than proportional figure, KIAT operating costs in the first year of the facility were higher than expected due to the lower than expected program activity during the start-up phase. Costs have also not reduced in line with reduced annual allocations.106 The management fee is therefore higher as a proportion of the value of the facility than estimated at the time of contracting.

      3.57 Conversely, increases to aid program funding for the PAGP have improved its administration to aid ratio. While the procurement fee has increased in absolute value terms, management fees have remained the same, reducing the overall amount of fees paid proportionate to program activity. In addition, a higher level of expenditure on aid activities means that fixed operational costs are reduced as a proportion of expenditure. However, actual expenditure on aid administration is greater than expected because of higher aid activity costs to which the procurement fee applies.

      Analysing facility costs with reference to DFAT workforce expenditure

      3.58 The administrative costs referenced in Table 3.1 are those specifically associated with the managing contractor for each facility. An accurate analysis of efficiency gains from a facility must incorporate an understanding of DFAT’s own staffing costs associated with managing the facility, together with the nature and complexity of the development activities being delivered and the operational environment. DFAT guidance states that achieving efficiencies through facilities may require changes to staffing structures and management. A 2019 DFAT commissioned review recommended quantifying the expected DFAT resourcing burden of a proposed facility relative to other delivery models such as direct delivery by DFAT.

      3.59 DFAT receives two appropriations — administered and departmental. Administered funding refers to items the entity does not control but administers on behalf of the Government. DFAT uses administered funding for expenditure on the support, design and delivery of Australia’s aid programs and objectives.

      3.60 Departmental funding is for the day-to-day operation of the department, including salaries and the purchase of goods and services.107 However, if an officer is considered to spend more than 50 per cent of their time on the ‘direct delivery of aid’, DFAT funds their salary and costs from the administered budget. In 2018–19, $255 million in DFAT staffing costs were included in the Australian Government’s reported spend on Official Development Assistance.

      3.61 Table 3.3 analyses staff resourcing data for 2019–20 for the KIAT facility108 and the PAGP.109

      Table 3.3: DFAT staff at post with KIAT and PAGP responsibilities

       

      KIAT

      PAGP

       

      A-baseda

      LESb

      A-based

      LES

      All staff working on facility

      1.25

      4.5

      13

      18

      Number of these staff funded from the administered budget

      1.25

      4.5

      8.5

      12.75

      Estimate of DFAT staff costs for facility in 2019–20 (administered budget component only)c

      $375,000

      ($375,000)

      $135,000

      ($135,000)

      $3.9 million

      ($2.55 million)

      $540,000

      ($400,000)

      Total estimate of all DFAT staff costs (departmental and administered)

      $510,000

      $4.4 million

      Estimate of all DFAT staff costs (departmental and administered) as a percentage of total expenditure on facility contract

      1.7%d

      3.5%e

               

      Note a: A-based staff are posted Australian staff. These staff generally have both policy and program implementation responsibilities.

      Note b: LES refers to locally engaged staff. LES may have administration and/or program management responsibilities.

      Note c: The audit has used DFAT’s estimate of the average cost of an EL1 posted officer. This does not include costs associated with the officer’s housing and corporate overheads. The audit has estimated that an LES officer costs around $2500 month.

      Note d: Based on an expected KIAT expenditure on the facility contract in 2019–20 of $30.9 million (excluding staffing costs).

      Note e: Based on an expected PAGP expenditure on the facility contract in 2019–20 of $127 million (excluding staffing costs).

      Source: ANAO analysis of data provided by DFAT.

      3.62 The table indicates that if DFAT staffing costs associated with managing the KIAT facility are included in annual expected expenditure on the facility contract, then this administration cost would represent approximately 1.7 per cent of total expenditure (equating to approximately $1.1 million since the facility’s inception). For the PAGP, DFAT staff would account for approximately 3.5 per cent of total expenditure (equating to approximately $14.3 million since the facility’s inception). Adding in these costs would result in the administration to aid ratios increasing to 16:84 (from 14:86) for KIAT and 24:76 (from 21:79) for PAGP.110

      3.63 DFAT’s internal reviews have also found that the department does not: estimate or validate resourcing requirements throughout the different phases of a facility’s lifespan; track staff time spent on facility management and administration; or review the efficiency of management arrangements.111 Including DFAT staffing in the calculation of administration costs would provide a basis for comparing the efficiency of different modes of delivery, particularly within the same operating environment.

      Monitoring supply chain costs

      3.64 Administrative costs associated with sub-contracting arrangements include:

      • sub-contractor management fees, personnel costs, and operational costs; and
      • costs of undertaking procurement activity.

      The sub-contractor management fees, personnel costs and operational costs are categorised as aid activity expenditure.112

      Sub-contractor management fees

      3.65 Jakarta post requires the KIAT managing contractor to scrutinise sub-contractor costs. The contract requires Cardno to seek written approval from DFAT if the sub-contractor’s proposed management fee is higher than 20 per cent of the contract value for direct source contracts and 40 per cent for open tendered contracts.

      3.66 The PAGF contract does not place limits on management fees charged by sub-contractors. If stipulated, fees charged by sub-contractors may be included as a proportion of personnel costs or as a fixed amount stipulated in the contract. The PAGP does not have the capacity to monitor and assess overall costs incurred.

      Procurement activity costs

      3.67 The PAGP design estimated savings of 25 per cent in procurement costs compared with existing management arrangements.113 DFAT has not examined whether anticipated savings have been realised. The KIAT design did not provide an estimate of savings.

      3.68 The procurement of sub-contractors creates facility operating costs, which are paid by DFAT. This was forecast in the KIAT tender submission to be $328,916 for implementation phases one and two of the KIAT facility.114 The PAGP submission did not detail procurement related expenses.

      Value for money in sub-contracting

      3.69 When procuring sub-contractors, DFAT requires the KIAT and PAGP managing contractors to act in accordance with the CPRs, which require open and competitive procurement to maximise value for money. Both the KIAT and PAGP facility operations manuals stipulate a requirement for all procurement activity to comply with Commonwealth legislative requirements and policies, including the CPRs.

      3.70 The KIAT facility operations manual details requirements for procurements to be conducted in accordance with value for money principles. It notes that steps should be taken to avoid ‘directed procurements’, whereby it is insisted that a specific supplier is chosen or where contracts are repeatedly given to the same supplier.

      3.71 Port Moresby post advised that it may direct procurements when there is a pressing time requirement; restricted domestic competition; or a need to access specific skill sets. These circumstances may also involve a request from the PNG Government.115 Procurements which are directed by DFAT (referred to as ‘client-directed’) are recorded by the managing contractor via a template that states that DFAT, rather than the managing contractor, is responsible for assessing value for money. As at September 2019, 18 active PAGP subcontracts valued at $8.5 million (29 per cent of total value) were procured using a client-directed procurement approach.

      Recommendation no.3

      3.72 That DFAT establishes processes to enable visibility, monitoring and analysis of facility administration and supply chain costs.

      Department of Foreign Affairs and Trade response: Agreed.

      4. Evaluation

      Areas examined

      This chapter examines whether the department appropriately evaluates and reports on value for money achieved through facility arrangements, using the KIAT and Papua New Guinea–Australia Governance Partnership (PAGP) facilities as case studies.

      Conclusion (and/or findings)

      DFAT has effective frameworks for evaluating and reporting whether aid investments are achieving their intended purpose. There are suitable frameworks and processes in place for assessing progress in the implementation of investments and contractor performance, but there is scope to improve the transparency of evaluation and reporting on the performance of facility arrangements.

      Area for improvement

      The ANAO made suggestions for improvement in relation to the collection of data against a standard set of quantitative efficiency indicators, the use of benchmarking and more detailed reporting on the Government’s spend on the non-aid component of facility contracts.

      4.1 Evaluation processes provide evidence that public money is being spent effectively and efficiently to achieve the purposes for which it has been appropriated. The audit reviewed the department’s:

      • use of performance frameworks, which support the measurement and reporting of outcomes delivered in relation to facilities; and
      • publication of information about the performance of facility investments and the performance of the facility contracts.

      Does DFAT have suitable frameworks to evaluate whether the delivery of aid through facilities achieves results expected by the Government effectively and efficiently?

      DFAT has suitable frameworks for evaluating the performance of aid investments such as facilities. These are effective in supporting internal and public reporting on DFAT’s performance in achieving the results expected by government. There are opportunities for improvement in relation to the collection of data against a standard set of quantitative efficiency indicators.

      DFAT performance framework

      4.2 Performance frameworks must be established by Australian Government entities to give an account of how the Government’s objectives are being met and that the investment of public resources represents value for money. DFAT has a detailed performance framework, which is outlined at Figure 4.1. This reporting primarily aims to inform the Parliament and the public of how Australian funds are contributing to international development. Facilities are subject to the same processes of assessment and reporting as all other DFAT investments.

      Figure 4.1: DFAT performance assessment framework and processes

      This figures outlines the components of DFAT’s performance assessment framework and process. There are a range of internal and external monitoring and reporting mechanisms, which assess performance at the whole of aid program level, as well as at the individual aid programs.

      Source: ANAO analysis of data provided by DFAT.

      4.3 Figure 4.1 shows that the objectives and priorities for Australia’s aid program are captured in several public policy documents. The Government’s longer-term foreign policy priorities are published in its 2017 Foreign Policy White Paper116, with DFAT’s annual policy outcome articulated in its Corporate Plan117 and Portfolio Budget Statements.118 These also set out the high-level criteria against which DFAT’s performance will be measured. (DFAT’s performance criteria for 2019–20 are provided at Appendix 7).

      4.4 Aid program results achieved over the course of a year are published through a suite of reports. These cascade from the whole of aid program perspective provided in the annual Performance of Australian Aid report to the program-level Australian Aid Program Reports.

      4.5 Investment level reporting occurs through internal processes for reviewing the performance of investments (Aid Quality Checks) and delivery partners (Partner Performance Assessments). Further information about DFAT’s performance reporting framework is provided at Appendix 8.

      4.6 In order to support performance reporting, DFAT establishes performance assessment frameworks at a whole of country or region level (program PAFs) and at an individual investment level (investment PAFs). A PAF specifies how performance is to be measured using key evaluation questions, benchmarks, baselines, indicators and sources of data. Data collected at these levels contributes to public reporting on the performance of the aid program as a whole and at a country or regional level.

      Program performance reporting

      Program-level assessment process

      4.7 Bilateral and regional programs with $15 million total or greater in annual aid expenditure have an Aid Investment Plan (AIP), which sets out the Government’s development objectives at a country or regional level. There are AIPs in place for Australia’s aid relationship with Indonesia and Papua New Guinea. DFAT draws on data captured by program-level performance assessment frameworks (program-level PAFs) to provide an account of how all individual investments established under an AIP (including facilities) relate to each other and have contributed to achieving its objectives.

      Investment-level assessment processes

      4.8 An investment-level PAF supports reporting on the results achieved by individual investments, including facilities. As individual investments also contribute to the higher-level objectives of an AIP, an investment-level PAF may include specific targets that align with these objectives. This provides a line of sight between the individual investment and the bilateral or regional program.

      KIAT

      4.9 The KIAT monitoring and evaluation framework has 12 performance indicators in total. Table 4.1 shows the two indicators that directly align with the higher-level objectives for the Australia–Indonesia aid relationship (third column). In addition to these AIP-related indicators, the managing contractor reports against a further 10 indicators, which are specific to the facility. Targets for indicators are set on an annual basis.

      Table 4.1: Australia–Indonesia Aid Investment Plan and KIAT Performance Assessment Framework

      AIP objectives

      AIP infrastructure-related outcomes

      KIAT performance indicators aligned with AIP objectives

      Relevant AIP target

      KIAT contribution to AIP target

      1. Effective economic institutions and infrastructure

      Infrastructure is better planned, delivered and maintained (Outcome 2)

      Amount of additional financing co-invested in infrastructure planning, delivery, management and maintenance

      $1 billion in additional financing is leveraged

       

      AUD $174,597,332

      1. Human development for a productive and healthy society

      More households can access water and sanitation (Outcome 5)

      Number of districts with improved service delivery practices and policies for infrastructure planning, delivery management and maintenance.

      10,000 people with disabilities in rural areas benefit from disability inclusive-designed water and sanitation infrastructure

      A total of 14 people with disabilities have improved access to sanitation infrastructure

       

               

      Source: ANAO analysis of data provided by DFAT.

      PAGP

      4.10 The PAGP’s approach to investment performance assessment generally accords with DFAT guidance and processes, as described in paragraph 4.7. However, due to the wide scope of PAGP activity, separate PAFs have been established for each of its work streams (seven PAFs in total).

      4.11 Results captured by each of the work stream PAFs are aggregated into a ‘whole of facility’ PAF. This performance information, in turn, contributes to reporting against the strategic governance objectives of the AIP for Papua New Guinea. Examples of performance indicators and targets in the PAGP PAF for 2018–19 are:

      • number of women and men trained with Australian support and engaged by PNG in positions where they can practice new knowledge and skills (19,537 against a target of 6779);
      • number of women, men and youth using improved services provided by civil society, private sector partners or sub-national governments (841,420 against a target of 36,375); and
      • value of development investment attracted to PNG facilitated by or value-added to by Australia ($11.4 million against a target of $67 million).

      4.12 The compilation of data against the PAF allows DFAT to cite development impacts. In May 2019, the managing contractor of the PAGP also developed a separate PAF for the facility’s corporate unit. The PAF sets a 2019–20 milestone for the development of improved value for money assessment and reporting processes across the platform and work streams.

      Use of Monitoring, Evaluation and Learning Frameworks

      4.13 DFAT performance assessment and reporting at the facility level is further supported by the use of monitoring, evaluation and learning frameworks (MELF). Each DFAT aid investment valued $3 million and above requires a monitoring, evaluation and learning framework. These are designed to generate information about whether individual activities are performing to expectation; enable reporting against PAFs; and guide the continuous, systematic collection of data to support a number of functions, which help to improve value for money outcomes.

      KIAT

      4.14 KIAT has had a MELF in place since August 2017. The purpose of the framework is to deliver information about the day-to-day performance of the facility and its activities, and to support the management committee to make informed decisions about priorities and the allocation of resources. It also provides a basis for longer-term evaluations.

      4.15 Implementation of the KIAT MELF is supported by a work plan, which sets out actions for implementing the framework, including the development of standards; training for facility staff and contractors; and use of data for reporting. MELF indicators align with whole of country and facility-level objectives.119 Sources of information include regular external reviews; stakeholder feedback; case studies and activity data. Performance information generated through the MELF is used to report against the Indonesia program PAF, Aid Quality Checks (refer to paragraph 4.21), and six-monthly reports to the management committee.

      PAGP

      4.16 MELFs were first established for the PAGP for three work streams in late 2016 following the development of an interim strategy for the facility. However, poor monitoring and evaluation capabilities contributed to the PAGP being assessed as an investment requiring improvement in 2017.

      4.17 The PAGP then established MELFs based on theories of change for all of its work streams, except for one.120 However, following a number of reviews, PAGP MELFs for all work streams are being developed or revised to reflect changes in the role of the facility platform and work streams. As at February 2020, this work is ongoing.

      4.18 DFAT guidance on facilities states that a facility PAF is aimed at measuring effectiveness and efficiency outcomes across the activities of the facility. Neither the KIAT nor the PAGP performance assessment frameworks and MELFs require the collection of data against a standard set of quantitative efficiency indicators, noting that some efficiency data is provided by KIAT to DFAT in Bi-Annual Performance and Planning Reports and is reported in Aid Quality Checks. DFAT should establish a set of standard efficiency indicators for this purpose.

      4.19 In 2018, the Office of Development Effectiveness reviewed DFAT’s system of investment monitoring.121 It recommended that DFAT expectations of monitoring and evaluation be standardised across contracts to improve quality and consistency.122

      Investment performance reporting

      4.20 Separately to the process of assessing performance for the purposes of public reporting, DFAT undertakes an annual process of assessing and reporting on aspects of the operational performance of individual aid investments through the preparation of aid quality checks and partner performance assessments. Aid Quality Checks (AQCs) measure the performance of a specific investment. Partner Performance Assessments (PPAs) review how well implementing partners are delivering services.123

      Aid Quality Checks

      4.21 An AQC must be completed on an annual basis for all aid investments valued at $3 million or more. Program areas — typically with posts in the lead — prepare AQCs using evidence gathered from managing contractor reporting, monitoring visits, reviews and evaluations. AQCs provide an account of whether an investment is on track according to budget and schedule, and delivering the results expected.124 Assessment criteria include effectiveness and efficiency, and the efficacy of implementing arrangements.

      4.22 In 2018, 353 AQCs were completed for 358 aid investments125, with a total approved value of $13.4 billion.126 DFAT does not require AQCs to be completed for all investments.127 In 2017–18 investments exempt from the assessment process had a total value of $8.5 billion.

      4.23 Each criterion is assessed on a scale of one (very poor) to six (very good). A score of three and below is considered unsatisfactory. Scoring of investments against the effectiveness and efficiency criteria contributes to external reporting against DFAT’s value for money strategic target.128

      4.24 The audit reviewed AQCs for the KIAT and PAGP facilities, focussing on those completed in 2019. For KIAT, a whole-of-facility AQC is undertaken and supported by four AQCs for individual programs. For the PAGP, there is no single AQC that provides performance information at a whole-of-facility level. DFAT completes AQCs for each of the six work streams and the facility administrative unit each reporting cycle.

      Effectiveness

      4.25 AQC reporting against the effectiveness criterion is intended to answer the question — Are we achieving the outputs and outcomes that we expected? Analysis indicates that AQCs for KIAT and the PAGP included an appropriate focus on budget and schedule control, and clear information about the status of deliverables or outputs.

      4.26 The effectiveness of an investment also depends on whether facility outputs are contributing to the broader goals of the aid investment. An investment’s ‘theory of change’ links specific outputs to strategic policy outcomes, with MELFs designed to provide evidence of whether activities are bringing about the desired changes. Since 2019, assessors have been required to explicitly refer to evidence to support their assessment on the extent to which an investment is achieving its outcomes.

      4.27 The 2019 AQC for the KIAT facility assessed itself as adequate against the effectiveness criterion by referencing achievement against the facility’s three outcomes. PAGP AQC reporting is mixed in making explicit links between outputs and outcomes.129

      Efficiency

      4.28 Reporting on efficiency is intended to answer the question — Is the investment making appropriate use of Australia’s and our partners’ time and resources to achieve outcomes? Efficiency is assessed primarily with reference to: whether resources are being used appropriately; how expenditure is tracking compared to budget; success in gaining additional resources from other sources; and the efficiency of the modality and its functioning.

      4.29 Analysis of AQCs for KIAT and PAGP indicates that budget expenditure is mainly discussed in the context of avoiding shortfalls in the disbursement of funds against the financial year allocation. AQCs do not always clearly distinguish between administration and program expenditure in reporting.

      4.30 A ratings matrix sets out broad indicators to support consistency in the evaluation of value for money. However, DFAT does not use a set of quantitative targets or benchmarks to ensure performance ratings are assigned in a consistent manner over the life of an investment and across different investments. Assessments of effectiveness and efficiency are not related. There is therefore no explicit overall conclusion on the question of value for money.

      Underperforming investments

      4.31 DFAT uses AQC ratings to identify ‘Investments Requiring Improvement.’ An investment rated three or less on both the effectiveness and efficiency criteria, twice in succession, is subject to cancellation. Only the PAGP has been rated three, and no facility has been rated three twice.130

      4.32 The first AQC completed for the PAGP in May 2017 (eight months after commencement) assessed the investment as ‘underperforming’ with respect to effectiveness, efficiency, gender equality and monitoring and evaluation in relation to its core governance streams.131 Contractor performance was identified as the key contributing factor to the facility’s overall underperformance.

      4.33 The PAGP contract was subsequently placed under formal remediation. The remediation plan set out actions to improve understanding of strategic priorities; human resource development and internal systems and processes; and DFAT oversight of performance, including additional six-monthly Partner Performance Assessments (discussed at paragraphs 4.30 to 4.35). The managing contractor was assessed as having addressed these matters in May 2018.

      Partner Performance Assessments

      4.34 For all commercial agreements and grants valued at $3 million and above, the responsible DFAT manager is required to assess implementing partners (contractors, non-government organisations, multilateral organisations) on an annual basis against five criteria, which include the achievement of objectives and value for money.132 For a contractor, an unsatisfactory rating may result in DFAT withholding a KPI payment. Ratings may also be considered by DFAT in tender assessments.

      4.35 In assessing value for money, DFAT managers are required to consider how the implementing partner has:

      • eliminated inefficiency and duplication and applied lessons learnt to value for money;
      • delivered services within budget, with predicted budgets comparing well to actual expenditure;
      • pursued the most cost-effective options and proportionality in planning and allocating resources;
      • used robust systems for monitoring and managing value for money during implementation; and
      • operated in accordance with DFAT value for money principles.

      4.36 A rating of four (on a scale of one to six) indicates performance is ‘adequate’, defined as ‘on balance satisfies criteria; and does not fail in any major area’. In 2019, the average rating for commercial partners against the criteria associated with value for money was 4.9. This was an improvement over the 2018 score of 4.8. Only one of the 207 PPAs completed for commercial partners in 2019 recorded a rating of ‘less than adequate’ for achieving value for money.

      4.37 A DFAT commissioned review of the PPA process in 2018 reported that staff found that assessing value for money was challenging because of the absence of ‘a clear definition of value for money or innovation; there being different ways to interpret the criteria, and not having the right evidence (particularly relating to value for money)’. It is further noted that PPAs are not moderated or reviewed by the Office of Development Effectiveness.

      4.38 Cardno’s average PPA rating for KIAT is broadly in line with its general performance on other DFAT aid investments. Abt’s average PPA rating for the PAGP is slightly lower compared to its performance in relation to other DFAT investments.

      Is there transparent reporting on DFAT’s achievement of value for money in the delivery of aid through facility arrangements?

      DFAT’s reporting on achievement of value for money in the delivery of aid through facility arrangements is largely transparent in the detail of its reporting on individual aid investments. Performance reporting on contractors occurs only at the aggregate level. Transparency would be enhanced through increased reporting on the Government’s spend on the non-aid component of facility contracts.

      4.39 Effective reporting on performance helps the Parliament and the public form judgements about whether an entity is delivering its intended results.133 The audit has examined DFAT’s facility-related reporting against strategic targets; value for money; use of benchmarks and comparators; and public information about contracts.

      Reporting against strategic targets

      4.40 DFAT has established 10 strategic targets used to assess the aid program against the Australian Government’s key goals and priorities.134 DFAT reports on its achievement of these targets through its annual Performance of Australian Aid Report (refer to Figure 4.1).

      4.41 As defined in Making Performance Count, strategic target eight (‘Ensuring value for money’) requires DFAT to ‘deliver high standards of value-for-money in at least 85 per cent of aid investments’. DFAT considers this target to have been met if at least:

      • 85 per cent of aid investments are rated at least ‘satisfactory’ for effectiveness; and
      • 85 per cent of aid investments are rated at least ‘satisfactory’ for efficiency.

      4.42 All investments assessed through an AQC process as ‘satisfactory’ (a rating of four or above) are considered to meet the definition of delivering a ‘high standard of value for money’.135 As at November 2019, the KIAT facility had been in operation for two cycles of AQC reporting and the PAGP for three.136 In 2018, KIAT received a score of three (less than adequate) for efficiency and PAGP administrative unit was assessed at level three for both effectiveness and efficiency in 2017. Both KIAT and PAGF were rated four for effectiveness and efficiency in their most recent AQCs.

      Reporting on the extent to which value for money has been achieved

      4.43 Performance information should indicate the extent to which an entity is meeting its purpose through the activities it undertakes.137 The audit examined whether DFAT makes full use of the rating scale to enable judgements to be made about how well investments are performing.

      Reporting against categories of value

      4.44 DFAT reports on the aggregate results for all investments only, irrespective of value. Specifically, in 2017–18, 92 per cent of investments were rated as satisfactory for effectiveness and 88 per cent were rated as satisfactory for efficiency.

      4.45 In order to determine whether there would be benefit in reporting at a more granular level, the ANAO broke down the results of AQC assessments into three value categories that could be used to assess performance. The results of this analysis are provided at Appendix 9.

      • category one — $3 million to $15 million (as AQCs are required for investments valued at $3 million and above; at $15 million there is a requirement for an Aid Investment Plan);
      • category two — $15 million to $100 million (as investments valued at $100 million and above are required to be reviewed by the Aid Governance Board); and
      • category three — $100 million and above.138

      4.46 The analysis indicates that ratings do not significantly differ across categories. Total AQC results from 2013–14 to 2017–18 for the effectiveness and efficiency criteria show that a slightly higher proportion of investments has been assessed as having as being unsatisfactory for efficiency at least once in the relevant period compared to effectiveness, suggesting a need for a greater focus on investment efficiency.

      Use of benchmarks

      4.47 Benchmarks can provide information about the efficiency of a program or activity if they are used to compare similar activities.139 Benchmark comparisons may be undertaken against similar organisations (domestic and international) or against the organisation’s own performance (for example, across facilities). Benchmarks can focus on the inputs or outputs of aid program activity (for example, cost of vaccines) or on implementing arrangements (for example, proportion of staff working on monitoring and evaluation).

      4.48 A number of international aid agencies undertake evaluations aimed at linking aid expenditure to development outcomes. As an example, in 2015 the United States Agency for International Development commenced work to evaluate the cost-effectiveness of a set of its programs by comparing their per dollar impact with that of a cash transfer arrangement. In 2018, it published the results of a benchmarking study of aid to Rwanda, which had examined the cost-effectiveness of alternative programs in improving community nutrition.

      4.49 The UK Department for International Development (DIFD) identifies key costs and value for money indicators in the design of individual programs to be monitored over life of the investment. This enables baselines to be established and original assumptions about value for money to be tested.140

      4.50 The ANAO did not find examples internationally of agencies benchmarking administration costs in the delivery of aid. DFAT currently does not benchmark the cost-effectiveness of different mechanisms for the delivery of aid, including facilities.141 While it may be difficult for DFAT to benchmark the efficiency and effectiveness of aid activities against other aid organisations, it could benchmark its contract implementation practices against other Australian Government entities with large international contracts (such as the departments of Defence or Home Affairs).142

      Public information about contracts

      4.51 DFAT’s commitment to transparency and accountability in aid reporting includes publishing aid policies, plans, evaluations and research on its website, together with an annual report on the performance of Australian aid.143 The ANAO reviewed DFAT’s public reporting on the means by which aid is delivered, including via facility contracts, and on the costs associated with these.

      Performance of Australian Aid Report and the Annual Report

      4.52 There is no information in DFAT’s 2018–19 annual report about expenditure on contractors within the aid program or by the department more broadly. DFAT’s annual Performance of Australian Aid Report reports against a strategic target for ‘working with the most effective partners’. The report provides the overall value of commercial agreements and the average Partner Performance Assessment score of commercial partners. DFAT should consider providing more detailed public reporting on the Government’s spend on the non-aid component of facility contracts.

      4.53 Some information about the performance of individual contractors is published in reporting against country Aid Investment Plans. In the 2017 report on its achievement of aid objectives in Papua New Guinea, DFAT advised that it would be ‘working closely with the PAGP managing contractor to improve program delivery’ and that ‘the PAGP was subject to a remediation plan and enhanced senior level oversight.’144

      4.54 Since 2015, entities have been required to report each contract and amendment at or above $10,000 (GST inclusive) on the AusTender website.145 AusTender reporting is optional for contracts resulting from goods or services procured and used overseas. DFAT advised that its practice is to report all contracts on AusTender, including overseas contracts. 146

      Appendices

      Appendix 1 Entity response

      Response from the Secretary of the Department of Foreign Affairs and Trade

      Appendix 2 KIAT timeline

      KIAT timeline

      The figure is a timeline of the KIAT facility. It covers the: planning; procurement; design; mobilisation; and implementation periods. The timeline for the KIAT facility commences in April 2014 with the design for Indonesia infrastructure assistance.

      Source: ANAO analysis of data provided by DFAT.

      Appendix 3 KIAT and PAGP program stream outlines

      Work stream

      Description of aim

      KIAT

      Urban Water Supply and Sanitation

      Increase levels of services from water utilities by improving utility management and facilitating effective use of Indonesian central government funding through incentive grants.

      Expand levels of urban sanitation coverage form publicly owned and operated sewerage systems.

      Urban Solid Waste Management

      Support the Government of Indonesia in developing strategies, pilot programs and investments to improve urban solid waste management. Prepare solid waste projects involving financing from the Multilateral Development Banks and potential private sector partners.

      Road Network Development and Management

      Support development of a National Road Network Masterplan highlighting priority arterial roads for targeted upgrading and maintenance.

      Strengthen management of existing road networks through increasing investment in road maintenance; getting better value for money from road infrastructure delivery; addressing institutional capacity constraints; and systematic planning and allocating resources.

      Urban Mobility and Road Safety

      Plan for improved urban mobility with other development partners including Multilateral Development Banks to develop downstream public transport infrastructure improvement projects.

      Infrastructure Financing and Funding

      Identify opportunities to promote private sector engagement in infrastructure financing and delivery.

      Gender Equality and Social Inclusion

      Strengthen dialogue between government and civil society to ensure that infrastructure meets citizen needs.

      Immediate and Emerging Needs

      Respond to urgent infrastructure-related needs and to identify potential new areas for support.

      PAGP

      Decentralisation and Citizen Participation

      Develop local solutions; increase citizen engagement and participation in development; and ensure PNG’s decentralisation policies are well developed and well understood.

      PNG Partnership Fund (grants)

      Improve coverage and quality of basic services and achieve critical and sustainable outcomes at scale.

      Public Sector Leadership and Reform

      Develop a capable and professional public service in PNG through cultural change driven by senior executives across the PNG public service.

      Kokoda Initiative

      Protect the Kokoda Track, Brown River catchment and Owen Stanley Range; improve services for local communities; enhance the trekking experience; improve safety; and preserve the region’s unique culture and military history.

      Economic Governance and Inclusive Growth

      Promote inclusive and sustainable economic growth through macroeconomic stability, strengthened fiscal management, and private sector development.

      Bougainville Program

      Strengthen core government functions under Bougainville’s autonomy arrangements: economic development; peace and stability; and community cohesion.

      Institutional Partnerships Program/ Deployment Support Services

      Deliver logistical support services to deployees to PNG from Australian Government entities.

         

      Source: ANAO analysis of data provided by DFAT.

      Appendix 4 PAGP timeline

      PAGP timeline

      The figure is a timeline of the PAGP facility. It covers the: design; procurement; mobilisation; and implementation periods. The timeline for the PAGP facility commences in March 2012 with reviews of predecessor programs and covers up until June 2019 when

      Source: ANAO analysis of data provided by DFAT.

      Appendix 5 KIAT facility structure

      This figures illustrates the structure of the KIAT facility. It shows the relationship between the Department of Foreign Affairs and Trade, the Government of Indonesia and the managing contractor. It also outlines the organisational arrangements that the managing contractor has in place to manage the facility.

      Source: ANAO analysis of data provided by DFAT.

      Appendix 6 PAGP facility structure

      PAGP facility structure

      This figure illustrates the structure of the PAGP facility. It shows that there are six streams of work which are supported by a shared services platform.

      Source: ANAO analysis of data provided by DFAT.

      Appendix 7 DFAT portfolio budget statement performance criteria 2019–20

      1. DFAT Programs 1.2 and 1.3 relating to the Australian Government’s delivery of Official Development Assistance contribute to the portfolio’s achievement of Outcome 1:

      The advancement of Australia’s international strategic, security and economic interests including through bilateral, regional and multilateral engagement on Australian Government foreign, trade and international development policy priorities.

      2. DFAT’s performance criteria for 2019–20 for Programs 1.2 and 1.3 are:

      Program 1.2:
      • Official Development Assistance promotes Australia’s national interest by contributing to sustainable and inclusive economic growth and poverty reduction.
      • Through assessments, surveys, evaluations, audits or other measures, DFAT will demonstrate whether these performance criteria are: Achieved / On track / Not on track.
      • Specific targets:
        • Investments promote sustainable and inclusive economic growth and poverty reduction.
        • 90 per cent of country attributable development assistance spent in the Indo-Pacific.
        • Indo-Pacific countries make progress towards achieving the Sustainable Development Goals.
        • At least 85 per cent of investments are assessed as satisfactory on both effectiveness and efficiency criteria in the Aid Quality Check process.
        • All country and regional Aid Program Performance Reports published on the department’s website annually.
      Program 1.3:
      • Official Development Assistance Multilateral Replenishments assist developing countries by contributing to sustainable and inclusive economic growth and poverty reduction through contributions to multilateral organisations.
      • Through assessments, surveys, evaluations, audits or other measures, DFAT will demonstrate whether these performance criteria are: Achieved / On track / Not on track.
      • Specific targets:
        • Investments promote sustainable and inclusive economic growth and poverty reduction.
        • Indo-Pacific countries make progress towards achieving the Sustainable Development Goals.

      Appendix 8 DFAT performance reporting framework

      Performance reporting

      Purpose

      Performance of Australian Aid Report (PAAR)

      (annual)

      To report to the Parliament and the public on the performance of the Australian aid program over a financial year. Updates on progress towards the 10 strategic targets for the Australian aid program.

      The PAAR includes a summary of the performance of country, regional and global aid programs, as well as performance across priority investment areas.

      Aid Program Performance Reports (APPRs)

      (annual)

      Reports on the performance of the Australian aid program in a country or region over the course of a financial year, across relevant aid investments.

      APPRs draw on internal reporting, including Aid Quality Checks and Partner Performance Assessments (see below), and assessments by external quality and technical assurance consultancies.

      Aggregate Development Results (ADRs)

      (annual)

      Collates data on development impact within a country. Data can be aggregated across the aid program to demonstrate the contribution of Australian aid to meeting the Government’s objectives.

      Aid Quality Checks (AQCs)

      (annual)

      Assessment of the performance of an aid investment over the previous 12 months.

      There are three types of AQCs: standard AQCs; humanitarian AQCs; and Final AQCs completed in the last year of an investment to assess performance over its lifespan. More than one AQC may be prepared for an investment; and more than one investment may be assessed under an AQC.

      AQCs measure performance against standard quality criteria; record actions undertaken to improve performance; and provide information about the effectiveness and efficiency of the investment.

      Partner Performance Assessments (PPAs)

      (annual)

      Examines the performance of implementing partners’ management of investments.

      PPAs may inform future tender evaluations and decisions on aid grants and funding to multilateral organisations.

      Performance Assessment Frameworks (PAF) milestone reporting

      (six-monthly)

      PAF milestone reporting tracks progress toward the achievement of bilateral or regional policy objectives set out in Aid Investment Plans.

      A PAF may exist at country and investment level. The PAGP has PAFs in place for each of its work-streams.

      Monitoring, Evaluation, Reporting (and Learning) Framework

      (MELF) reporting

      (continuous)

      Monitors the performance of an investment throughout implementation.

      MELFs enable the collection and analysis of performance data to inform the implementation of an investment and reporting against PAFs, as well as to support future evaluation and reporting on aid outcomes.

      Program and organisational assessments

      (as required)

      Targeted reviews commissioned to support assessments of progress or outcomes of an investment or the performance of an implementing partner and/or mechanism.

         

      Source: ANAO analysis of data provided by DFAT.

      Appendix 9 Effectiveness and efficiency measures, aggregated results 2013–14 to 2017–18

      Value category ($million)

      1

      2

      3

      4

      5

      6

      Total

      Satisfactory investments (%)

      Unsatisfactory investments (%)

      Effectiveness

      $3m to $15m

      0

      8

      125

      506

      388

      12

      1039

      87.2

      12.8

      $15m to $100m

      0

      4

      94

      409

      416

      17

      940

      89.6

      10.4

      $100m+

      0

      0

      10

      68

      89

      2

      169

      94

      6

      Total

      0

      12

      229

      983

      893

      31

      2148

      88.8

      11.2

      Efficiency

      $3m to $15m

      0

      14

      161

      497

      361

      6

      1039

      83.2

      16.8

      $15m to $100m

      1

      12

      115

      471

      330

      11

      940

      86.4

      13.6

      $100 m+

      0

      1

      16

      66

      79

      7

      169

      90

      10

      Total

      1

      27

      292

      1034

      770

      24

      2148

      85.1

      14.9

                         

      Source: ANAO analysis of data provided by DFAT.

      Note: The total value of investments across the five financial years is around $86 billion. The value of an investment may be counted more than once if it is active in more than one year.

      Footnotes

      1 The remainder is delivered through other Federal government entities and state and territory governments.

      2 DFAT’s total ODA spend was $3976 million in 2018–19.

      3 DFAT also manages enabling facilities, which deliver transactional services in support of an aid program, such as recruitment and logistical support services.

      4 The value of individual investments ranged from $11 million to $775 million.

      5 ODA refers to all government aid that specifically targets the economic development and welfare of developing countries. It includes assistance provided by state and local governments and other executive entities. Organisation for Economic Co-operation and Development (OECD), What is ODA? [Internet], OECD, France, 2019, available from http://www.oecd.org/dac/financing-sustainable-development/development-finance-standards/What-is-ODA.pdf [accessed 30 March 2020].

      6 Department of Foreign Affairs and Trade, Australian Aid: Promoting Prosperity, Reducing Poverty, Enhancing Stability [Internet], DFAT, available https://www.dfat.gov.au/about-us/publications/Pages/australian-aid-promoting-prosperity-reducing-poverty-enhancing-stability [accessed 30 March 2020].

      7 Department of Foreign Affairs and Trade, Making Performance Count: Enhancing the Accountability and Effectiveness of Australian Aid [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/making-performance-count-enhancing-the-accountability-and-effectiveness-of-australian-aid [accessed 30 March 2020].

      8 DFAT defines investment as ‘an intervention designed to achieve specific outputs and outcomes and contribute to the overall outcomes of a program.’ Department of Foreign Affairs and Trade, Aid Programming Guide [Internet], available from https://www.dfat.gov.au/about-us/publications/Pages/aid-programming-guide [accessed 30 March 2020], p. 45.

      9 Department of Foreign Affairs and Trade, Annual Report 2013–14 [Internet], DFAT, available from https://dfat.gov.au/about-us/publications/corporate/annual-reports/pages/annual-reports.aspx [accessed 30 March 2020].

      10 As at 30 June 2019, 24 of 120 posts had aid management responsibilities.

      11 Paragraph 15(1)(a) of the Public Governance, Performance and Accountability Act 2013 (Cth), Compilation No. 4, Office of Parliamentary Counsel, Canberra.

      12 Department of Finance, Commonwealth Procurement Rules [Internet], Department of Finance, available from https://www.finance.gov.au/sites/default/files/2019-11/CPRs-20-April-2019_1.pdf [accessed 30 March 2020].

      13 Department of Foreign Affairs and Trade, Value for Money Principles [Internet], DFAT, available from https://dfat.gov.au/aid/who-we-work-with/value-for-money-principles/Pages/value-for-money-principles.aspx [accessed 30 March 2020].

      14 Ibid.

      15 In 2014, the Australian Government set a target of reducing the number of individual investments by 20 per cent by 2016–17 to focus effort and reduce transaction costs. Department of Foreign Affairs and Trade, Making Performance Count: Enhancing the Accountability and Effectiveness of Australian Aid [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/making-performance-count-enhancing-the-accountability-and-effectiveness-of-australian-aid [accessed 30 March 2020]. As at 1 July 2018, the number of aid investments had reduced by 32 per cent (against baseline data from 1 July 2013), in Department of Foreign Affairs and Trade, Performance of Australian Aid Report 2017–18 [Internet], DFAT, p. 9 available from https://dfat.gov.au/about-us/publications/Pages/performance-of-australian-aid-2017-18.aspx [accessed 30 March 2020].

      16 The Accra Agenda for Action 2008 reaffirmed the goals of the 2005 Paris Declaration on Aid Effectiveness, which included the objective to consolidate aid investments. See [Internet], OECD, France, 2019, available from https://www.oecd.org/dac/effectiveness/parisdeclarationandaccraagendaforaction.htm [accessed 30 March 2020].

      17 DFAT’s total ODA spend was $3976 million in 2018–19.

      18 DFAT also manages enabling facilities, which deliver transactional services in support of an aid program, such as recruitment and logistical support services.

      19 Corporate functions generally include human resource management, procurement, finance and audit functions, as well as strategic planning, and evaluation and reporting.

      20 The current term of the KIAT contract is 1 September 2016 to 30 June 2021. The design of the facility commenced in September 2016 and implementation began in April 2017.

      21 The first phase of implementation of the KIAT facility is April 2017 to June 2021. An option to extend the facility to 2026 depends on a future decision by government on investment funding.

      22 The grants funding is not included in the KIAT aid program budget, as funds are transferred by DFAT to the Government of Indonesia directly. The grants relate to programs for improving water, sanitation and road infrastructure and services.

      23 Cardno’s key services relate to the infrastructure and environment sectors and international development.

      24 A further 20 per cent of programs are delivered in other parts of Asia and around 30 per cent are delivered in Papua New Guinea.

      25 At that time, the PAGP was called the Papua New Guinea Governance Facility (PGF). In October 2017 the Australian and Papua New Guinea governments agreed to rename the PGF the PAGP.

      26 Separate funding of $3.7 million (plus $0.4 million GST) per year is provided to support external evaluations of the facility’s performance.

      27 This figure is subject to change based on bilateral budget allocations and changing priorities.

      28 Total ODA expenditure to PNG in 2018–19 was reported as $578 million. See Aid Program Performance Report 2018–19, Papua New Guinea, September 2019, [internet], Department of Foreign Affairs and Trade available from https://www.dfat.gov.au/about-us/publications/Pages/papua-new-guinea-aid-program-performance-report-2018-19, p. 5.

      29 Fifteen per cent are delivered in Indonesia and 10 per cent are delivered in Timor-Leste.

      30 These definitions are provided in the Standard on Assurance Engagements ASAE 3500 Performance Engagements issued by the Auditing and Assurance Standards Board and is applied by the ANAO in its audit work.

      31 It should be noted that employees of the ANAO participate in an Australian Government program, termed the Institutional Partnerships Program (IPP), which provides governance-related technical advice to the PNG Government. The IPP is managed by DFAT, with ANAO employees deployed to PNG supported by logistics services delivered by the PAGP. The value of ANAO involvement in the IPP in 2019–20 is approximately $0.9 million. The audit excluded the IPP and the facility’s delivery of support services from its examination of the PAGP.

      32 Since 1 January 2018, joint Canberra policy and post sign-off has been mandatory for new investment concepts and designs valued at $10 million or more and/or high risk. Canberra-based delegates are accountable for verifying that investment designs meet minimum standards.

      33 Further criteria are: sustainability; gender equality; risk management and safeguards; innovation and the private sector.

      34 Department of Foreign Affairs and Trade, Aid Programming Guide [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/aid-programming-guide [accessed 30 March 2020].

      35 An investment valued at $100 million or more is considered to be high-value. Risk is self-assessed by program owners against nine categories. A high risk rating is based on three high risks or one very high risk.

      36 The IndII facility contract commenced 1 July 2011 and concluded on 30 June 2017. The managing contractor was SMEC International.

      37 Leveraging resources refers to the use of aid funds to attract additional expenditure (public or private) in priority areas during the life of an investment.

      38 DFAT commissioned three evaluation reports of the IndII facility between February 2014 and September 2016, (Impact Assessment Team (IAT) Mission Reports).

      39 A facility that primarily deploys technical advisers may not require the same level of organisational resources as one responsible for constructing projects or grants administration.

      40 Auditor-General Report No.39 2012–13, AusAID’s Management of Infrastructure Aid to Indonesia, and KPMG, IAT: Mission 3 Report, Indonesia Infrastructure Initiative (IndII) Phase 2 [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/indonesia-indii-program-mission-3-report [accessed 30 March 2020].

      41 A $138.6 million program (2009–2013) under which Australian public servants provided advice and capacity- building support to PNG government agencies.

      42 Department of Foreign Affairs and Trade, A New Direction for Australian Aid in PNG: Refocusing Australian Aid to Help Unlock PNG’s Economic Potential, Twenty-Second Papua New Guinea–Australia Ministerial Forum [Internet], DFAT, p. 3 available from https://www.dfat.gov.au/about-us/publications/Pages/a-new-direction-for-australian-aid-in-png-refocusing-australian-aid-to-help-unlock-pngs-economic-potential [accessed 30 March 2020].

      43 DFAT advised the ANAO that these reductions did not eventuate due to the greater than expected resource demands of establishing the PAGP and the subsequent addition of programs.

      44 As an example, the largest estimated saving was a reduced number of long-term advisers and locally engaged staff ($3.6 million annually), however it is not evident that this saving was benchmarked against the need for work performed by these personnel to be contracted into the facility.

      45 Documents relating to the facilities were examined, including investment concept and briefing notes, QRAU technical assessments and meeting minutes.

      46 One proposal (valued at $60 million over four years) noted that management fees would constitute 15 per cent of the budget, however information about likely facility operating costs was not provided.

      47 Department of Finance, Commonwealth Procurement Rules [Internet], Department of Finance, available from https://www.finance.gov.au/sites/default/files/2019-11/CPRs-20-April-2019_1.pdf [accessed 30 March 2020].

      48 When first established the AGB had two independent members. As at March 2020, the AGB had one independent member due to the second member being appointed to the role of Chief Economist within DFAT in November 2019.

      49 As at October 2019, 622 of 801 investments had a risk rating, with 18 identified as high risk.

      50 The basis for selecting the 15 investments was broadened beyond being of high value following the AGB’s consideration of risk criteria in November 2019.

      51 Department of Finance, Commonwealth Procurement Rules [Internet], Department of Finance, available from https://www.finance.gov.au/sites/default/files/2019-11/CPRs-20-April-2019_1.pdf [accessed 30 March 2020], p. 11.

      52 An open tender procurement process involves publishing an open approach to marketing and inviting submissions.

      53 Subsequent to the KIAT procurement, DFAT issued guidance to staff in February 2018 stating that the department does not endorse a design-implementation procurement approach. A design process managed by DFAT is preferred.

      54 It considered the risk of procuring implementation services ahead of a finalised design to be minimal given the continuing suitability of the existing IndII design.

      55 Design activity needed to be finalised before the cessation of the existing IndII facility in January 2017 and to allow time for the novation of IndII activities into the new facility.

      56 In February 2016, DFAT held an industry briefing session to provide an overview of the KIAT proposal and information about the procurement process. DFAT responded to 65 questions from the market throughout the tender process, attaching its responses to the RFT.

      57 The PAGP design had already been released on DFAT’s website on 7 April 2015, approximately four months before the RFT was published.

      58 DFAT offered face-to-face meetings with potential tenderers to answer their questions. The department attached questions and answers on significant matters to AusTender. DFAT answered 52 questions from potential tenderers.

      59 This assessment of the bidding organisation examines the capacity of the organisation to deliver the facility based on its past performance. The criterion focuses on the corporate capacity and capability of the organisation and its infrastructure and technical capability.

      60 The Evaluation Committee did not remove costs it considered should not have been included in the proposal and did not request Cardno to provide a revised financial proposal.

      61 DFAT engaged an independent financial firm to undertake a financial viability assessment. This assessment evaluates the risk that, over the life of a proposed contract, a tenderer may not be able to deliver the goods and services which are specified in the contract; or may not be able to fulfil guarantees or warranties provided for in the contract. Department of Finance, Assessing Financial Viability, available from https://www.finance.gov.au/government/procurement/buying-australian-government/assessing-financial-viability [accessed 30 March 2020].

      62 A guarantee is a contractual undertaking where one party assumes responsibility for the debt, or performance obligations, of another party should that other party default in some way. A performance guarantee involves a parent company agreeing to take over provision of the services if the contractor fails to provide the services. Department of Finance, Guarantees, available from https://www.finance.gov.au/government/commonwealth-investment-framework/commonwealth-investments-toolkit/guarantees [accessed 30 March 2020].

      63 Cardno’s Tenderer’s Declaration submitted as a requirement of bidding stated that it would seek to exclude a performance guarantee from any contract. The tenderer’s declaration affirms that the tenderer has sufficient financial resources to deliver the goods or services described in the RFT (including fulfilling any guarantees or warranty claims). Department of Finance, Assessing Financial Viability, available from https://www.finance.gov.au/government/procurement/buying-australian-government/assessing-financial-viability [accessed 30 March 2020].

      64 The delegate was the Australian Ambassador to Indonesia.

      65 The EB was supported by two sub-committees: the Technical Evaluation Committee; and the Operations Evaluation Committee.

      66 The value for money index measures the financial cost of each technical and operational point given to the tenderer by the Evaluation Committee.

      67 Abt was ranked second for scenario one (total activity expenditure of $200 million) and second for scenario two (total activity expenditure of $250 million). Neither of these scenarios eventuated.

      68 At Deputy Secretary level.

      69 Implementation phase costs were reduced by 37.3 to 38 per cent compared to the original proposal, however DFAT agreed to pay mobility allowances for eligible advisers, representing an increase of 19.5 per cent.

      70 The management fee total confirmed in the first contract amendment is around $450,000 less than Cardno proposed in its original bid. The management fee as a percentage of the total of tendered management costs reduced from 39 per cent to 28 per cent.

      71 This amount is the allocation for both aid activities and administration over a period of four years and three months.

      72 Department of Finance guidance states that an entity should consider whether a variation to a contract may compromise the original procurement’s value for money assessment. Department of Finance, Australian Government Contract Management Guide [Internet], Department of Finance, available from https://www.finance.gov.au/sites/default/files/2019-11/Australian%20Government%20Contract%20Management%20Guide%20-%20Nov%202019.pdf [accessed 30 March 2020].

      73 According to the original schedule the facility had been due to commence operations ahead of the expiry of extant agreements at the end of December 2015. These agreements were then extended to 1 July 2016.

      74 Issues for negotiation were outlined in advice to the delegates approving the tender evaluation and negotiation outcomes.

      75 This includes formalising: the negotiating team and their responsibilities; conduct of the negotiations; specifying negotiation issues; and reporting.

      76 In total, there have been 12 service orders since the PAGP’s inception.

      77 DFAT has not needed to exercise this option in relation to the PAGP.

      78 The procurement fee is similar to a management fee. For both the management fee and procurement fee, 80 per cent is paid upon acceptance of milestone deliverables and 20 per cent is paid quarterly on acceptance or achievement of KPIs.

      79 This includes the period of the contract extension.

      80 The design detailed activities and cost estimates for the first 15 months of the facility (April 2017 to June 2018).

      81 DFAT advised that ‘it is now widely accepted that KIAT’s first year spending target was unrealistic’.

      82 Six months later than originally envisaged.

      83 These had been extended by six months from January to end June 2016. Department of Foreign Affairs and Trade, Economic and Public Sector Program — Completion Report [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/png-epsp-completion-report [accessed 30 March 2020]. In its 2016—17 Aid Program Performance Report, Papua New Guinea, DFAT reported that it had novated or established 37 grants in the six months from July 2017.

      84 Services include: financial management and reporting; procurement and contracting; grant administration and management; logistics and travel; human resource management; and risk management.

      85 In December 2017, the Secretary of DFAT appointed Senior Responsible Officers to five high risk, high value investments, including the PAGP.

      86 Department of Finance, Australian Government Contract Management Guide [Internet], Department of Finance, available from https://www.finance.gov.au/sites/default/files/2019-11/Australian%20Government%20Contract%20Management%20Guide%20-%20Nov%202019.pdf [accessed 30 March 2020].

      87 This included activity proposals (concept notes and design briefs), approval emails, and formal reporting to facility committees.

      88 Commonwealth entities have legislative record-keeping obligations to under the Archives Act 1983, the Public Governance, Performance and Accountability Act 2013 and the Criminal Code Act 1995. See guidance available from https://www.finance.gov.au/government/procurement/clausebank/record-keeping-management [accessed 30 March 2020].

      89 The Commonwealth Procurement Rules require documentation to provide accurate information about the requirement of the procurement; the process followed; how value for money was considered and achieved; relevant approvals; and relevant decisions and the basis of those decisions (CPR 7.3).

      90 Section 16(a), Public Governance, Performance and Accountability Act 2013 and the Commonwealth Risk Management Policy.

      91 DFAT published a Risk Management Guide for Aid Investments, (dated January 2019) in October 2019, available from https://www.dfat.gov.au/about-us/publications/Pages/risk-management-for-aid-investments-guide [accessed 30 March 2020]

      92 Paragraph 10 of the Public Governance, Performance and Accountability Rule 2014 (Cth), Compilation No. 28, Office of Parliamentary Counsel, Canberra.

      93 Fraud committed against DFAT may extend beyond the unauthorised use of funding to encompass matters such as influencing a tender assessment; supplier selection process or the awarding of other benefits; insider trading; and failure to report conflicts of interests.

      94 1 July 2017 and 30 June 2019. Figures include allegations closed on the basis of the fraud not being substantiated.

      95Fraud Control in the Department of Foreign Affairs and Trade (due to table June 2020), information available from https://www.anao.gov.au/work/performance-audit/fraud-control-the-department-foreign-affairs-and-trade [accessed 30 March 2020].

      96 Risks relate to fraud and corruption; sanctions; counter-terrorism; employee integrity; risk management systems; child protection; and the prevention of sexual exploitation; abuse and harassment; and environmental and social safeguards.

      97 Department of Foreign Affairs and Trade, Environmental and Social Safeguard Policy [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/environmental-social-safeguard-policy [accessed 30 March 2020].

      98 The PAGP is currently in the process of also developing a MIS.

      99 Personnel costs include facility management or administrative staff and contracted technical advisers.

      100 Anticipated efficiencies related to contractor staff; office set-up and running costs; and DFAT staffing.

      101 The Aid Quality Check for 2019 reported on: engagement of senior staff located in Port Moresby; reduced housing and mobilisation costs ($2.5 million since 2017–18); improved fleet management, property and security services savings ($1.3 million for the previous calendar year); and consolidation of Abt management services in one office location ($0.7 million since 2017–18). DFAT advised the ANAO in January 2020 that a total of $9.1 million in savings had been achieved as at 2018–19.

      102 Until June 2019, the managing contractor of the PAGP had maintained a value for money workbook aimed at enabling facility operations to be assessed against DFAT principles. However, cost savings recorded were not verified.

      103 Jakarta and Port Moresby posts are able to obtain detailed expenditure information for the KIAT and PAGP facilities from the managing contractors. This information is mainly used to track actual aid expenditure against the annual investment allocation compared to forecast expenditure.

      104 The procurement fee only applies to the PAGP — the KIAT contract does not include a procurement fee.

      105 A report commissioned by DFAT in 2017 assumed a 50/50 split between administrative and aid delivery personnel. This was based on information provided by Abt, the managing contractor of the facility (Australia Timor–Leste Partnership for Human Development). If a 50/50 split for personnel is used for the KIAT and PGF facilities, the administration to aid ratios are 18:82 and 29:71 respectively.

      106 In part, this reflects reductions to the country allocation for Indonesia in 2019–20.

      107Public Governance, Performance and Accountability (Financial Reporting) Rule 2015, section 5 – Definitions.

      108 Figures correct as at July 2019.

      109 Figures correct as at November 2019.

      110 For this calculation, it is assumed that all DFAT staffing costs are administrative and that DFAT staffing has remained constant since the commencement of the facility.

      111 The KIAT design did not specify resourcing required to manage the KIAT facility. The PAGP design estimated the facility would deliver efficiency savings of $6 million over five years as result of reduced staffing requirements, but it has not sought to verify whether these savings have been achieved.

      112 All sub-contracting costs under a facility are reimbursed by DFAT. Proposals that use sub-contractors therefore require DFAT approval prior to being let.

      113 Abt did not specify costs for procurement of sub-contractors in its bid.

      114 An amount of $144,274 is from years one to five and $184,642 from years six to ten.

      115 The CPRs outline exemptions from open tender approaches, including: when for reasons of extreme urgency brought about by events unforeseen by the relevant entity, the goods and services could not be obtained in time under open tender; or when the goods and services can be supplied only by a particular business and there is no reasonable alternative or substitute (CPRs 10.3). See https://www.finance.gov.au/sites/default/files/2019-11/CPRs-20-April-2019_1.pdf [accessed 30 March 2020].

      116 Department of Foreign Affairs and Trade, 2017 Foreign Policy White Paper [Internet], DFAT, available from https://www.fpwhitepaper.gov.au/ [accessed 30 March 2020].

      117 Department of Foreign Affairs and Trade, Corporate Plan 2019–20 [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/corporate/Pages/dfat-corporate-plan [accessed 30 March 2020].

      118 Australian Government, Portfolio Budget Statements, Budget Related Paper No 1.8 [Internet], Commonwealth of Australia, Canberra, 2019 available from https://dfat.gov.au/about-us/corporate/portfolio-budget-statements/Pages/portfolio-budget-statements.aspx [accessed 30 March 2020] pp. 29–31.

      119 Examples of indicators are: ‘amount of additional financing of infrastructure; number of districts with improved service delivery; number of kilometres of roads maintained; number of feasibility studies contributing to project preparation; number of instances of substantive collaboration between financing institutions.’

      120 The MELF Public Sector Leadership and Reform stream of work developed by Abt has not been approved by DFAT.

      121 Department of Foreign Affairs and Trade, Evaluation of DFAT Investment Level Monitoring Systems [Internet], DFAT, available from https://dfat.gov.au/aid/how-we-measure-performance/ode/strategic-evaluations/Pages/evaluation-of-investment-level-monitoring-systems.aspx [accessed 30 March 2020].

      122 ibid., p. 6. This was Recommendation 5, which was agreed to by DFAT management.

      123 PPAs are completed for commercial agreements and grants valued at $3 million and above.

      124 There are three types of AQCs: standard AQCs; humanitarian AQCs; and Final AQCs completed in the last year of an investment to assess performance over its lifespan. These are typically completed by middle managers (Executive Level 2). Since 2017 reports are required to be validated by a senior program manager within the responsible division.

      125 Individual AQCs can cover more than one investment.

      126 Department of Foreign Affairs and Trade, Performance of Australian Aid 2017–18, p. 75.

      127 The AQC process is optional for investments under $3 million. Exemptions may apply to investments implemented for less than six months. All exemptions must be approved by the First Assistant Secretary of Contracting and Aid Management Division and recorded in AidWorks.

      128 Department of Foreign Affairs and Trade, Making Performance Count: Enhancing the Accountability and Effectiveness of Australian Aid. p. 10.

      129 In its 2016—17 Aid Program Performance Report, Papua New Guinea, DFAT reported that an overall decline in AQC scores for PNG investments was attributable to ‘management increasingly demanding more robust and evidence-based data linking inputs and outputs to higher level outcomes.’, p. 31.

      130 The sAIIG program, a component of the KIAT facility, was listed as an Investment Requiring Improvement in 2016.

      131 The Bougainville program, Kokoda Initiative and Deployee Support Services were not considered to be underperforming and therefore were not included in the remediation plan.

      132 Other criteria relate to effective working relationships and use of personnel; conformance with DFAT policies; and innovation. The ODE quality assures a sample of AQC reports each year.

      133 Department of Finance, Resource Management Guide No.131, Developing Good Performance Information (RMG 131) [Internet], Department of Finance, available from https://www.finance.gov.au/government/managing-commonwealth-resources/developing-good-performance-information-rmg-131 [accessed 30 March 2020].

      134 Department of Foreign Affairs and Trade, Making Performance Count: Enhancing the Accountability and Effectiveness of Australian Aid [Internet], DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/making-performance-count-enhancing-the-accountability-and-effectiveness-of-australian-aid [accessed 30 March 2020]. A new international development policy is being prepared. This policy will include a revised performance framework.

      135 Ratings of good (level five) or very good (level six) mean that the ‘investment satisfies the criteria in almost all areas’ and ‘all areas’, respectively.

      136 Paragraph 4.24 outlines the AQCs that are undertaken for the KIAT and PAGP facilities.

      137 Department of Finance, Resource Management Guide No.131, Developing Good Performance Information [Internet], Department of Finance, available from https://www.finance.gov.au/government/managing-commonwealth-resources/developing-good-performance-information-rmg-131 [accessed 30 March 2020].

      138 The total value of investments across the five financial years is around $86 billion. The value of an investment may be counted more than once if it is active in more than one year.

      139 Department of Finance, Resource Management Guide No. 131, Developing Good Performance Information [Internet], Department of Finance, available from https://www.finance.gov.au/government/managing-commonwealth-resources/developing-good-performance-information-rmg-131 [accessed 30 March 2020].

      140 DIFD does not, however benchmark across aid investments.

      141 Reviews commissioned by DFAT in 2018 and 2019 have recommended that DFAT collect data to enable it to benchmark performance. This audit focusses on benchmarks regarding the operation of the facility, rather than benchmarks for the aid programs and activities that are being delivered.

      142 As a facility makes use of standard business systems and processes and staffing arrangements to deliver facility outputs, efficiency may be achieved through: reductions in unit, transaction or other production costs; rationalisation of functions resulting in reduced staffing requirements; and/or the consolidation of assets and/or operating systems.

      143 Department of Foreign Affairs and Trade, Australian aid: promoting prosperity, reducing poverty, enhancing stability, DFAT, available from https://www.dfat.gov.au/about-us/publications/Pages/australian-aid-promoting-prosperity-reducing-poverty-enhancing-stability [accessed 30 March 2020].

      144 Department of Foreign Affairs and Trade, Performance of Australian Aid Report 2017–18 [Internet], DFAT, available from https://dfat.gov.au/about-us/publications/Pages/performance-of-australian-aid-2017-18.aspx [accessed 30 March 2020].

      145 Pursuant to the 14 May 2015 amendment to the Senate Order on Contracts 2001.

      146 DFAT has reported the KIAT and PAGP contracts on AusTender. For KIAT, this information was uploaded onto AusTender as a result of this audit. DFAT advised that its failure to publish the contract was an administrative error.