Audit snapshot

Why did we do this audit?

  • Between 2012–13 and 2021–22, the Australian Renewable Energy Agency (ARENA) has access to over $2.2 billion to provide financial assistance to improve the competitiveness and supply of renewable energy in Australia.
  • As ARENA approaches the end of its legislated funding, it is important to provide the Parliament with assurance over the effectiveness of ARENA’s grants management.

Key facts

  • Twenty-eight per cent of 992 applications assessed by ARENA between July 2012 and July 2019 were successful in becoming ARENA funded projects.
  • $1.669 billion in funding assistance has been managed by ARENA to January 2020.
  • Nine renewable energy technologies have been supported by ARENA, including solar photovoltaic, grid integration, solar thermal, hybrid, bioenergy and storage technologies.
  • Fifty-nine per cent of the 44 funding agreements reviewed by ANAO had been varied, most commonly to extend milestone dates.

What did we find?

  • ARENA’s management of its grants programs is largely effective.
  • Grant project selection is aligned with ARENA’s objectives.
  • Management of grant funding agreements is largely effective.
  • Performance reporting and evaluation frameworks do not position ARENA to demonstrate that it is achieving its objectives.

What did we recommend?

  • The Auditor-General made six recommendations to ARENA to strengthen its:
    • performance measurement framework;
    • assurance that it is not funding activities that would occur without grant funding;
    • use of information systems in its grant management processes;
    • management of variations to funding agreements;
    • assurance of its performance measurement and reporting; and
    • conflict of interest management and disclosure for external evaluations.

Summary and recommendations

Background

1. The Australian Renewable Energy Agency (ARENA) is a corporate Commonwealth entity within the Industry, Science, Energy and Resources Portfolio.1 ARENA is established under the Australian Renewable Energy Agency Act 2011 (ARENA Act) with the objectives to:

  • improve the competitiveness of renewable energy technologies; and
  • increase the supply of renewable energy in Australia.2

2. The ARENA Act provides over $2.2 billion of Australian Government funding from 2012–13 to 2021–22.3 ARENA’s functions under the Act include:

  • providing financial assistance for research, development, demonstration, commercialisation or deployment of renewable energy technologies, or the storage and sharing of information and knowledge about renewable energy technologies; and
  • collecting, analysing, interpreting and disseminating information and knowledge relating to renewable energy technologies and projects.4

3. As at January 2020 ARENA had approved 538 applications seeking $1.669 billion in funding. Funded projects have included research, development, demonstration, commercialisation or deployment activities across nine types of renewable energy technologies. ARENA undertakes a range of knowledge sharing activities, including publishing reports and datasets on its website.5

Rationale for undertaking the audit

4. Substantial funding has been committed to renewable energy activities through ARENA. As ARENA approaches the end of its legislated funding, it is important to provide the Parliament with assurance over the effectiveness of ARENA’s grants management in improving the competitiveness of renewable energy technologies and increasing the supply of renewable energy in Australia.

Audit objective and criteria

5. The objective of the audit was to assess the effectiveness of grant program management by ARENA. The following high-level criteria were used to form a conclusion against this objective:

  • Does grant selection support the achievement of ARENA’s objectives?
  • Are grant funding agreements managed effectively?
  • Does evaluation of grant programs indicate that ARENA is achieving its objectives?

Conclusion

6. While ARENA’s grant program management is largely effective, its evaluation and performance reporting frameworks do not clearly demonstrate that its grant funding is increasing the supply and competitiveness of renewable energy in Australia beyond what would otherwise have occurred.

7. Strategic planning and grant project selection largely aligns with ARENA’s objectives. ARENA’s performance measurement framework does not provide a reliable basis to demonstrate to the Parliament and the public that ARENA is achieving its objectives.

8. ARENA’s management of grant funding agreements is largely effective. Improvements are required to ARENA’s management of variations and its integration of electronic systems with its business processes.

9. ARENA’s external evaluations since 2017 do not clearly demonstrate the extent to which ARENA’s programs are impacting on its legislative objectives of improving the supply and competitiveness of renewable energy in Australia.

Supporting findings

Strategic plans and application assessment

10. In 2018–19 ARENA’s corporate plan, general funding strategy, annual work plan and investment plan were mostly clear and consistent with ARENA’s objectives. The performance framework as set out in the corporate plan did not provide a reliable basis to demonstrate to the Parliament and the public that ARENA is achieving its objectives.

11. ARENA’s grant guidelines are appropriate and aligned with strategic plans except in relation to describing its assessment of additionality to determine whether proposed projects would achieve outcomes that would not otherwise occur without public funding. Clearly outlining how it assesses additionality would assist ARENA to demonstrate value for money in decision-making and avoid funding activities that would proceed without ARENA support.

12. Individual application assessments examined by the ANAO were generally consistent with grant guidelines and ARENA’s internal assessment framework. ARENA would be able to provide greater assurance over its grant selection by improving its use of information systems and standardising its assessment and decision-making processes, and record keeping.

Grant management

13. ARENA’s arrangements for monitoring the performance of individual funding agreements are largely appropriate. ARENA’s ability to provide assurance over the quality of its grants management would be enhanced by improved variation processes and increased integration of its business processes and information systems.

14. Governance arrangements for the oversight of grant funding agreements are robust. Funding agreement governance structures include an appropriate level of senior management oversight. Key data sources and reports were automated and digitised in late 2019, reducing the risk of the presentation of incorrect or incomplete information.

15. Risk and issues management processes are largely appropriate. The overall management of risks and issues in funding agreements has been improved by recent transitions of some of these activities onto information systems. Further integration of risk and issue management into business processes on information systems would enhance ARENA’s risk and issue management.

Evaluation and performance

16. In 2018–19 ARENA’s evaluation and performance policies and procedures were not appropriate. Evaluation and performance measurement activities were not aligned. The assurance process over the performance framework was not effective. Some improvements have been made to these policies and procedures in 2019–20.

17. The eight outcome evaluations undertaken since 2017 generally conclude that ARENA is delivering its programs, but only three provided clear conclusions about the impact of these programs on ARENA’s objectives. There is scope for ARENA to improve its evaluations so that they present clear, impact focused conclusions against ARENA’s legislative objectives. This and the implementation of recommendations on performance measures and additionality will enhance ARENA’s ability to attribute its outcomes to its legislative objectives and functions.

18. While lessons from ARENA’s evaluations were mostly actioned appropriately, there is scope for ARENA to strengthen its arrangements for closing recommendations.

Recommendations

Recommendation no.1

Paragraph 2.26

The Australian Renewable Energy Agency improves the reliability and completeness of its performance measurement and reporting framework.

Australian Renewable Energy Agency response: Agreed.

Recommendation no.2

Paragraph 2.43

The Australian Renewable Energy Agency clearly outlines in its grant guidelines how it assesses the additionality of project applications so that it can better demonstrate that the projects it funds would not have proceeded without public funding.

Australian Renewable Energy Agency response: Agreed.

Recommendation no.3

Paragraph 3.20

The Australian Renewable Energy Agency continues the integration of its assessment and grant management processes with its information systems to strengthen its assurance over its grant management activities.

Australian Renewable Energy Agency response: Agreed.

Recommendation no.4

Paragraph 3.31

The Australian Renewable Energy Agency improves its funding agreement variation policies and processes to ensure appropriate consideration of value for money, merit against grant program criteria and probity when making decisions on variations.

Australian Renewable Energy Agency response: Agreed.

Recommendation no.5

Paragraph 4.11

The Australian Renewable Energy Agency implements policies and processes to provide effective assurance that its performance measurement framework and reporting is fully consistent with the Commonwealth performance framework.

Australian Renewable Energy Agency response: Agreed.

Recommendation no.6

Paragraph 4.39

The Australian Renewable Energy Agency assesses and actively manages conflicts of interests of organisations engaged to conduct its evaluations and disclose, where relevant, any conflicts in evaluation reports and material provided to decision-makers.

Australian Renewable Energy Agency response: Agreed.

Summary of entity response

Australian Renewable Energy Agency

It is pleasing that the ANAO has concluded that ARENA’s management of its grants programs is largely effective, that grant project selection is aligned with ARENA’s objectives, and that management of grant funding agreements is largely effective. ARENA agrees to all recommendations and has commenced implementation.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit that may be relevant for the operations of other Australian Government entities.

Governance and risk management

Grant programs

Performance and impact measurement

1. Background

Introduction

Renewable energy in Australia

1.1 In 1992 the Australian Government ratified the United Nations Framework Convention on Climate Change (UNFCCC). The objective of the UNFCCC framework is the ‘stabilisation of greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic interference with the climate system’.6 Under the framework, Australia set emissions reduction targets for the two Kyoto Protocol periods (2008–12 and 2013–20) and for 2030 under the Paris Agreement.7

1.2 Successive governments have taken a range of actions to support the development of Australia’s renewable energy industry8 and meet emissions reduction targets. These actions include the establishment of the:

  • Renewable Energy Target in 2001 to encourage additional generation of electricity from sustainable and renewable resources and reduce greenhouse gas emissions in the electricity sector9;
  • Clean Energy Finance Corporation (CEFC)10 and the Australian Renewable Energy Agency (ARENA) in 2012 to support investment in renewable technologies;
  • Emissions Reduction Fund in November 2014, a voluntary scheme that provides incentives for new practices and technologies that reduce emissions11; and
  • Safeguard Mechanism in July 2016 to provide a framework for large emitters to measure, report and manage their greenhouse gas emissions.

Australian Renewable Energy Agency

1.3 ARENA is a corporate Commonwealth entity within the Industry, Science, Energy and Resources Portfolio.12 It is established under the Australian Renewable Energy Agency Act 2011 (ARENA Act). The ARENA Act’s objectives are to:

  • improve the competitiveness of renewable energy technologies; and
  • increase the supply of renewable energy in Australia.13

1.4 The ARENA Act provides ARENA with over $2.2 billion of Australian Government funding from 2012–13 to 2021–22.14 ARENA’s functions under the Act include15;

  • providing financial assistance16 for research, development, demonstration, commercialisation or deployment of renewable energy technologies, or the storage and sharing of information and knowledge about renewable energy technologies; and
  • collecting, analysing, interpreting and disseminating information and knowledge relating to renewable energy technologies and projects.

1.5 Australian businesses, research institutions, Commonwealth entities, state or territory owned corporations or their subsidiaries and local government or councils are eligible for ARENA funding. ARENA classifies its funding responsibilities on an ‘innovation chain’ that is measured using technological and commercial readiness scales. ARENA’s role in the renewable energy sector with respect to other funding bodies and delivery partners is depicted in Figure 1.1.

Figure 1.1: ARENA’s role in the renewable energy ‘innovation’ chain

This figure shows ARENA’s concept of the renewable energy innovation chain. It also presents the funding bodies, policy mechanisms, project proponents and partners that ARENA identifies as operating across this innovation chain.   The renewable energy innovation chain has six stages: research; development; demonstration; deployment; market accumulation; and diffusion. Research, development and demonstration activities are classified on a nine point technology readiness scale. Demonstration and deployment activities are classified on a six point commercial readiness scale. Technology readiness levels seven, eight and nine align with commercial readiness scales one, two and three.  ARENA’s activities occur from the research stage at technology readiness level two through to the deployment stage at commercial readiness level five. At January 2020, ARENA had committed $313 million to research and development grants, $657 million to demonstration grants and $578 million to deployment grants. A further $121 million had been committed to studies that do not directly align to renewable energy innovation chain stages.  The Australian Research Council’s activities precede ARENA’s activities at the research stage. The funding bodies and policy mechanisms described in paragraph 1.2 have roles after ARENA in the deployment, market accumulation and diffusion stages.   The organisations described at paragraph 1.5 are shown in this figure as project partners and proponents. Universities are active at the research and development stages. The other partners and proponents are active across ARENA’s renewable energy innovation chain.

Note a: These figures are ARENA funding for approved applications up to January 2020. ARENA has also approved applications of $121 million for grants it classifies as studies. These studies cut across the innovation chain categories shown in this figure.

Source: ANAO analysis from ARENA documents.

Funding co-contributions

1.6 ARENA considers the amount of co-contribution from project proponents and partners as part of the merit assessment process. ARENA’s reported breakdown of the contributions from other organisations involved in ARENA funded projects is presented in Table 1.1.

Table 1.1: Committed co-contributions as at 31 December 2019

 

Study

Research and Development

Demonstration

Deployment

For each $1 of ARENA funding

$1.56

$1.77

$1.84

$4.75

         

Source: ARENA, ARENA at a glance — 30 December 2019.

Types of renewable energy technology funded

1.7 ARENA’s reported breakdown of funding commmitments by renewable energy technology type or focus at 30 June 2019 is presented in Figure 1.2. Solar photovoltaic (PV) related projects have received the largest funding commitments.

Figure 1.2: ARENA funding commmitments by technology type 2012–19, at 30 June 2019

 

Source: ARENA, Annual Report 2018–19, Table 2.

Grant program and project funding

1.8 A breakdown of ARENA’s approved grant applications by year as at January 2020 is provided in Table 1.2.

Table 1.2: Approved applications and funding by financial year as at January 2020

 

<2011–2012a

2012–2013a

2013–2014

2014–2015

2015–2016

2016–2017

2017–2018

2018–2019

2019–2020c

Total

Applications approved

92

96

14

48

30

35

87

89

47

538

Funding requested ($million)b

236.7

443.1

19.2

200.9

66.8

120.8

205.1

215.7

160.9

1,669.5

                     

Note a: When ARENA was established it became responsible for a number of existing Commonwealth renewable energy related grants programs. ARENA managed all aspects of these grant programs after 1 July 2012.

Note b: This is the amount of funding requested in the application, not the final amount of funding expended.

Note c: These figures are current at January 2020.

Source: ANAO analysis of ARENA data.

1.9 ARENA utilises two grant funding processes: continually open application processes and targeted, competitive funding rounds. Applications to the continually open application processes are individually assessed against the merit criteria for the program. Targeted competitive rounds fund the highest merit applications up to the available funding allocated by ARENA to the round. The majority of funding has been awarded through continually open rounds (Figure 1.3).

Figure 1.3: Funding awarded through open and competitive processes at January 2020

 

Source: ANAO analysis of ARENA data.

Recent grant programs

1.10 Since 2016–17 ARENA has awarded the majority of grant funding through the Advancing Renewables Program (Table 1.3). This program provides grants through both open processes and competitive rounds. Funded activities include the development, demonstration, deployment or commercialisation of a renewable energy technology, including feasibility studies, pilot projects and demonstration projects.

Table 1.3: Proportion of Advancing Renewables Program projects and funding

 

 

2012–2013

2013–2014

2014–2015

2015–2016

2016–2017

2017–2018

2018–2019

2019–2020

Total

Advancing Renewables Program

Projects

3

21

55

72

42

193

Funding ($million)

1.3

120.1

171.1

192.7

160.4

645.7

Other grant programs

Projects

96

14

48

27

2

32

17

5

333

Funding ($million)

443.1

19.2

200.9

65.4

0.8

33.9

23.1

0.5

787.1

                     

Source: ANAO analysis of ARENA data provided in January 2020.

Governance and staffing arrangements

1.11 The ARENA Board is the accountable authority under the Public Governance, Performance and Accountability Act 2013 (PGPA Act).17 It is responsible for ensuring that ARENA complies with the ARENA Act and setting ARENA’s strategies and policies.18

1.12 Under the ARENA Act, ARENA can only employ two staff members — the Chief Executive Officer (CEO) and the Chief Financial Officer (CFO).19 Other staff are required to be provided by the Department of Industry, Science, Energy and Resources (the department).20 ARENA may also directly engage consultants to provide technical and specialist advisory services21, and use labour hire firms. ARENA’s total full time equivalent (FTE) at 30 June 2019 was 98.3 —the CEO, CFO, 28 staff provided by the Department of the Environment and Energy, and the remainder engaged as contractors and consultants.

1.13 Restrictions under the ARENA Act that limit ARENA’s ability to directly employ workers, combined with reducing departmental staff allocations over time, has led to what ARENA considers to be a heavy reliance on contractor (labour hire) and consultancy arrangements. ARENA has noted that the staffing model is of greater cost to the Commonwealth than could otherwise be the case.

Rationale for undertaking the audit

1.14 Substantial funding has been committed to renewable energy activities through ARENA. As ARENA approaches the end of its legislated funding, it is important to provide the Parliament with assurance over the effectiveness of ARENA’s grants management in improving the competitiveness of renewable energy technologies and increasing the supply of renewable energy in Australia.

Audit approach

Audit objective, criteria and scope

1.15 The objective of the audit was to assess the effectiveness of grant program management by ARENA. To form a conclusion against this objective, the ANAO developed the following high-level criteria:

  • Does grant selection support the achievement of ARENA’s objectives?
  • Are grant funding agreements managed effectively?
  • Does evaluation of grant programs indicate that ARENA is achieving its objectives?

1.16 The audit scope focused on ARENA’s:

  • strategic planning;
  • design of grant program guidelines;
  • application assessment and selection processes;
  • grant funding agreement management; and
  • evaluation and reporting of grants programs and organisational performance.

1.17 The scope of the audit did not include:

  • the $200 million Clean Energy Innovation Fund which is financed and primarily managed by the CEFC, with subject matter expertise and contract management assistance provided by ARENA22; and
  • ARENA’s legislated functions that do not directly relate to grants management, such as providing advice to the Minister and liaising with state and territory governments and other authorities.

Audit methodology

1.18 Audit procedures included:

  • examination of ARENA documentation;
  • analysis of grant management system data;
  • sample testing of application assessments and funding agreement management documentation;
  • observation of grants assessment processes;
  • field visits to projects funded by ARENA;
  • consideration of public contributions to the ANAO; and
  • interviews with ARENA staff.

1.19 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $489,600.

1.20 The team members for this audit were Joshua Francis, Kate Wilson, Isaac Gravolin, Taela Edwards, Sara Casey, Mark Rodrigues, and Michael White.

2. Strategic plans and application assessment

Areas examined

This Chapter examines if the Australian Renewable Energy Agency’s (ARENA’s) grant project selection supports the achievement of ARENA’s objectives by assessing ARENA’s strategic plans, grant guidelines and grant assessment and selection processes. ARENA’s performance measurement framework is also assessed.

Conclusion

Strategic planning and grant project selection largely aligns with ARENA’s objectives. ARENA’s performance measurement framework does not provide a reliable basis to demonstrate to the Parliament and the public that ARENA is achieving its objectives.

Areas for improvement

The ANAO made two recommendations designed to improve ARENA’s performance measurement framework and strengthen its assessment of the additionality of projects.

The ANAO also suggested that ARENA: improve the consistency of its key strategic documents; outline the circumstances under which requirements may be waived in its grant guidelines; and clearly document the resolution of funding conditions in its records of decision.

2.1 To effectively achieve its legislated objectives of increasing the supply and competiveness of renewable energy, ARENA’s grant selection needs to fund activities that support those objectives. The following criteria were used to assess if ARENA’s grant selection supports the achievement of its objectives:

  • Are strategic plans clear and consistent with ARENA’s objectives?
  • Are grant program guidelines appropriate?
  • Are grant application assessments and selections consistent with grant guidelines and frameworks?

Are strategic plans clear and consistent with ARENA’s objectives?

In 2018–19 ARENA’s corporate plan, general funding strategy, annual work plan and investment plan were mostly clear and consistent with ARENA’s objectives. The performance framework as set out in the corporate plan did not provide a reliable basis to demonstrate to the Parliament and the public that ARENA is achieving its objectives.

2.2 In 2018–19 ARENA had four key strategic planning documents, three of which are required by legislation (Table 2.1).

Table 2.1: ARENA’s 2018–19 strategic planning documents

Planning document

Legislation

Published

Purpose

ARENA Corporate Plan 2018/19 – 2021/22

PGPA Act section 35

Yes

Sets out the entity’s purposes; what the entity will do to achieve its purposes (activities); and how the entity will know that it has achieved its purposes (intended outcomes).a

ARENA General Funding Strategy 2018/19 – 2021/22

ARENA Act section 19

Yes

States ARENA’s principal objectives and priorities for the provision of financial assistance under the ARENA Act.b

ARENA Business Plan 2018-19

ARENA Act section 27

No

This included the annual work plan (required under the ARENA Act) for setting out how the general funding strategy is proposed to be implemented during the year and the main activities proposed to be undertaken.b

Innovating Energy: ARENA’s Investment Plan 2017

None

Yes

Provides details on the general funding strategy’s four investment priorities and general information that potential applicants should consider before approaching ARENA for funding.c

    

Note a: Department of Finance, Corporate plans for Commonwealth entities — Resource Management Guide No. 132, January 2017.

Note b: Paragraph 4; ARENA Act subsections 19(3) and 27(3).

Note c: Innovating Energy: ARENA’s Investment Plan 2017, p. 5. This investment plan remained in force until 2019.

Source: ANAO analysis.

The corporate plan

2.3 The Commonwealth performance framework sets out that the corporate plan is designed to be an entity’s primary planning document.23 Corporate plans are developed at the beginning of the reporting cycle to set out an entity’s strategies for achieving its purposes and how success will be measured.24

2.4 The Public Governance, Performance and Accountability Rule 2014 (PGPA Rule) section 16E sets out the minimum content requirements for all Commonwealth entities’ corporate plans. The ARENA Corporate Plan 2018/19 – 2021/22 includes all required elements as shown in Table 2.2. However, as discussed at paragraphs 2.9 to 2.25 below, issues have been identified with the presentation of the 2018–19 performance framework.

Table 2.2: ARENA 2018–19 corporate plan content requirements

Minimum requirements

Content included?

Rating

Statement of preparation and period of coverage

Purposes

Environment

Performance

Capability

Risk oversight and management

   

Legend:  Meets minimum requirements Information presented but improvements required

Source: ANAO analysis.

ARENA’s purpose statement

2.5 Under the PGPA Act, the purposes of a Commonwealth entity are to include the objectives, functions or role of the entity. Entities should also consider existing authoritative documents when defining their purposes, such as enabling legislation as passed by the parliament.25

2.6 A concise and measurable statement of the purposes of an entity underpins a robust performance reporting framework.26 A clear and concise purpose statement clearly states:

  • what the entity aims to achieve;
  • how the entity will do this; and
  • to what effect, that is, the impact achieved when an entity successfully delivers its purposes.27

2.7 The ARENA Corporate Plan 2018/19 – 2021/22 states that ARENA’s purpose is ‘to accelerate Australia’s shift to affordable and reliable renewable energy’.28 This purpose statement is readily identifiable, concise and easily understood.

2.8 However, the purpose does not explicitly address all of the objectives and functions of the ARENA Act.29 In addition, while the purpose statement clearly states what ARENA aims to achieve, the purpose would benefit from including how it will achieve this aim and the expected effect on Australians once this is achieved.30

Do the activities adequately address the purpose?

2.9 Under the Commonwealth performance framework31, the description of purposes and activities in the corporate plan is the foundation for developing meaningful performance information. The corporate plan should focus on the high-level activities that will provide a reader some insight and understanding of how the purpose will be achieved. Each activity should be explicitly linked to a purpose and specify the contribution it makes to achieving an entity’s purposes.32

2.10 ARENA’s 2018–19 performance framework is made up of its purpose, activities, outcomes, performance measures and impacts. A summary is presented in Table 2.3. ARENA has taken a similar approach to presenting the information in 2019–20.

Table 2.3: Summary of ARENA Corporate Plan 2018–19 performance framework

Purpose

Activities

Outcomes (Results)a

Impactsb

To accelerate Australia’s shift to affordable and reliable renewable energy

  • Provide financial assistance and leverage private investment
    • 5 activity performance measures, 2 with targets.
  • More solutions for delivering secure, reliable and affordable energy and increasing the value provided by renewable energy
    • 2 outcome or result performance measures, none with targets.
  • Improved competitiveness of renewable energy technologies
    • 1 outcome or result performance measure with no target.
  • Increase in supply of renewable energy
  • Ongoing improvements in the competitiveness of renewable energy technologies
  • Secure, reliable and affordable electricity system with a significantly higher share of renewable energy
  • Increase in employment in renewable energy activities
  • Improvements in energy productivity enabling achievement or exceeding of targets in the National Energy Productivity Plan
  • Commercial scale export value chains in renewable energy established
  • Administer financial assistance
    • 3 activity performance measures, 0 with targets.
  • Jointly manage Clean Energy Innovation Fund with the Clean Energy Finance Corporation
    • 3 activity performance measures, 3 with targets.
  • Provide information, advice and knowledge to advance renewable energy
    • 3 activity performance measures, 0 with targets.
  • Industry and government better informed to navigate the energy transition
    • 3 outcome or result performance measures, 0 with targets.
  • Collaborate with, and facilitate collaboration between, other persons, organisations and governments (including international collaborations)
    • 2 activity performance measures, 0 with targets.
  • Increased collaboration on energy innovation
    • 1 outcome or result performance measure with no target.
    

Note a: ARENA has included the results of their activities within the outcomes section.

Note b: The impacts were not identified by ARENA as performance measures.

Source: ANAO analysis of the ARENA Corporate Plan 2018/19 – 2021/22, p. 15.

2.11 The activities described in the 2018–19 corporate plan’s performance framework mostly support appropriate performance reporting. The activities could be more clearly aligned to ARENA’s purpose statement, as there is insufficient context for the reader to align the activities to the ‘affordable’ and ‘reliable’ elements of the purpose. This should be considered in the context of the suggested improvements to the purpose statement (see paragraph 2.8).

Are the performance measures appropriate?

2.12 In order to allow entities to assess whether performance measures are appropriate, the Department of Finance has provided guidance to entities on the characteristics of ‘good’ performance information — relevant, reliable and complete.33 The ANAO has drawn on the Department of Finance’s guidance and other sources to develop audit criteria to assess the appropriateness of performance information.34 These criteria can be found at Appendix 2 and have been used in each of the ANAO’s audits of entities’ performance statements to date.

2.13 This assessment examined the 23 performance measures defined in the ARENA Corporate Plan 2018–19 and ARENA’s two performance measures in the Environment and Energy Portfolio Budget Statements for the same year.35

2.14 In addition to these performance measures, ARENA’s 2018–19 performance framework identified six impacts and four outcomes.36 Under the Commonwealth performance framework, these impacts and outcomes meet the definition of performance measures.37 In the annual report, ARENA also reported directly against two of six of these impacts and all four of the outcomes using case studies. Therefore, these six impact and four outcome statements are also included in this assessment.

Relevance

2.15 A relevant performance measure allows readers to assess an entity’s progress in fulfilling its purpose. Relevant performance measures:

  • clearly indicate who benefits and how they benefit from the entity’s activities;
  • are focused on a significant aspect of the entity’s purpose, via the activities, and makes the attribution of the entity’s activities clear; and
  • are understandable, providing sufficient information in a clear and concise manner.38

2.16 ARENA’s performance measures are either relevant or mostly relevant. The relevance assessment of ARENA’s 2018–19 performance measures is presented in Table 2.4. The most consistent strengths of the performance measures is that they are understandable and use concise, plain English. The relevance of these performance measures would be improved by clearly and consistently:

  • indicating who will benefit and how they will benefit from the achievement of the performance measures;
  • articulating how each individual measure will contribute to the achievement of ARENA’s purpose; and
  • clarifying the attribution of results, including acknowledging when a result would be significantly impacted by factors beyond the direct effect of ARENA’s activities.

Table 2.4: Relevance assessment of ARENA’s 2018–19 performance measures

Characteristica

Yes

Mostly

Partly

No

Total

Relevance — Benefit

10

17

0

8

35

Relevance — Focus

5

18

10

2

35

Relevance — Understandable

20

7

5

3

35

      

Note a: See Appendix 2 for a description of each characteristic.

Source: ANAO analysis.

Reliability

2.17 A reliable performance measure allows readers to form clear expectations and consistently assess an entity’s progress in fulfilling its purpose. Reliable performance measures are:

  • measurable — they use and disclose information sources and methodologies (including a basis or baseline for measurement or assessment, for example a target or benchmark) that were fit-for-purpose; and
  • free from bias — they allow for clear interpretation and an objective basis for assessment of the results.39

2.18 The 2018–19 performance measures are only partly reliable, or not reliable because the reader cannot independently interpret the results reported — the reader must rely on ARENA to tell them what constitutes ‘good’ performance (Table 2.5). This was because most of the measures did not:

  • specify a target or baseline; and/or
  • did not provide a clear measurement method and information source.

Table 2.5: Reliability assessment of ARENA’s 2018–19 performance measures

Characteristica

Yes

Mostly

Partly

No

Total

Reliability — Measurable

2

4

17

12

35

Reliability — Free from bias

1

2

2

30

35

      

Note a: See Appendix 2 for a description of each characteristic.

Source: ANAO analysis.

Completeness

2.19 A complete set of performance criteria allow for the overall assessment of an entity’s progress in fulfilling its purpose. A complete set of performance measures are:

  • balanced — they provide a basis for assessment of both the efficiency and effectiveness of the entity in fulfilling its purpose, rely on a mixture of quantitative and qualitative measurement bases and assess a mixture of short, medium and long-term objectives; and
  • collective — collectively address the entity’s purpose through the activities identified in the corporate plan.40

2.20 In order to support the Parliament and the public in assessing how well an entity is performing, including how they are using the resources that have been entrusted to them, entities should identify a set of measures that demonstrate both efficiency and effectiveness.41 ARENA’s 2018–19 performance measures did not include any efficiency measures.

2.21 ARENA’s 2018–19 performance measures did present an appropriate balance of qualitative and quantitative information.

2.22 Accountability performance information (the highest level of the performance information hierarchy) is information that reports on outcomes and impacts to demonstrate whether public resources are delivering on government objectives.42 Performance reporting on accountability information is the focus of the PGPA Act.43 While lower levels of performance information are also important, they should be used to support and advance accountability information.44

2.23 Of ARENA’s 35 performance measures in 2018–19, 29 per cent are accountability information, 60 per cent are tactical/strategic information and 11 per cent are management information.

2.24 ARENA’s 2018–19 performance measures collectively address ARENA’s 2018–19 purpose to ‘accelerate Australia’s shift to affordable and reliable renewable energy.’ However, the ARENA 2018–19 Annual Report did not report against four of the six impacts and therefore did not present a complete picture of the extent to which ARENA is achieving its purpose.

2.25 Overall, improvements to ARENA’s performance measurement framework are required to provide the Parliament and the public meaningful performance information with a clear line of sight between planned and actual performance. In January 2020, following preliminary findings from the ANAO, ARENA commenced a process to improve its performance measurement framework.

Recommendation no.1

2.26 The Australian Renewable Energy Agency improves the reliability and completeness of its performance measurement and reporting framework.

Australian Renewable Energy Agency response: Agreed.

2.27 ARENA revised its Performance Framework during the course of the audit and proposes to include it in its republished Corporate Plan.

Alignment of the PGPA Act and ARENA Act planning documents

2.28 To achieve a shared understanding within ARENA and among stakeholders of what ARENA will focus on in pursuit of its objectives, additional strategic planning documents should be aligned to the corporate plan.

2.29 The documents are mostly consistent in identifying ARENA’s purpose, impacts, focus areas and investment priorities. However, while the focus areas and investment priorities (key elements of the ARENA Act documents) are stated in the corporate plan, they are not integrated into or covered by the performance framework. The ‘guiding principles for ARENA investment’, one of the two key elements of the ARENA General Funding Strategy 2018/19 – 2020/21, are not aligned to any of the key elements of the other strategic planning documents.

2.30 While it is not expected that all documents will present precisely the same information, to be meaningful to the user it is expected that they would be built around and expand on the same key concepts. The strategic documents do not consistently present ARENA’s outcomes (Table 2.6) and activities (Table 2.7).

Table 2.6: Presentation of outcomes across the strategic planning documents

Corporate plan outcomesa

Investment plan outcomesa

ANAO assessment

  • More solutions for delivering secure, reliable and affordable energy
  • More affordable, higher value renewable energy sources
  • More options for secure and reliable electricity supply

  • Improved competitiveness of renewable energy technologies

  • Industry and government better informed to navigate the energy transition
  • Increased skills and knowledge

  • Increased collaboration on energy innovation

  • Data to inform new business models and regulatory changes

  • Innovative ways to improve energy productivity

   

Legend:  appropriately aligned partially aligned not aligned – no comparable item

Note a: ‘Outcomes’ did not appear in any of the other strategic documents. The outcome statement in the PBS was not considered as part of this assessment, as under the Commonwealth performance framework it is intended to be a higher-level, purpose-style statement.

Source: ANAO analysis of ARENA Corporate Plan 2018/19 – 2020/21 and Innovating Energy: ARENA’s Investment Plan 2017.

Table 2.7: Presentation of activities across the strategic planning documents

Corporate plan activitiesa

Annual work plan ‘Categories’ and ‘business focus areas’ that activities are arranged undera

ANAO assessment

  • Provide financial assistance and leverage private investment
  • Administer financial assistance
  • Jointly manage Clean Energy Innovation Fund with the Clean Energy Finance Corporation
  • Making new investments

  • Knowledge sharing and provision of advice to inform decision-making
  • Maximising benefits from existing investments (includes a lower-level knowledge sharing activity)

  • Facilitating collaboration within the energy sector

  • Organisational improvement

   

Legend:  appropriately aligned partially aligned not aligned – no comparable item

Note a: ‘Activities’ did not appear in any of the other strategic documents.

Source: ANAO analysis of ARENA Corporate Plan 2018/19 – 2020/21 and Annual Business Plan 2018–19.

2.31 A shared understanding within ARENA and among stakeholders of what ARENA will focus on in pursuit of its purpose may be inhibited because the key planning documents are not fully aligned. There is merit in ARENA reviewing and updating these documents to improve the consistency in its messages.

Are grant guidelines appropriate and aligned with strategic plans?

ARENA’s grant guidelines are appropriate and aligned with strategic plans except in relation to describing its assessment of additionality to determine whether proposed projects would achieve outcomes that would not otherwise occur without public funding. Clearly outlining how it assesses additionality would assist ARENA to demonstrate value for money in decision-making and avoid funding activities that would proceed without ARENA support.

Are grant program guidelines appropriate?

Commonwealth Grants Rules and Guidelines and internal grant framework and policies

2.32 The Commonwealth Grants Rules and Guidelines (CGRGs) are issued by the Finance Minister under the PGPA Act and establish the expectations for all non-corporate Commonwealth entities in relation to grants administration. While ARENA is a corporate Commonwealth entity, ARENA’s Internal Governance and Control Framework states:

Although ARENA is not subject to the Commonwealth Grant Rules and Guidelines (CGRGs) or Commonwealth Procurement Rules (CPRs), wherever possible ARENA has developed its frameworks, policies and plans utilising the core principles contained in these documents to ensure consistency with recognised Commonwealth best practice.45

2.33 ARENA’s Financial Assistance Manual outlines the rules, procedures and good practices its officials are to apply when administering its grants.46 The manual is structured around ARENA’s four main stages of program delivery. These are the:

  • design stage — development and approval of grant program guidelines;
  • selection stage — identification of funding recipients and the execution of funding agreements with successful applicants;
  • delivery stage — management of funding agreements; and
  • evaluation stage — assessment of the performance of its grant projects and programs.

2.34 The manual is supported by other policy and procedure documents, including grant administration specific documents such as the Standard Operating Procedure Assessment Process and the ARENA Conflict of Interest Policy. The financial assistance manual and other grants related policy and procedure documents refer ARENA officials to the grant program guidelines.

Consistency with Commonwealth Grants Rules and Guidelines

2.35 The majority of ARENA’s funding since 2016–17 has been committed though the Advancing Renewables Program (see paragraph 1.10 and Table 1.3). The Advancing Renewables Program guidelines were tested for consistency against key guideline related elements of the CGRGs.

2.36 Consistent with the CGRGs, the program’s guidelines and funding agreement template:

  • contain straightforward and easily understood eligibility and merit criteria;
  • specify the weighting of assessment criteria;
  • specify the grant application approval processes;
  • inform applicants of terms and conditions they will need to meet during the life of the grant such as performance and financial reporting;
  • describe complaint handling and review mechanisms;
  • are revised where significant changes have been made to a grant opportunity;
  • outline conflict of interest policies; and
  • provide for consideration of the risk associated with proposed projects.
Value for money — assessing additionality

2.37 A key principle of the CGRGs is that achieving value for money should be a prime consideration in all phases of grants administration.47 The Advancing Renewables Program guidelines state that ‘once a proposal has been assessed to have satisfied the eligibility criteria, it will be assessed for overall value for money against the merit criteria.’48 The five merit criteria are:

  1. Contributes to the Program Outcomes;
  2. Applicant capability and capacity;
  3. Activity design, methodology, risk and compliance;
  4. Financial viability and co-funding commitment; and
  5. Knowledge sharing.49

    2.38 For each merit criterion, the guidelines provide examples of the types of information that applicants can provide to demonstrate merit. These information types map to most of the value for money principles set out in the CGRGs.

    2.39 The CGRGs also note that guidelines should support an assessment of the additionality of applications: ‘a fundamental appraisal criterion is that a grant should add value by achieving something worthwhile that would not occur without the grant’.50 The assessment of program additionality has been a feature in Commonwealth grant programs51 and in other programs delivered by corporate Commonwealth entities.52

    2.40 In the context of providing financial assistance to a growing industry, there is a risk that ARENA may support projects that might have proceeded without public funding. In November 2019, 11 per cent of ARENA funding recipients (that responded to market research commissioned by ARENA) indicated that their project would have been successful without ARENA’s involvement.53

    2.41 The ARENA General Funding Strategy 2018/19 – 2020/21 outlines the principles that:

    • ARENA does not fund activities that are commercially viable without ARENA support; and
    • ARENA seeks to provide the minimum financial assistance needed, and in the most appropriate form, to deliver the desired outcome, thus maximising value for public investment.54

    2.42 However, the Advancing Renewables Program guidelines do not describe how applications will be assessed for additionality, except in relation to the knowledge sharing activities related to projects. The guidelines do not reference the general funding strategy’s principles as a standard that will be used in the merit assessment. There are no other descriptions of how ARENA assesses if the grant will add value by achieving something worthwhile that would not occur without the grant.

    Recommendation no.2

    2.43 The Australian Renewable Energy Agency clearly outlines in its grant guidelines how it assesses the additionality of project applications so that it can better demonstrate that the projects it funds would not have proceeded without public funding.

    Australian Renewable Energy Agency response: Agreed.

    2.44 The Guidelines will be amended to make an express reference to additionality and put to the Minister for approval.

    Are grant programs aligned with strategic plans?

    Grant program guidelines design and approval

    2.45 As noted in Chapter 1, paragraph 1.9, ARENA has provided funding through two distinct grant processes: targeted competitive funding rounds and a continually open process (non-competitive funding). The majority of ARENA’s funding has been awarded through open processes (Figure 1.3).

    2.46 The ARENA Corporate Plan 2018/19 – 2020/21 describes the expected benefits for each application process:

    • targeted funding rounds are intended to create competitive tension, support price discovery and allow the pursuit of specific outcomes in focus areas; and
    • continually open processes are intended to provide applicants with a funding mechanism that best meets their needs and the needs of the market.55

    2.47 ARENA’s continually open process has been run through the Advancing Renewables Program since 2016 and its predecessor the Emerging Renewables Program between 2012 and 2016.56 Prior to 2016, ARENA also ran a number of other open grant processes under different sets of grants guidelines.

    2.48 ARENA runs competitive processes when it identifies the circumstances to achieve value for money through a competitive process. This included the four Research and Development program competitive rounds and five competitive processes run through the Advancing Renewables Program guidelines. ARENA’s changing investment priorities over time, with reference to large scale solar projects, are described in Case study 1.

    Case study 1. ARENA’s changing investment priorities: large scale solar

    Large scale solar demonstration and deployment projects have been the subject of ARENA’s changing funding priorities since 2013. This case study illustrates the changes to ARENA’s funding priorities with respect to large scale solar.

    2013

    ARENA’s Investment Plan 2013/14 – 2015/1657 identified deploying utility scale renewable energy as a strategic initiative. In 2013 ARENA provided financial assistance to a series of large scale solar projects under this initiative. These included grants of $166.7 million AGL Solar Project58 and the $101.7 million to the Moree Solar Farm.59 Both of these projects are complete with the solar farms operational and knowledge sharing material on the ARENA website.

    2015

    The investment plan issued in July 2015 set out that ARENA would run a competitive round designed to reduce the gap in commercial competitiveness between large-scale solar photovoltaics and wind generation.60 Under this round, ARENA committed $90 million of grant funding towards 12 projects. Eight of these projects also received a total of $320 million debt finance from the Clean Energy Finance Corporation.61 An evaluation of this funding round identified that there has been rapid growth of the large scale solar industry since the competitive round.

    2019

    ARENA’s Innovating Energy Investment Plan 2019 contains three investment priorities: integrating renewables into the electricity system; supporting industry to reduce emissions; and accelerating hydrogen.62 The 2019 investment priorities no longer have a specific reference to large scale solar energy.

    2.49 The CGRGs state that it is best practice for officials to identify the most appropriate application processes for achieving value for money. Both competitive and open grant processes are identified as options in the CGRGs. Documenting the rationale for not using competitive merit-based application processes when grant guidelines are approved is considered best practice under the CGRGs.63 Consistent with the CGRGs and the ARENA Act, ARENA:

    • documented the rationale for the Advancing Renewables Program as a continually open program by referencing the Emerging Renewables Program’s design in a paper to the ARENA Board; and
    • gained the approval of the Minister for Industry and Science for the Advancing Renewables Program guidelines to use a continually open program in July 2015.64

    2.50 Given the high proportion of its funding that is awarded through open application processes, it is important that the ARENA Board continually assures itself that this approach is delivering value for money.

    Investment priorities and alignment with grant program guidelines

    2.51 ARENA considers that the updating of its investment priorities to favour new innovation over the areas it has funded in the past is an important element of ensuring value for money through its open grants processes. Since its establishment in 2012, ARENA has published five updates to its general funding strategy and four updates to its investment plan. These updates have seen projects that would have been considered of high merit under the investment priorities in 2012–13 no longer be a focus under the 2019 investment priorities.

    2.52 The reliance on the investment priorities in ARENA’s strategic plans requires alignment between strategic planning documents and grant guidelines, which was demonstrated by:

    • program and round objectives either mapping to or directly aligned with key elements of ARENA’s strategic planning documents; and
    • the merit assessment criteria outlined in each of the grant program guidelines appropriately aligning with their respective program objectives.

    Are grant application assessments and selections consistent with grant guidelines and frameworks?

    Individual application assessments examined by the ANAO were generally consistent with grant guidelines and ARENA’s internal assessment framework. ARENA would be able to provide greater assurance over its grant selection by improving its use of information systems and standardising its assessment and decision-making processes, and record keeping.

    2.53 ARENA’s grant program guidelines set out the application assessment processes, criteria and requirements for applicants. In assessing the applications, ARENA’s staff are guided by an internal assessment framework made up of the policies, procedures, templates, systems and controls.

    Application assessment process

    2.54 ARENA uses two different processes to assess all applications: a one-stage process that involves the assessment of a ‘full application’ and a two-stage process that involves an ‘expression of interest’ followed by a ‘full application’.65 Applications to the Advancing Renewables Program seeking more than $500,000 are to be assessed through the two-stage process, unless this requirement is waived.66 An overview of the assessment process is provided in Figure 2.1.

    Figure 2.1: Overview of ARENA application assessment process

    This figure is described in paragraphs 2.54 and 2.55.

    Note: Steps in green are the decision points covered by ANAO’s sample testing.

    Source: ANAO analysis from ARENA documentation.

    2.55 The ANAO identified four key decision points in the full application assessment process:

    • eligibility assessment undertaken by ARENA staff against the grant guidelines eligibility criteria and approved by ARENA manger or executive;
    • merit assessment undertaken by ARENA’s advisory panel against the grant guidelines merit criteria67;
    • funding decision by the board or the board’s delegate to enter into funding agreement negotiations, informed by the merit assessment report and supporting analysis from ARENA; and
    • execution decision for ARENA to enter into the grant funding agreement by the board’s delegate. This is the final approval in the application assessment process.

    2.56 As at 23 July 2019 ARENA had completed assessment of 992 potential projects68, with 28 per cent successful in becoming funded projects with executed funding agreements (Table 2.8).

    Table 2.8: Potential Project Assessment Outcomes

    Ineligible

    EoI not successful

    EoI approved, no full application submitted

    Full application not successful

    Full application successful, funding agreement not executed

    Funding agreement executed

    Total

    30 (3%)

    404 (41%)

    59 (6%)

    190 (19%)

    36 (4%)

    273 (28%)

    992

           

    Source: ANAO analysis of ARENA Grants Management System data extracted 23 July 2019.

    Application assessments

    2.57 The ANAO reviewed decision records of a sample of applications against key requirements identified from the guidelines and ARENA’s internal assessment framework. These key requirements apply to the decisions at the eligibility assessment, merit assessment, funding and execution decision steps in the full application assessment process (green boxes in Figure 2.1).69

    2.58 ARENA’s assessment and selection processes were generally implemented in accordance with its grant guidelines and assessment framework, as:

    • eligibility assessments were completed;
    • the correct decision-maker70 was engaged at each key stage;
    • applications assessed as ‘high’ merit were funded;
    • consideration of risk was documented; and
    • consideration of value for money was generally appropriate, except for additionality.
    Consideration of additionality

    2.59 ARENA’s grant guidelines do not adequately describe how it will assess the additionality of project applications so that it does not fund projects that would proceed without ARENA support (see paragraphs 2.39–2.42). In assessing how additionality is presented to the ARENA decision-maker as part of the broader value for money merit assessments for the 44 sampled applications, it was identified that:

    • all included evidence of consideration of additionality with respect to knowledge sharing;
    • the consideration of additionality from a financial perspective was not as clearly presented to the ARENA decision-maker. Advice on the financial viability of a project without ARENA grant funding was documented in 11 out of 44 application assessments; and
    • for two successful applications the merit assessment report stated the projects may be viable without ARENA funding, and it could not be identified in subsequent decision records how these additionality issues had been addressed.71

    2.60 As well as outlining how it will assess additionality in its grant guidelines, there is merit in ARENA standardising how it documents its consideration of additionality. This will assist ARENA in demonstrating proper use of public funds under the PGPA Act.

    Batched approvals of competitive funding rounds

    2.61 ARENA batches the assessment and approval of the applications for competitive funding rounds. This differs from continuously open Advancing Renewables Program applications, where each individual application has a decision document that covers that specific application.

    2.62 The ANAO’s sample of 44 applications included 18 applications that were part of batched approval processes. Decision documents across the competitive rounds also do not have a consistent approach to the presentation of information to the decision-maker.

    2.63 The application-specific information provided to the decision-maker for batched approvals for competitive funding rounds was often limited to summarised merit assessment information and did not attach individual project merit assessment reports. The format of this summarised information varied across the different approval documents review. It generally included a merit ranked list of all applications with an available funding cut off, with summary project description and summary assessment comments. When compared to decisions on the open Advancing Renewables Program which attached individual project merit assessment reports, this approach provided less application-specific information to the decision-maker.

    Conditions imposed on merit and funding decisions

    2.64 In the merit assessment reports and the funding decision approvals, conditions or caveats can be placed on approvals to be addressed for the application to be awarded funding. In the sample of 44 applications, conditions or caveats could be identified in 15 merit assessment reports and 17 funding decisions. The review of these application assessments found:

    • nine instances where a condition or caveat made in a merit assessment report could not be clearly identified as being addressed in the subsequent funding approval decision documentation; and
    • four instances in execution decision documentation where there was insufficient information to determine if conditions imposed by the decision-maker at the previous funding decision approval had been met.

    2.65 For transparency and clarity, records of decision should clearly and consistently document the recommendations and conditions from previous stages and how these have been addressed.

    Workflow systems

    2.66 ARENA’s ability to provide quality assurance over its application assessment process is limited by its lack of information system-based workflows, standardised templates and electronic record keeping. These limitations are also relevant to ARENA’s grants management processes. Further analysis is presented in paragraphs 3.17–3.19. A related recommendation, Recommendation no.3, is at paragraph 3.20.

    3. Grant management

    Areas examined

    This Chapter examines the effectiveness of the Australian Renewable Energy Agency’s (ARENA’s) management of grant funding agreements, including the monitoring of performance agreements, governance oversight and risk management.

    Conclusion

    ARENA’s management of grant funding agreements is largely effective. Improvements are required to ARENA’s management of variations and its integration of electronic systems with its business processes.

    Areas for improvement

    The ANAO made two potential recommendations aimed at better integrating grant management business processes and information systems and strengthening ARENA’s grant variations framework. The ANAO also suggested that ARENA include performance information requirements in its grant funding template.

    3.1 Effective grant management is required to ensure that ARENA’s grants achieve its objectives with value for money. The following criteria were used to assess the effectiveness of ARENA’s grant management:

    • Are appropriate arrangements in place to monitor the performance of funding agreements?
    • Are governance arrangements for the oversight of grant funding agreements robust?
    • Are risks and issues identified and dealt with appropriately?

    Are appropriate arrangements in place to monitor the performance of funding agreements?

    ARENA’s arrangements for monitoring the performance of individual funding agreements are largely appropriate. ARENA’s ability to provide assurance over the quality of its grants management would be enhanced by improved variation processes and increased integration of its business processes and information systems.

    Grant funding agreements

    3.2 The Commonwealth Grants Rules and Guidelines (CGRGs) and ARENA’s internal grants management framework72 recognise that well-structured and appropriately managed grant funding agreements underpin effective grant management. Well-structured grant funding agreements contain core project management elements such as milestones, outcomes and performance measures. Commonwealth entities that monitor project milestones and take appropriate actions to manage risks to grant outcomes are well-placed to deliver effective grants management.

    Grant agreement elements

    3.3 ARENA’s funding agreement template includes core project management elements such as risk management plans, milestones, deliverables, outcomes, reporting and acquittal arrangements. The funding agreement template does not contain clear guidance to support the development of performance information for inclusion in funding agreements.

    3.4 The CGRGs note that ‘performance reporting requirements and other information sought from grantees are key inputs used by officials in evaluating whether outcomes have been achieved and whether a particular grant activity achieved value with relevant money’.73 In a November 2018 paper to its audit committee, ARENA recognised the need to improve the definition and recording of project level performance information. The funding agreement template and guidance has not been updated to address this shortcoming. There is merit in ARENA improving its grant funding template guidance for developing performance information in funding agreements.

    Milestones and payments

    3.5 Grant funding agreements require clear milestone and payment structures that are proportionate to risk if they are to be effectively managed.74 ARENA’s grant management policy notes that risk increases with the value of ARENA funding committed.

    3.6 A sample of 44 funding agreements showed the structure of agreement milestones to be largely proportionate in terms of number of milestones, proportion of total funding to be paid at each milestone and spacing over the project duration.75 Analysis of ARENA’s grant management system shows the average number of milestones recorded for each project funded increases as the value of ARENA’s committed funding increases (Figure 3.1).

    Figure 3.1: Average number of milestone by value of committed ARENA funding

     

    Source: ANAO analysis of ARENA Grants Management System data extracted 23 July 2019.

    Milestones scheduled beyond ARENA’s current funding

    3.7 While financial year 2021–22 is the final year that ARENA can request funding from the Consolidated Revenue Fund76, the ARENA Act does not contain sunset provisions for ARENA’s functions post-2021–22. The 2019–20 portfolio budget statements for the Environment and Energy Portfolio forecast that ARENA will operate at a deficit in 2021–22 and 2022–23.77 ARENA’s 2019–20 corporate plan outlines that in the absence of any funding changes, ARENA intends to:

    • commit the majority of its remaining funds in 2019–20 so that these projects have at least two years to be completed and for funds to be paid out; and
    • manage residual contract obligations beyond 30 June 2022.78

    3.8 Since 2015 the ARENA board has considered a series of papers on post-2022 arrangements for grant funding agreements, including legal advice on ARENA’s funding options and legislated obligations. Based on this advice and its considerations, ARENA estimates it will have access to $55.5 million at the end of 2021–22 and $9.3 million at the end of 2022–23 available to meet its grant and operating expenses when it can no longer draw down from the Consolidated Revenue Fund.

    3.9 In June 2019 the ARENA board agreed to the structuring of milestone payments in 2022–23 and beyond. The board was presented with two key reasons for moving away from the previous policy of not scheduling grant payments beyond 30 June 2022. These were:

    • the optimal structuring of new projects requires milestone payments to be scheduled post-30 June 2022; and
    • that 35 to 40 per cent of existing projects experience schedule delay each year. This results in grant expenditure shifting to future financial years by way of contract variation.79

    3.10 At 30 June 2019 ARENA also had 31 grant funding agreements with recoupment mechanisms that will operate out to 2036. The identifiable maximum recoupable value across these mechanisms is $400.5 million.80 These recoupment mechanisms will require ongoing monitoring to determine if the conditions of recoupment are met.

    3.11 As ARENA identified in its 2019–20 corporate plan, post-2021–22 operations are now in its four-year planning timeframe. The board has considered an appropriate range of matters in relation to post-2022 operations. The ARENA Board’s most recent discussion of funding, operational and legal issues relating to post-2022 operations occurred at its December 2019 meeting.81 It tasked ARENA management five actions on these matters and made one workforce planning related decision.

    Assurance of funding agreement compliance

    3.12 Effective ongoing monitoring and management of grant funding agreements are elements of the CGRGs key principles.82 This provides assurance that grants are proceeding as planned and is an important part of achieving value for money. ARENA’s grant management framework identifies that monitoring of projects is essential to ensure compliance with the terms and conditions of the financial assistance, particularly through the delivery of milestones.

    3.13 ARENA’s milestones and payments approval process involves a grant manager making a recommendation to the ARENA decision-maker to accept evidence that the milestone has been completed and release any associated payment. This approval is documented through a Milestone Approval Minute that the decision-maker approves in hard copy. The milestone payment approval processes are not system-based.83

    3.14 A review of the records of supporting evidence for 167 decisions to approve milestones or make funding payments across the 44 sampled funding agreements found that:

    • supporting documents could be validated for 93 per cent of decisions reviewed; and
    • the supporting evidence could not be validated for seven per cent of decisions reviewed.84

    3.15 ARENA’s milestone payment procedures require the record of decision to be stored on a project’s official hard copy file and supporting evidence on ARENA’s electronic shared drive. ARENA’s milestone payment procedures do not require records of decision and supporting evidence for approvals to be saved or linked in the grant management system. Storing or linking decision records and evidence against the relevant milestone in the grant management system would provide a clear and robust record of the decision and supporting information.

    3.16 ARENA has largely demonstrated that milestones were substantiated with documentary evidence. However, ARENA’s use of hard copy supporting and decision documents outside of its grants management system increases risks that appropriate documentation to support decisions is not provided to the decision-maker. The current paper-based record keeping approach also limits ARENA’s ability to provide assurance that the appropriate evidence was considered by the decision-maker.

    Grants management business process and information systems

    3.17 At the time of the audit field work, two-thirds of the 62 steps in ARENA’s grants assessment and management processes are not performed directly on its grants management system (Table 3.2).85 These non-system-based steps occur via email, spreadsheets, word-processor documents and hard copy files, with the grants management system updated at some steps in the processes. Manually-compiled, spreadsheet- and document-based processes have an increased risk of error and less effective controls compared to automated, system-based processes.86

    Table 3.1: Percentage of assessment and grant management process steps by system

    Information system used to perform step

    % of process steps

    Non-system-based: email; spreadsheet; print, sign and scan

    66%

    Partly performed on the grant management system

    15%

    Fully performed on the grant management system

    11%

    Performed on the financial management information system

    8%

       

    Source: ANAO analysis of ARENA documents.

    3.18 Previous reviews and internal audits have made recommendations that ARENA better aligns its grant administration processes and its information systems. ARENA has implemented a range of actions in response, however it has still not achieved fully digital processes with complete electronic information management.

    3.19 Increasing the integration between ARENA’s grant administration processes and its information systems will reduce the risk of errors in the grants administration processes and allow ARENA to provide greater assurance over its management of grants.

    Recommendation no.3

    3.20 The Australian Renewable Energy Agency continues the integration of its assessment and grant management processes with its information systems to strengthen its assurance over its grant management activities.

    Australian Renewable Energy Agency response: Agreed.

    3.21 Integration activities will continue and will be prioritised in accordance with the benefits to the agency.

    Funding agreement variations

    3.22 ARENA’s grant management framework recognises that there are circumstances where it is appropriate to vary a grant. Effective variations management ensures that varied grants continue to deliver value for money, achieve program outcomes and do not create probity issues.

    Number and type of variations

    3.23 ARENA has not established a benchmark or tolerance level for the number of variations it will accept per grant. During the course of audit fieldwork in June 2019, ARENA management reported to the Board that between 35 and 40 per cent of projects experience schedule delays each year, leading to project variations.

    3.24 In July 2019, ARENA implemented a variation tracking module in its grant management system.87 Prior to this, information on project variations could be determined only from the individual project files. This limited ARENA’s ability to inform itself on whether the rate of variations is increasing or decreasing over time.88

    3.25 Of the 44 projects examined as part of sample testing, 26 (59 per cent) had been varied at least once.89 Changes to agreement timing by milestone was the most common rationale in the variations reviewed. The classification of the rationales presented to the ARENA decision-maker90 for these variations is presented in Table 3.2.

    Table 3.2: Rationales presented in the funding agreement variations review by ANAO

    Rationale

    Counta

    ARENA funding: increasing or decreasing ARENA’s funding contribution to the grant.

    10 (12.3%)

    Scope: including changes to outcomes, activities, partners and key personnel.

    6 (7.4%)

    Timing: changes to milestones dates.

    47 (58%)

    External funding: increasing or decreasing funding contributions from the grantee or other partners.

    16 (19.7%)

    Other reasons: including reallocation of funds within grant, non-key personnel changes.

    31(38.2%)

       

    Note a: 81 variations were identified. Some variations have more than one rationale. Percentage is expressed as a proportion of the 81 variations.

    Source: ANAO analysis of 81 variations across 44 ARENA funding agreements.

    Variation policies, procedures and information systems

    3.26 The variation policy and process documents use inconsistent terminology, particularly in the classification of variations (Table 3.3). ARENA had limited procedural guidance for variations prior to the introduction of standard operating procedures for variations in July 2019. While these procedures have improved its variation documentation, inconsistencies remain across the suite of procedural guidance and policy. These inconsistencies and the lack of system-based approvals limit the assurance that varied funding agreements still deliver value for money, achieve program outcomes and do not raise probity issues.91

    Table 3.3: Comparison of ARENA’s variation documents

    Document

    Classification of variations

    Financial assistance manual: October 2017, internal rules and procedure for grants management

    ‘Changes in scope’, ‘extension of time’, ‘change in value’.

    Variation policy: January 2018, public document

    ‘Material’, ‘non-material’ and ‘variations that should not be considered’.

    ARENA delegations: November 2018 and September 2018

    Delegations for variations are based on project risk rating and financial values.

    Funding Agreement template: January 2019

    Only defines ‘minor variations’.

    Standard operating procedures — variations: July 2019

    Refers to variation policy for classifications.

       

    Source: ANAO analysis of ARENA documentation.

    Variation consideration of value for money, program criteria and probity

    3.27 ARENA’s policy and process documents identify that value for money, program criteria and probity are important matters when considering a variation. The ANAO assessed evidence of value for money, program criteria and probity considerations in the records of decision for 81 variations across the 44 sampled funding agreements.

    3.28 Variation approval records containing clear evidence of the decision-maker directly considering all three elements of value for money, program criteria and probity of the variations could be identified for only four of these variations.92 ARENA advised ANAO that these matters are only included in information provided to a decision-maker if the matter is considered material in the context of the specific variation.

    3.29 Case study 2 presents an illustrative example of the risks and issues that can arise when controls are not effective through variation policies and procedures. This case study was selected as it highlights all three issues of value for money, program criteria and probity of the variations. It is not representative of the process undertaken for all variations.

    Case study 2. Consideration of value for money, program criteria and probity in a variation

    Background

    ARENA committed a total of $1.29 million through a grant funding agreement comprising 66 per cent of the total cost of the project:

    • The original grant funding agreement was executed in 2014 under the Emerging Renewables Program, providing $450,000 in ARENA funding with grantee contributions of $159,500 for activities to be delivered by 2018.
    • ARENA approved a variation in December 2016 for a second stage. This provided $420,000 in ARENA funding with an intended additional contribution from the grantee of $205,289. This variation also extended the grant to 2020.
    • ARENA approved a second variation in June 2019 for a third stage, providing the final $420,000 in ARENA funding with an intended additional contribution from the grantee of $202,800. This variation also extended the grant to 2022.
    • The 2019 variation was executed after ARENA’s Variation Policy came into effect in 2018. ARENA’s Variation Policy:
      • classifies variations that give effect to ‘a second stage of a project’ as ‘variations that will not generally be considered by ARENA’; and
      • provides that for this classification of variation, applicants are to be ‘invited to submit a proposal through ARENA’s open programs.’

    Consideration of the 2019 variation

    • The ARENA decision-maker was not presented with documented advice on the application of ARENA’s Variation Policy and why this variation should not be submitted as a new application.
    • The decision-maker was advised that ARENA’s contribution to the project ‘does not meet the matched funding requirement’ but the variation should be approved because of ‘high “public good” knowledge sharing benefits’.
    • The advice to the decision-maker did not disclose a probity issue that the applicant is also an evaluation service provider to ARENA. The applicant has completed a publicly available evaluation of over 300 funded projects with positive findings about ARENA.

    ARENA’s decision to not consider the variation’s value for money and merit against program guidelines through the established application assessment process creates risk that ARENA is perceived to treat applicants inconsistently. It also limited ARENA’s ability to provide assurance that these matters were effectively assessed. ARENA’s management of probity in its procurement of evaluation providers is further discussed in Chapter 4 (see paragraphs 4.31–4.38).

    3.30 Effective management of variations is also required due to the rate of variations to milestone dates and ARENA’s timeline under its legislation. ARENA’s variations policy recognises the importance of value for money, merit against program guidelines and probity considerations when assessing variation requests. The variation policies and procedures do not provide effective controls to ensure these matters are appropriately considered for all variations.

    Recommendation no.4

    3.31 The Australian Renewable Energy Agency improves its funding agreement variation policies and processes to ensure appropriate consideration of value for money, merit against grant program criteria and probity when making decisions on variations.

    Australian Renewable Energy Agency response: Agreed.

    3.32 ARENA’s Financial Assistance Manual, Delegations, Variation Policy and Variation Process will be reviewed and redrafted to ensure consistency and to ensure that all relevant considerations identified by the ANAO are clearly documented in relation to all variation decisions.

    Are governance arrangements for the oversight of grant funding agreements robust?

    Governance arrangements for the oversight of grant funding agreements are robust. Funding agreement governance structures include an appropriate level of senior management oversight. Key data sources and reports were automated and digitised in late 2019, reducing the risk of the presentation of incorrect or incomplete information.

    Funding agreement oversight arrangements

    Grant funding governance and operational structures

    3.33 ARENA has established an oversight structure that links the operational teams managing grant funding agreements, through operational and executive management, to the ARENA Board:

    • at the operational level, the Project Delivery Branch is responsible for grant funding agreement management. The allocation of individual grant funding agreements between two teams is based on project risk93;
    • there are clear management roles, responsibilities and reporting lines between the Project Delivery Branch’s teams, the Executive Leadership Team and the ARENA Board; and
    • this main line of reporting is supported by additional committees, including the Project Portfolio Management Committee and the Risk and Audit Committee.

    3.34 Overall, ARENA has in place an appropriate range of business management and oversight committees with a suitable range of governance responsibilities.

    Funding agreement oversight information

    3.35 The Project Delivery Branch’s managers and the ARENA Board receive structured and regular reporting on grants management. Specific matters for noting or decision are brought to committees as agenda items with papers when required.

    3.36 During the course of the audit, ARENA implemented business improvement projects that improved the information available to its staff, managers and governance committees with:

    • a set of electronic reporting dashboards covering performance metrics such as milestones due and payment processing times. These dashboards can be accessed on ARENA’s intranet. Some of these metrics are incorporated into governance committee reporting, such as the CEO and Financial Reports to the ARENA Board;
    • the transition of the Project Delivery Branch’s spreadsheet-based risk assessment process for grants to a system-based process undertaken in ARENA’s financial management information system94; and
    • a standardised reporting dashboard in the grant management system that replaced a manually compiled management report that captured operational information including milestones overdue; contracts with major issues, variations; and summary grant management system statistics.

    3.37 The implementation of structured, electronic reporting tools will assist in addressing the data integrity and control risks that previously existed with ARENA’s reliance on manually compiled reports. It also increases the type and quality of information available to support funding agreement oversight.

    Are risks and issues identified and dealt with appropriately?

    Risk and issues management processes are largely appropriate. The overall management of risks and issues in funding agreements has been improved by recent transitions of some of these activities onto information systems. Further integration of risk and issue management into business processes on information systems would enhance ARENA’s risk and issue management.

    Organisational risk management

    3.38 Accountable authorities of all Commonwealth entities must establish and maintain appropriate systems of risk oversight, management and internal control.95 ARENA’s board has established a risk framework that is consistent with the nine elements of the Commonwealth Risk Management Policy. This framework includes:

    • policies and procedures such as a risk management policy, a governance and internal control framework;
    • operational policies, procedures and systems with embedded risk management;
    • a quarterly strategic risk report that includes the risk appetite statement and strategic risk management information; and
    • project level risk information in the CEO report presented at each board meeting that includes summary information about extreme risk projects and in some months, statistics on the risk categories across the project portfolio (Table 3.4).

    Table 3.4: Project level risk classifications presented to the ARENA Board

     

    Low risk projects

    Medium risk projects

    High risk projects

    Extreme risk projects

    Total

    Projects

    190

    18

    8

    3

    219

               

    Source: ARENA Board Reporting September 2019.

    Grant funding agreement risk management

    3.39 ARENA’s risk assessment procedure for grant funding agreements was updated in February 2019. The procedure documents how ARENA assigns an overall risk classification to each project.

    3.40 A project risk assessment is to be completed for each funding agreement on a quarterly basis by the relevant ARENA grant manager. Grant managers are also required to upgrade these risk ratings at any time a major matter comes to their attention. The risk assessments are used to classify each project in one of four categories from low to extreme risk, as shown in Table 3.4.

    3.41 The risk classification assigned to each project is used to inform grant management decisions, including:

    • which Project Delivery Branch team manages the project96; and
    • the approval of milestones and variations by ARENA staff under powers delegated from the CEO. These delegated approval limits are proportional to risk, with more senior staff able to approve higher value milestones and variations.

    3.42 In February 2019, the project risk assessment was transitioned from a spreadsheet-based process to the financial management information system. Converting this process into a system-based process improves the controls over this risk information. Off-system document-based processes, such as variations and milestone approvals, still need to be manually updated with a project’s risk classification.

    Presentation of risk and issue information to decision-makers

    3.43 Between August and October 2017 ARENA updated its contract management policies, procedures and milestone approval templates, including the requirements for the presentation of risk and issue information to decision-makers. The updated milestone approval minute template included a section for presenting project risk rating information. These policies and procedures did not mandate the use of the milestone approval minute template.

    3.44 The sample of 44 funding agreements included 97 milestone approvals made after October 2017 (Table 3.5). Of the 97 milestone approval decisions since October 2017, 80 contained appropriate communication of project risks and issues to the decision-maker.

    3.45 Decision records that did not communicate risks and issues to the decision-maker included two milestone approval minutes that did not complete the project risk rating section and 15 approvals that did not use the milestone approval minute template. The presentation of risk and issues would be improved by mandating a standard approach for documenting and approving milestone approvals.

    Table 3.5: Communication of project risks and issues to decision-maker

     

    Decision record

    Are project risks and issues communicated to the decision-maker?

    Milestone approval minute

    Other minute

    Email

    Total

    Yes

    76

    4

    80

    No

    2

    12

    3

    17

    Total

    78

    16

    3

    97

             

    Source: ANAO analysis of ARENA documents.

    Funding agreement sanctions

    3.46 Grant funding agreements require appropriate sanctions that provide mechanisms for effective risk and issue management. The funding agreement template contains clauses for the: suspension of funding; reduction in funding; termination or reduction in scope with cause; termination for an insolvency event; and the repayment of ARENA funding.

    3.47 At July 2019 eight per cent of ARENA’s funding agreements were classified as terminated (Table 3.6). The ARENA 2018–19 Annual Report identifies that ARENA has received $63.5 million through the return of grants since 2014–15.

    Table 3.6: ARENA funding agreement status — 23 July 2019

     

    Pending financial close / conditions precedenta

    Active

    Under variationb

    Terminated

    Closed

    Total

    Funding agreements

    5 (1%)

    200 (42%)

    15 (3%)

    38 (8%)

    221 (46%)

    479

                 

    Note a: Executed ARENA funding agreements where the project’s activities have not yet commenced.

    Note b: This status only applies to projects under variation at a particular point in time. When variations are complete, that status is changed back to Active. It is not a cumulative count of all grants that have been varied.

    Source: ANAO analysis of ARENA Grants Management System data extracted on 23 July 2019.

    3.48 No actions were identified that were inconsistent with sanctions established by the guidelines and funding agreements in the review of 44 sampled funding agreements.

    3.49 ARENA informed the ANAO that no funding recipients have lodged formal disputes under the dispute clauses of the funding agreements. ANAO’s review of ARENA’s complaints register did not identify any complaints that relate to ARENA undertaking actions that may be inconsistent with funding agreements.

    Management of probity risks

    3.50 The Joint Committee of Public Accounts and Audit has highlighted the importance of Commonwealth entities fully complying with conflict of interest requirements.97 The CGRGs identify probity and accountability as key principles of grants administration.98 This includes that accountable authorities should:

    • ensure that entity policy and management processes for conflict of interest are published to support probity and transparency; and
    • put in place appropriate mechanisms for identifying and managing potential conflicts of interest for grant opportunities.

    3.51 ARENA has a range of probity measures in place:

    • documented conflict of interest policies, supported by an ongoing training program;
    • probity registers are maintained for the board, executive, staff and assessment panel;
    • an annual declaration of interests process, in addition to ongoing disclosure requirements; and
    • mechanisms recognising that ARENA deals with market sensitive information, such as an embargo register.

    3.52 ARENA application processes and templates assist the board, executive, staff and the assessment panel to identify potential probity issues. Applicants are required to identify all entities involved in the application. This information is then provided to those involved in the assessment process to assist them in identifying any probity issues. Board and assessment panel papers clearly identify entities that are subject to deliberation.

    3.53 The assessment advisory panel member independence declaration process is the only probity activity that is managed on an information system.99 ANAO analysis of data extracted from the grant management system on 23 July 2019 found:

    • this step was completed for all 3,201 panel member assessments recorded in system; and
    • for the eight assessments where a panel member identified a conflict, the system did not allow those panel members to continue assessing that application.100

    3.54 The advisory panel member’s merit assessment of an application is only one of the 31 steps in ARENA’s application assessment process.101 All other probity activities and registers across the application assessment and grant management processes are managed in spreadsheets and documents. This creates the same integrity and control risks that are to be addressed by Recommendation no.3 (see paragraph 3.20).

    3.55 Probity issues in relation to evaluation of service providers are further discussed in paragraphs 4.31–4.38.

    4. Evaluation and performance

    Areas examined

    This Chapter examines if evaluations indicate that the Australian Renewable Energy Agency (ARENA) is achieving its objectives. It also covers ARENA’s reporting arrangements and whether lessons from evaluations are actioned appropriately.

    Conclusion

    ARENA’s external evaluations since 2017 do not clearly demonstrate the extent to which ARENA’s programs are impacting on its legislative objectives of improving the supply and competitiveness of renewable energy in Australia.

    Areas for improvement

    The ANAO made two recommendations designed to improve assurance that ARENA’s performance measurement framework is consistent with the Commonwealth performance framework and disclose potential actual or perceived conflicts of interest for organisations engaged to evaluate the effectiveness of ARENA’s activities.

    The ANAO also suggested that ARENA publish its evaluations and strengthen procedures to close recommendations from evaluations. The recommendations on performance measures and additionality made in Chapter 2, paragraphs 2.26 and 2.43 are also relevant to the improvement of ARENA’s evaluation activities.

    4.1 Evaluating grant activities to ensure that objectives are achieved and relevant money is appropriately managed is a key part of grants management. The following criteria were used to assess if evaluation of grant programs indicate that ARENA is achieving its objectives:

    • Are evaluation frameworks and reporting arrangements appropriate?
    • Does evaluation demonstrate the impact of objectives?
    • Are lessons from evaluations actioned appropriately?

    Are evaluation frameworks and reporting arrangements appropriate?

    In 2018–19 ARENA’s evaluation and performance policies and procedures were not appropriate. Evaluation and performance measurement activities were not aligned. The assurance process over the performance framework was not effective. Some improvements have been made to these policies and procedures in 2019–20.

    ARENA’s policies and procedures for evaluation and performance reporting

    4.2 ARENA’s financial assistance manual sets out general principles for the evaluation of its grant activities.102 The key evaluation and performance reporting policy and procedure documents consist of:

    • the 2019 Evaluation Framework and supporting program and portfolio specific monitoring and evaluation plans; and
    • the 2018–19 Performance Framework Guide (draft).103

    4.3 The 2019 Evaluation Framework was a major update and restructure to the 2015 version it replaced. This update made improvements that included establishing clear governance and reporting arrangements and a schedule of evaluations to be conducted from 2019 to ‘post 2022’.104

    4.4 For the versions of the evaluation framework and performance framework guide in place during 2018–19, there was a lack of clear references between documents. The performance framework guide did not clearly explain how ARENA’s evaluation and performance processes aligned with the Commonwealth performance framework. The poor linkage between ARENA’s evaluation framework and its performance measurement framework limited the potential contribution of evaluations to ARENA’s performance statements.105

    4.5 ARENA’s performance framework would be improved by clearly and publicly identifying in advance how the evaluations will be used as performance measurement methods and information sources in ARENA’s performance framework. This is related to Recommendation no.1 (see paragraph 2.26).

    Reporting and assurance arrangements

    4.6 Section 16F of the PGPA Rule requires an accountable authority to certify that their performance statements accurately present the entity’s performance for the reporting period. It is important that an accountable authority has confidence that the entity has adequate processes in place to provide sufficient assurance to support this certification.106 Relevant processes may include:

    • detailed audit committee review and endorsement;
    • effective use of internal audit resources;
    • documented procedures and processes supporting the performance reporting cycle;
    • progressive reporting of results to internal stakeholders such as the accountable authority and audit committee throughout the year; and
    • management certifications supported by suitable quality assurance of results.107
    Assurance by the ARENA Risk and Audit Committee and ARENA Management

    4.7 Section 45 of the PGPA Act requires accountable authorities to appoint an audit committee and that audit committee must perform its functions in accordance with the PGPA Rules. ARENA’s Risk and Audit Committee’s (RAC) charter incorporates the audit committee functions specified in section 17 of the PGPA Rules.108

    4.8 The ARENA RAC Charter includes that the RAC is to review and provide written advice as assurance about the appropriateness of ARENA’s performance reporting. This includes reviewing the framework for developing and reporting key performance indicators and ARENA’s annual performance statements. The RAC’s 2018–19 and 2019–20 work plans include responsibilities and actions that aligned with Department of Finance guidance for audit committee’s roles and functions for reviewing the appropriateness of an organisation’s performance reporting.109

    4.9 The RAC considered performance framework matters at each of its 10 meetings between August 2017 and November 2019. The ARENA Board received 25 papers related to performance framework matters from the RAC or management between June 2017 and December 2019. ARENA also commissioned an internal audit that reviewed the 2017–18 and 2018–19 performance statements that was completed in November 2019.

    4.10 Despite this range of activities by ARENA management and the RAC, shortcomings with ARENA’s 2018–19 Performance Measurement Framework outlined in Chapter 2, paragraphs 2.13–2.23, were not identified. The ARENA Board’s ability to identify these shortcomings when approving the 2018–19 Performance Framework in June 2018 was diminished by the lack of effective management and audit committee assurance.110

    Recommendation no.5

    4.11 The Australian Renewable Energy Agency implements policies and processes to provide effective assurance that its performance measurement framework and reporting is fully consistent with the Commonwealth performance framework.

    Australian Renewable Energy Agency response: Agreed.

    4.12 A program of work will be planned to ensure that the performance framework complies with relevant rules, is supported by appropriate data collection and that the assurance methodologies provide an appropriate level of assurance.

    Reporting against the corporate plan

    4.13 Regular reporting to key internal stakeholders of progress in achieving the measures and other commitments outlined in an entity’s corporate plan supports good decision-making and is an indicator of an entity’s commitment to positioning the corporate plan as the entity’s primary planning document.111

    4.14 ARENA reports on its corporate plan performance measures on an annual basis to the audit committee and the ARENA Board through presentation of the draft annual report. ARENA’s executive leadership team meetings review the performance measures in the annual work plan on a bi-monthly basis.112 The annual work plan measures do not directly align to the performance measures outlined in ARENA’s corporate plan.

    4.15 More frequent internal reporting using the corporate plan’s performance framework would increase ARENA’s ability to measure its performance and address issues as they arise.

    Reporting of periodic evaluations to ARENA governance bodies

    4.16 The outcomes and recommendations of evaluations should be considered by relevant governance bodies to ensure that appropriate action is taken and tracked. To determine whether there is appropriate governance oversight of evaluations, the ANAO examined the papers of ARENA’s key governance bodies.113 A consistent approach to the presentation of periodic evaluation to governance bodies could not be identified.

    4.17 The 2015 Evaluation Framework did not establish clear governance roles. This issue has been addressed in the 2019 version. Application of the governance roles defined in the 2019 Evaluation Framework will facilitate the improved oversight of periodic evaluations by ARENA’s governing committees.

    Public reporting of knowledge sharing

    4.18 One of ARENA’s functions under its Act is ‘to collect, analyse, interpret and disseminate information and knowledge relating to renewable energy technologies and projects’.114 ARENA publishes a range of information on its website as part of this legislative knowledge sharing function. Data shows 364 reports, presentations and other knowledge sharing documents were available on the ARENA website as at 14 October 2019. These documents were downloaded 20,698 times from March 2018 to October 2019.115

    4.19 ARENA’s knowledge sharing section of its website uses a thematic search approach that has been developed to allow users to find knowledge sharing products without needing to know the name of a specific project.116 However, not all knowledge sharing material is directly linked to the project that produced it.

    4.20 A comparison of the knowledge sharing documents data and the number of ARENA projects with committed funding shows:

    • 229 of these documents are directly linked to 139 grant projects in the data set. This was 29 per cent of all projects with committed funding as at 30 September 2019.
    • 135 of these documents are not linked to an individual grant project in the data set. ARENA advised that not all documents can be linked back to individual grants as some documents are not produced by a specific project.
    • 71 per cent of ARENA’s projects do not have a linked document in the knowledge sharing document data set supplied to ANAO.

    4.21 As noted in Chapter 2, knowledge sharing is a key element of the additionality of ARENA’s grant funding.117 As well as the thematic search approach, presenting specific information on the knowledge sharing outputs that will be publicly released from each grant would provide stakeholders with a clearer understanding of what is to be produced. Stakeholder expectations for clear project level linkages to knowledge sharing outputs was highlighted in market research presented to ARENA in November 2019.

    4.22 Links to knowledge sharing products on each project’s ARENA webpage, or a description of what these deliverables are if they cannot be published, would improve accountability and transparency.

    Public reporting through corporate documents

    4.23 The PGPA Act and ARENA Act require ARENA to publish a range of strategic and corporate documents and its grants guidelines. The required documents are available on ARENA’s website.118 ARENA’s annual report includes an index of compliance with legislated reporting requirements.119 These requirements include lists of financial assistance provided. ARENA includes its Annual Performance Statements in its annual report as required by the Commonwealth performance framework.

    4.24 The 2018–19 annual report did not provide a complete or reliable assessment of ARENA’s progress against the performance measures and impacts set out in the 2018–19 corporate plan.120 The 2018–19 performance statement included information against ARENA’s identified performance measures except for the ‘cost and/or revenue improvements for renewable energy technologies’.

    4.25 ARENA also used three case studies to report against two of its six impact statements, but did not link any quantitative performance measures to them.121 Case studies should not be relied upon as a stand-alone measurement, unless: the scope is predetermined; activities considered are clearly stated; and measurement methods detailed in advance.122 ARENA’s 2018–19 corporate plan did not include this context for the use of its case studies. The quantitative performance measures presented in the 2018–19 annual report did not support a complete and reliable assessment of ARENA’s performance for the reasons presented in Chapter 2.123

    4.26 In the two years to December 2019 ARENA finalised 10 evaluations under its evaluation framework. Of these, eight were focused on ARENA’s outcomes. Two are publically available on the ARENA website.124 In order to provide greater transparency on the effectiveness in delivering its legislated objectives, ARENA should publish all of its evaluations.

    Does evaluation demonstrate the impact of objectives?

    The eight outcome evaluations undertaken since 2017 generally conclude that ARENA is delivering its programs, but only three provided clear conclusions about the impact of these programs on ARENA’s objectives. There is scope for ARENA to improve its evaluations so that they present clear, impact focused conclusions against ARENA’s legislative objectives. This and the implementation of recommendations on performance measures and additionality will enhance ARENA’s ability to attribute its outcomes to its legislative objective and functions.

    4.27 ARENA uses two main approaches to inform itself on whether it is meeting its objectives: its performance measurement framework, examined in Chapter 2, 2.11–2.24, and periodic evaluations.

    4.28 In relation to evaluations ARENA’s Evaluation Framework requires that evaluations meet specified criteria to demonstrate the proper use of public resources, including for:

    • Appropriateness — the extent to which the objective of the activity contributes to ARENA’s purpose, for example, what would have happened in the absence of the activity or would alternative approaches have been more appropriate?
    • Effectiveness — the extent to which intended outcomes are being achieved, including any unintended positive or negative consequences; and
    • Efficiency — the extent to which inputs are minimised to achieve the activity’s objectives and outcomes, including whether processes were efficient and represent value for money.

    Evaluations undertaken

    4.29 A breakdown of the type and focus of the 10 periodic evaluations since 2017 are presented in Table 4.1.

    Table 4.1: Characteristics of ARENA evaluations finalised since 2017

    Type of evaluationa

    Evaluation Focus

    Process: investigates enabling processes

    Economic: examines economic impactb

    Outcome: evaluates if objectives are achieved

    Total

    Program or round

    4

    4

    Portfolioc

    1

    3

    4

    All of ARENA

    1

    1

    2

    Total

    1

    1

    8

    10

         

    Note a: Types of evaluation have been adapted from Auditor-General Report No. 47 of 2018–19 Evaluating Aboriginal and Torres Strait Islander Programs.

    Note b: ARENA’s legislated objectives do not explicitly refer to economic impacts.

    Note c: ARENA uses the Portfolio concept to evaluate a group of grants with a common characteristic. This can be a technology type such as solar, or another characteristic such as regional locations.

    Source: ANAO analysis.

    Evaluation findings

    4.30 Of the eight outcome evaluations, three were considered to clearly conclude on the impact of the grant programs on ARENA’s objectives. Overviews of these conclusions are presented in Table 4.2. The other five outcome evaluation made positive findings about ARENA’s administration of its grant activities, but did not provide clear and direct conclusions about the impacts of grant activities on ARENA’s legislative objectives.

    Table 4.2: Summary of evaluation conclusions that address ARENA’s impact

    Evaluation

    Findings

    Advancing Renewables Program — Large Scale Solar (LSS) PV competitive round Evaluation — November 2019; Outcome evaluation

    This evaluation makes the following high-level conclusions:

    • intended outcomes/objectives of the Competitive Round were met;
    • objectives of the large-scale solar projects prior to the Competitive Round were achieved; and
    • aside from industry cost reduction, the ARENA funding of LSS projects provided broader benefits to the market that have contributed to the achievement of program objectives and also unintended negative consequences.

    In the spotlight: Australian solar energy research and development outcomes and achievements in a global context; July 2018; Outcome evaluation

    The evaluation concluded that ‘ARENA funding has significantly boosted Australia’s solar R&D effort, and in turn the research outcomes and capability which has kept Australia in the forefront of this burgeoning new international solar industry’.

    All of ARENA Evaluation; September 2019; Outcome evaluation

    The evaluation concluded that ‘ARENA’s funding support has helped accelerate the shift to a more affordable and reliable renewable energy system. ARENA has improved the competitiveness of renewable energy through programs such as the Large-Scale Solar program and Advancing Renewables Program. ARENA’s extensive knowledge sharing activates are driving industry and government to be better informed to navigate the energy transition.’

      

    Source: Evaluation reports provided to ARENA.

    Robustness of evaluations

    4.31 All 10 evaluations were commissioned by ARENA and conducted by third parties. The ANAO reviewed the robustness of ARENA’s 10 evaluations based on the questions presented in Table 4.3.

    Table 4.3: ANAO robustness assessment of evaluations

    Questions to assess robustness

    Average traffic light rating

    Does the evaluation clearly identify the type of evaluation and its methodology?

    Does the evaluation rely on a reasonable range of evidence?

    Does the evaluation include benchmarking or assessment against targets?

    Was the evaluation conducted by a third party?

    Did the evaluation include information about the independence of the third party?

    Were potential threats to independence disclosed or identified?

    Does the evaluation draw clear conclusions that address the evaluations objectives?

    Are conclusions impact-focused?

    Is the evaluation report well-presented with a cohesive and logical flow?

    Are there other obvious issues or problems with the evaluation?

      

    Legend:  robust minor issues major issues

    Source: ANAO analysis.

    4.32 The strengths of the evaluations are that they all clearly identify the type of evaluation and the methodologies that will be used.

    4.33 The issues identified across the evaluations included:

    • most of the outcome evaluations did not present clear, impact focused conclusions against the objectives, limiting the clarity and value of the evaluation to the reader; and
    • most evaluations either did not clearly outline the evidence base for the evaluations, or did not balance internal or secondary information sources, such as interviews or existing ARENA records, with other external evidence sources or benchmarking.

    4.34 Three of the eight parties commissioned to conduct evaluations are also recipients of ARENA grant funding. The final reports for these evaluations did not contain disclosure of these potential conflicts and any actions taken to manage them. ARENA defines a conflict of interest as:

    a situation where there is, or appears to be, a divergence between the interests of a person and their professional responsibilities to ARENA, such that an independent observer might reasonably question whether the actions or decisions of that person are influenced by their interests.125

    4.35 ARENA’s Conflict of Interest Policy requires all potential service providers involved in procurement to declare potential conflict of interest. This policy then requires the relevant ARENA delegate to assess the materiality of declared conflicts and identify actions to manage material conflicts.

    4.36 Table 4.4 describes the three cases where ARENA grant funding recipients were also evaluation services providers to ARENA. ARENA was unable to provide evidence that the materiality of these conflicts were considered by the delegate, or that actions to manage material conflicts were documented.

    Table 4.4: ARENA consideration of conflict of interests by evaluation providers.

    Case

    Conflict declared by provider in proposal

    ARENA materiality assessmenta

    ARENA recorded actions to managea

    Declared in final evaluation report

    1

    This evaluation provider made a general declaration in its proposal to ARENA that the evaluation provider and its staff have been involved in several ARENA and one of its predecessor organisation’s projects.b

    To manage this conflict, the provider stated ‘Should projects in which [evaluation provider] has been involved be cited via outcomes, Case Studies or otherwise, this will be noted.’

    2

    This evaluation provider identified several projects within the scope of the evaluation that they had been involved with.

    The evaluation provider stated it did not consider its involvement in this project to be potential conflicts of interest.

    3

    This evaluation was undertaken by a university that also receives ARENA funding. No conflicts of interest were declared in the proposal to ARENA.

         

    Note a: These actions are required under ARENA Conflict of Interest Policy.

    Note b: Grants from this organisation were transferred to ARENA in 2012 and were part the scope of this evaluation.

    Source: ANAO analysis of ARENA records.

    4.37 Two members of the evaluator team for Case 1 in Table 4.4 sit on ARENA’s advisory panel and were members of the predecessor organisation’s assessment research advisory panel. These panels were involved in the merit assessment process for grants covered by the evaluation. A declaration of this conflict could not be identified in the material provided to ARENA by the provider.

    4.38 There is no evidence that ARENA had identified this conflict, considered its materiality or documented actions relating to this procurement. The evaluation provider in Case 1 of Table 4.4 is also the organisation that received ARENA funding through processes discussed in Case Study 2, presented in Chapter 2.

    Recommendation no.6

    4.39 The Australian Renewable Energy Agency assesses and actively manages conflicts of interests of organisations engaged to conduct its evaluations and disclose, where relevant, any conflicts in evaluation reports and material provided to decision-makers.

    Australian Renewable Energy Agency response: Agreed.

    4.40 In March 2020, following audit field work, ARENA has revised its conflict of interest and probity procedures in relation to procurements of evaluators, and procurements more generally.

    ARENA’s impact on objectives

    4.41 As noted in Chapter 1, paragraphs 1.3–1.4, under the ARENA Act, ARENA’s:

    • objectives are to improve the competitiveness of renewable energy technologies and increase the supply of renewable energy in Australia; and
    • functions include providing financial assistance for, and disseminating information and knowledge relating to, renewable energy technologies and projects.
    Trends in Australian Renewable Energy since 2012

    4.42 The Australian Energy Regulator has reported the generation capacity of renewable energy in the National Electricity Market has increased since 2012–13. This is primarily through the installation of wind and solar (Figure 4.1).

    Figure 4.1: National Electricity Market Generation capacity change 2012–13 to 2018–19

     

    Source: Australian Energy Regulator, State of the energy market 2018 (2019 data update), Figure 2.20.

    4.43 The Australian Energy Regulator’s annual report on residential electricity price trends shows that residential electricity prices are expected to decrease between 2018–19 and 2021–22 for most Australian states and territories. The report noted:

    • New renewable generation means that prices of electricity are lower during peak renewable production periods, which may lead to lower wholesale electricity purchase costs depending on the hedging profiles of retailers.
    • Additional supply [provided by renewable energy generation] places downward pressure on wholesale pricing.126

    4.44 The Department of the Environment and Energy’s emissions projections for 2019 note that since 2016 falling emissions from electricity generation have been driven by large amounts of renewable generation entering the market.127

    Attribution of ARENA’s activities to these trends

    4.45 These trends in Australian renewable energy are consistent with ARENA’s legislative purpose and objectives. As noted in Chapter 1, paragraph 1.2, there are a range of Australian government activities designed to support the development of renewable energy technologies in Australia. These measures and other policy, economic and social factors, may be accelerating Australia’s shift to affordable and reliable renewable energy.

    4.46 It is important that ARENA’s evaluations identify the contribution of ARENA’s grant activities to the current trends in the Australian renewable energy to assess its impact on its objectives. In aggregate, ARENA’s evaluations do not demonstrate the extent to which ARENA’s outcomes are impacting on its purpose and legislated objectives as required by its evaluation framework.

    4.47 ARENA’s ability to attribute the impact of its grants activities to the trends in Australian renewable energy would be strengthened by:

    • addressing the shortcomings with the performance measurement framework as outlined in Recommendation no.1;
    • clearly assessing additionality in the grant selection process as outlined in Recommendation no.2; and
    • ensuring periodic evaluations make clear and robust conclusions as to whether the program or portfolio is effectively achieving ARENA’s objectives.

    Are lessons from evaluations actioned appropriately?

    While lessons from ARENA’s evaluations were mostly actioned appropriately, there is scope for ARENA to strengthen its arrangements for closing recommendations.

    4.48 To determine if lessons from evaluations were actioned appropriately, the ANAO reviewed:

    • board papers supporting the approval of strategic planning documents and grant funding guidelines;
    • the implementation arrangements for the Future Needs Review process evaluation; and
    • ARENA’s system for tracking the implementation of evaluation recommendations.

    Evaluation input to board approvals of strategic document and guidelines

    4.49 The board’s approval of strategic planning documents and grant funding guidelines are supported by papers from management describing the development of these documents. The ANAO’s review of 15 of these papers that were presented to the board between May 2017 and December 2019 found limited references to periodic evaluations in these papers. These board papers were informed by lessons from a range of other information sources including stakeholder consultation and workshops; outputs from ARENA funded grants; government priorities; ARENA’s investment priorities; and legal advice.

    Future Needs Review

    4.50 The process evaluation identified in Table 4.1 was the Future Needs Review completed in April 2017. The aim of the review was to provide the board with recommendations to improve ARENA’s effectiveness in pursuing priorities and objectives. It covered organisational structure, processes and procedures, stakeholder experience, skills and capabilities and response to previous reviews. The review made 29 recommendations, all of which were accepted by ARENA.

    4.51 To monitor the status of the implementation of recommendations, the ARENA board established a Review Implementation Oversight Committee in June 2017. The oversight committee provided regular updates to the board until November 2018 when the Board considered the recommendations to be implemented aside from two business as usual matters. Some information technology projects to address the remaining actions continued with direct reporting to the board. On advice from ARENA management, the board closed all outstanding recommendations from the review in April 2019.

    4.52 ARENA reported to the Board that key outcomes arising from the review have included:

    • stronger business planning processes, including earlier preparation of the corporate plan;
    • greater policy alignment with the department and other energy agencies;
    • increase in CEO delegations;
    • benchmarking of administrative costs, with costs kept within the established thresholds; and
    • approval of a Business Case for Enhanced Technology (ICT investment) which formed the basis for systems improvement including grants management system configuration and new financial management information system (FMIS).

    4.53 Initiatives arising from the review were examined by the ANAO with respect to its adoption of a portfolio management to grant management, grant management system improvements and increased focus on milestone processing and payments. As noted in the report, these initiatives will support ARENA’s management of its grant programs.128

    Evaluation recommendation implementation and tracking

    4.54 ARENA monitors the implementation of agreed evaluation recommendations in a spreadsheet evaluation tracker. The framework states the tracker is to document:

    • ARENA’s response to each recommendation;
    • ARENA’s agreed action in response to each recommendation;
    • a status update on the implementations of actions every six months; and
    • the Executive Leadership Team member that is responsible for implementing that action.

    4.55 The framework does not outline procedures for closing recommendations, such as the committee or officer responsible for reviewing and closing recommendations and the level of documentation required to substantiate closure. The inclusion of recommendation closure procedures in the framework would strengthen the assurance provided through the tracker.

    4.56 At February 2020, 75 evaluation recommendations were recorded on the ARENA evaluation tracker. The information recorded on the spreadsheet indicated that ARENA had taken appropriate action in relation to 67 of these recommendations. For six recommendations, the information recorded was not clear enough to determine whether appropriate action had been taken. Actions for two recommendations relating to the grants management system were marked as complete, stating the issues raised would be considered in another review, but no information on the outcomes of the further review was captured on the tracker. Based on the analysis presented in Chapters 2 and 3129, these two evaluation recommendations have not been appropriately actioned.

    Appendices

    Appendix 1 Australian Renewable Energy Agency response

    Page one of the response from the Chair of the Australian Renewable Energy Agency.

    Page two of the response from the Chair of the Australian Renewable Energy Agency.

    Page three of the response from the Chair of the Australian Renewable Energy Agency.

    Appendix 2 Criteria for the assessment of the appropriateness of performance information

    Table A.1: Criteria for the assessment of the appropriateness of performance information

    Dept. of Finance guidance

     

    Assessment characteristics

    Explanation

    Relevant

    Individual assessment

    Benefit

    The performance criterion clearly indicates who will benefit and how they will benefit from the entity’s activities.

    The performance criterion should explain who will benefit from the activity and how the recipient will benefit.

    Focus

    The performance criterion should address a significant aspect/s of the purpose, via the activities.

    The performance criterion should assist significantly in informing whether the purpose is being achieved and the attribution of the entity’s activities to it is clear.

    Understandable

    The performance criterion should provide sufficient information in a clear and concise manner.

    The performance criterion should be stated in plain English and signal the impacts of activities to inform users.

    Reliable

    Measurable

    The performance criterion should use information sources and methodologies that are fit for purpose.

    The performance criterion should be capable of being measured to demonstrate the progress of fulfilling the purpose. This includes documenting a basis or baseline for measurement or assessment, for example a target or benchmark.

    Free from Bias

    The performance criterion should be free from bias and where possible, benchmarked against similar activities.

    The performance criterion should allow for clear interpretation of results and provide an unbiased basis for assessment.

    Complete

    Overall assessment

    Balanced

    The performance criteria should provide a balanced examination of the overall performance story.

    The performance criteria should reflect a balance of measurement types (effectiveness and efficiency), bases (quantitative and qualitative) and timeframes (short, medium and long-term).

    Collective

    The performance criteria should collectively address the purpose.

    The performance criteria should demonstrate the extent of achievement against the purpose through the activities identified in the corporate plan.

           

    Footnotes

    1 ARENA was part of the Environment and Energy portfolio prior to machinery of government changes effective 1 February 2020.

    2Australian Renewable Energy Agency Act 2011 (ARENA Act), section 3.

    3 ARENA Act, section 64. The original version of the Australian Renewable Energy Agency Act 2011 that received Royal Assent on 4 December 2011 provided ARENA with payments of up to $292.5 million in 2012–13. The current version of the Act, as amended, provides ARENA with access to $1.936 billion over the period 2013–14 to 2021–22.

    4 ARENA Act, subsection 8 (a) and (c).

    5 ARENA, Knowledge and Innovation [Internet], available from https://arena.gov.au/knowledge-innovation/ [accessed 15 February 2020].

    6 United Nations Framework Convention on Climate Change, Article 2 — Objective.

    7 Department of the Environment and Energy, Australia’s 7th National Communication on Climate Change: A report under the United Nations Framework Convention on Climate Change – December 2017, pp. 11 and 217.

    8 Renewable energy is produced from sustainable resources such as wind, solar, geothermal and bioenergy. Sub-section 17(1) of the Renewable Energy (Electricity) Act 2000 identifies at least 19 sources that are eligible to produce renewable energy certificates.

    9 Auditor-General Report No. 18 of 2018–19 Administration of the Renewable Energy Target.

    10 An ANAO performance audit of the Clean Energy Finance Corporation’s investments commenced in January 2020.

    11 Auditor-General Report No. 14 of 2016–17 Abatement Crediting and Purchasing under the Emissions Reduction Fund.

    12 ARENA was part of the Environment and Energy portfolio prior to machinery of government changes that took effect on 1 February 2020.

    13 ARENA Act, section 3.

    14 ARENA Act, section 64. This $2.2 billion figure is made up of $292.5 million in 2012–13 provided through the original version of the ARENA Act that received Royal Assent on 4 December 2011 and the $1.936 billion from 2013–14 to 2021–22 provided through the current version of the ARENA Act as at January 2020. The funding provided under section 64 is to be used to provide financial assistance and meeting ARENA’s expense in discharging its functions (ARENA Act, section 67).

    15 ARENA Act, subsections 8(a) and (c).

    16 The ARENA Act defines financial assistance as grants, or any other types of assistance specified by the Minister by legislative instrument.

    17 PGPA Act, section 12 and division 2.

    18 ARENA Act, part 3.

    19 ibid., sections 50–61.

    20 ibid., section 62.

    21 ibid., section 63.

    22 The CEFC is subject to an ANAO audit, due to table in August 2020. The objective of the audit is to assess the effectiveness of the selection, contracting and ongoing management of investments by the CEFC and the extent to which the CEFC is meeting its legislated objective.

    23 Department of Finance, Resource Management Guide No. 132 — Corporate plans for Commonwealth entities, January 2017, paragraph 4.

    24 ibid., p. 8.

    25 ibid., paragraph 47.

    26 Department of Finance, Resource Management Guide 131 — Developing good performance information, April 2015, Part 2 — Understanding purposes and activities.

    27 Department of Finance, Resource Management Guide No. 132 — Corporate plans for Commonwealth entities, January 2017, p. 19.

    28 ARENA, ARENA Corporate Plan 2018/19 – 2020/21, p. 13.

    29 ARENA Act, section 3 and section 8.

    30 Department of Finance, Resource Management Guide 131 — Developing good performance information, April 2015, paragraphs 40–44.

    31 The Commonwealth performance framework consists of the Public Governance, Performance and Accountability Act 2013 (PGPA Act), the accompanying Public Governance, Performance and Accountability Rule 2014 (PGPA Rule) and guidance issued by the Department of Finance.

    Department of Finance, Resource Management Guide 131 — Developing good performance information, April 2015, p. 10.

    32 Department of Finance, Resource Management Guide 132 — Corporate plans for Commonwealth entities, January 2017, paragraph 29.

    33 Department of Finance, Resource Management Guide 131 — Developing good performance information, April 2015.

    34 These reference points included: the Framework for Assurance Engagements and the Australian Standards on Assurance Engagements 3000 and 3500, issued by the Australian Auditing and Assurance Standards Board; frameworks for conducting audits of performance information of national and international public sector audit organisations, including the Offices of the Auditor General for Western Australia and the Auditor-General of New Zealand; and the Framework for the Preparation and Presentation of Financial Statements issued by the Australian Accounting Standards Board.

    35 ARENA was part of the Environment and Energy Portfolio in 2018–19.

    36 ARENA, ARENA Corporate Plan 2018/19 – 2020/21, p. 15.

    37 Department of Finance, Resource Management Guide 131 — Developing good performance information, April 2015 defines performance measures as ‘mechanisms used by entities to generate performance information relating to the efficiency and effectiveness of their activities in pursuing their purpose(s)’. Impacts are defined as the ‘ultimate difference made by fulfilling a purpose defined in an entity’s corporate plan’.

    38 Auditor-General Report No. 17 of 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraph 3.34.

    39 Auditor-General Report No. 17 of 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraphs 3.38 and 1.8.

    40 Auditor-General Report No. 17 of 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraph 3.67.

    41 Auditor-General Report No. 17 of 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraphs 3.67–3.68.

    42 The performance information hierarchy consists of three main categories of information: Accountability information (Outcomes and Impacts); Strategic/tactical information (Outputs and Activities); and Management information (Inputs). The hierarchy is visually depicted in Figure 1.2 of Auditor-General Report No. 17 of 2018–19 Implementation of the Annual Performance Statements Requirements 2017–18 and is based on Department of Finance, Resource Management Guide No. 131 — Developing Good Performance Information, figure 2.

    43 Department of Finance, Resource Management Guide No. 131 — Developing Good Performance Information, paragraph 32.

    44 ibid., paragraph 2.52.

    45 ARENA, Governance and Internal Control Framework, version 3.0, May 2019, p. 4.

    46 ARENA, Financial Assistance Manual, ARENA Policy No.11, October 2017, pp. 7–8.

    47 CGRGs, paragraphs 11.1–11.2.

    48 ARENA, Advancing Renewables Program: Program Guidelines 2017, paragraph 3.1.

    49 ARENA, Advancing Renewables Program: Program Guidelines 2017, pp. 10–12.

    50 CGRGs, paragraph 11.2.

    51 For example, see Auditor-General Report No. 10 of 2016–17 Award of Funding under the Mobile Black Spot Programme, paragraphs 2.39 and 3.47–3.51.

    52 For example, see Auditor-General Report No. 11 of 2015–16 Indigenous Home Ownership Program, paragraphs 2.9–2.12, 3.13 and 4.27, and Auditor-General Report No. 44 of 2018–19 Effectiveness of the Export Finance and Insurance Corporation paragraphs 2.12–2.25.

    53 This research asked the question ‘Ultimately, do you believe your initiative would have been successful without ARENAs involvement?’ Eighty-four responses were received from individuals who identified as representing successful or terminated funding recipients.

    54 ARENA General Funding Strategy 2018–19, p. 9. The ARENA Act, section 10 requires that all grants are awarded in accordance with the General Funding Strategy.

    55 ARENA Corporate Plan 2018–19, pp. 16–17.

    56 Prior to ARENA’s creation on 1 July 2012, the Emerging Renewables Program was administered by the Australian Centre for Renewable Energy (ACRE). ACRE designed the Emerging Renewables Program through industry consultation and analysis. See Australian Centre for Renewable Energy, Strategic Directions for the Australian Centre For Renewable Energy, 2011.

    57 ARENA, Investment Plan 2013/14 — 2015/16 [Internet], available from https://arena.gov.au/assets/2017/01/2013-14-2015-16-Investment-Plan.pdf [accessed February 2020].

    58 ARENA, AGL Solar Project [Internet], available from https://arena.gov.au/projects/agl-solar-project/ [accessed February 2020].

    59 ARENA, Moree Solar Farm [Internet], available from https://arena.gov.au/projects/moree-solar-farm/ [accessed February 2020].

    60 ARENA, Investment Plan July 2015 [Internet], https://arena.gov.au/assets/2017/05/ARENA-Investment-Plan-July-2015.pdf [accessed February 2020].

    61 ARENA, ARENA’s perfect score in Large Scale Solar, 12 from 12 [Internet], available from https://arena.gov.au/news/arenas-perfect-score-large-scale-solar-12-12/ [accessed February 2020].

    62 ARENA, ARENA Corporate Plan 2019/20 — 2022/23 [Internet], available from https://arena.gov.au/assets/2019/08/arena-corporate-plan-2019-20.pdf [accessed February 2020], p. 18.

    63 CGRGs, paragraphs 11.5 and 13.10–13.11.

    64 Section 25 of the ARENA Act requires ministerial approval of guidelines for financial assistance in excess of $15 million.

    65 The Advancing Renewables Program Guidelines 2017 paragraph 4.6 states a one-stage process is permitted where: the activity for which funding is sought is the production of a study or report; the amount of ARENA grant funding sought is less than $500,000 (GST exclusive); or ARENA has waived the requirement.

    66 ARENA waived the requirement for 12 of 104 Advancing Renewables Program applications in 2019. Reasons for waiving included experienced proponents with previously funded projects and proposals at an advanced stage of development. Two of the three competitive rounds reviewed in sample testing used the two-stage process. These were the Research and Development Rounds 3: Solar R&D and 4: Hydrogen for Export.

    67 The members of ARENA’s advisory panel are not ARENA officials. The panel’s role includes assessing funding applications and providing specialist advice to ARENA. For more information on the panel see ARENA, Advisory Panel [internet], available from https://arena.gov.au/about/advisory-panel/ [accessed February 2020].

    68 This analysis treats two-stage assessments of an expression of interest and a full application as a single potential project. The projects that were approved by ARENA’s predecessor organisations are excluded from this analysis.

    69 The sample tested by the ANAO was made up of 44 randomly selected full grant applications received by ARENA between January 2017 and August 2019. These 44 applications represent around 15 per cent of all full applications submitted between January 2017 and August 2019.

    70 The ARENA board has established delegations that define the decision-makers for each stage in the application process: eligibility and merit assessment approval is delegated by the board to executives and senior managers; the funding approval decision-maker is the board for applications over $1 million. The Chief Executive Officer (CEO) holds a delegation from the board to approve projects under $1 million; and the execution decision is delegated to the CEO and to the Chief Financial Officer for projects under $1 million.

    71 In these two cases, the merit assessment report identified potential sources of funding other than ARENA, such as university funding or raising sufficient capital from private investors. These other sources of funding were not ruled out in the merit assessment report or in subsequent briefs for approval.

    72 See paragraphs 2.32–2.34 for a description of ARENA’s internal grants management framework.

    73 CGRGs, paragraph 10.7.

    74 CGRGs, paragraphs 9.4, 10.3, 10.4, 12.8, 12.10 and 12.12.

    75 The 44 grants funding agreements in the sample were randomly selected from the 290 managed by ARENA between 1 January 2017 and August 2019. The 290 funding agreements the sample was selected from had an ARENA Grants Management System project end date after 31 December 2016 and the status of active and its sub-statuses terminated or closed on 16 August 2019. This sample of 44 funding agreements is not related to the sample of 44 full applications analysed in Chapter 2 (refer footnote 69).

    76 The Consolidated Revenue Fund is established by section 81 of the Constitution. It consists of all revenues and moneys raised or received by the executive government of the Commonwealth. Part 5 of the ARENA Act specifies the payments that ARENA can request the Commonwealth pay to it from the Consolidated Revenue Fund.

    77 Australian Government, Portfolio Budget Statements 2019–20: Budget Related Paper No. 1.6: Environment and Energy Portfolio, Commonwealth of Australia, 2019, p. 104.

    78 ARENA, ARENA Corporate Plan 2019/20 –2022/23, August 2019, p. 11.

    79 ARENA’s variations are examined at paragraphs 3.23–3.30.

    80 Total potential recoupable amount is greater than $400.5 million as seven of these mechanisms have no maximum limit on recoupment.

    81 The December 2019 board meeting was the last meeting held before ANAO completed its fieldwork.

    82 Commonwealth Grants Rules and Guidelines, paragraph 11.2.

    83 The term ‘system-based process’ refers to a process that is performed on an information system with electronic workflows, controls and record keeping. Well-designed system-based processes increase consistency and reduce the risk of error compared to unstructured document- (email or spreadsheet) or paper- (print, sign and scan) based processes.

    84 When approved the records of decision were signed, scanned and stored on official hard copy project files. The supporting evidence was hyperlinked in the decision document that had been provided to the decision-maker electronically. When printed, the hyperlinks did not allow the supporting documentation to be validated as the printed hyperlink did not contain enough information to identify the supporting evidence provided to the delegate at the time of approval. The ANAO examined both ARENA’s working electronic files and official hard copy files.

    85 This includes the application assessment activities covered in Chapter 2, paragraphs 2.28–2.63.

    86 Business processes based on spreadsheets and word processor documents lack structure and formalised change controls. A lack of structure makes it difficult to ensure consistent information is presented to decision-makers. Without controls, these processes are unreliable for corporate data handling. Accidental or deliberate changes can be made to formulae and data, without there being a record of when, by whom and what change was made.

    See Auditor-General Report No. 17 of 2018–19 Implementation of the Annual Performance Statements Requirements 2017–18, paragraphs 4.33 to 4.35 and Auditor-General Report No. 20 of 2018–19 2017–18 Major Projects Report, paragraphs 1.50 to 1.56 for similar issues at other Commonwealth entities.

    87 The variation tracking module is not a system for approving variations and variations continue to be managed and approved outside the grant management system.

    88 The grants management system did not record variations prior to the July 2019. ARENA’s use of grant management information systems was examined in paragraphs 3.17–3.19. See also Recommendation no.3.

    89 The ANAO’s ability to provide assurance that the remaining 18 projects in the sample had not been varied was limited by ARENA’s use of paper official files and the lack of a system-based electronic approval and record keeping processes for variations.

    90 The ARENA board has established delegations that define the decision-makers for variation approvals. These delegations only apply to variations that do not significantly affect a project’s outcomes, key conditions precedent or would otherwise warrant the board’s attention:

    • the CEO can approve variations that increase funding by up to $1 million or decrease it by up to $3 million; and
    • the CFO, senior executives and senior managers hold variation approval delegations proportionate to the official’s seniority, the value of the variation and the project’s risk rating.

    91 As with ARENA’s grant funding and milestone decision-making, ARENA’s variation approvals process is document-based, with variations approved via email or hard copy documents presented to the decision-maker. The non-system-based risks discussed in paragraphs 3.12–3.19 also apply to ARENA’s variations process.

    92 The ANAO considered broad statements to the delegate such as ‘the requested variation will not impact the overall outcomes or time frames’ that were not supported with analysis as insufficient to inform these considerations.

    93 See paragraphs 3.38–3.42 for more information on ARENA’s project risk assessment process.

    94 See paragraphs 3.38–3.42 for more information on ARENA’s project risk assessment process.

    95Public Governance, Performance and Accountability Act 2013 section 16. The Commonwealth Risk Management Policy and related documents provide a risk management framework for Commonwealth entities.

    The Commonwealth Risk Management Policy defines risk as the effect of uncertainty on objectives. An effect is a deviation from the expected — positive and/or negative. Risk is often expressed in terms of a combination of the consequences of an event (including changes in circumstances or knowledge) and the associated likelihood of occurrence. If the effect of a risk is realised, it becomes an issue.

    96 See paragraph 3.33.

    97 Joint Committee of Public Accounts and Audit, Parliament of Australia, Report 472: Commonwealth Procurement—Second Report, Inquiry based on Auditor-General’s reports 9 and 12 (2017–18) and (2016–17), October 2018.

    98 CGRGs, section 13.

    99 See Auditor-General Report No. 5 of 2019–20 Australian Research Council’s Administration of the National Competitive Grants Program, paragraphs 3.35 to 3.40 for an example of a system-based conflict of interest management that is fully embedded though a grants assessment process.

    100 Eight of 3,201 applications does not indicate an issue with the process. Panel members make an initial document-based declaration of interests before being assigned the application on the grant management system. This results in a low number of conflicts for assigned assessments in the system as this is a secondary check.

    101 See paragraph 3.17.

    102 See paragraph 2.33.

    103 The 2018–19 Performance Framework Guide was marked as draft when supplied to ANAO in July 2019.

    104 The schedule does not indicate the proportion of ARENA grant funding that will be covered by an evaluation.

    105 See paragraphs 2.11–2.25.

    106 Auditor-General Report No. 17 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraph 4.71.

    107 ibid., paragraph 4.72.

    108 Section 17 of the PGPA Rules, Functions of the audit committee:

    (1) The accountable authority of a Commonwealth entity must, by written charter, determine the functions of the audit committee for the entity.

    (2) The functions must include reviewing the appropriateness of the accountable authority’s: financial reporting; performance reporting; system of risk oversight and management; and system of internal control for the entity.

    109 Department of Finance, A guide for corporate Commonwealth entities on the role of audit committees, Resource Management Guide 202, December 2018, section 1.2.2.

    110 The 2018–19 Performance Measurement Framework was approved by the ARENA Board at its June 2018 meeting. The paper seeking the Board’s approval did not provide assurance from either the RAC or management that ARENA’s 2018–19 Performance Framework was consistent with the Commonwealth performance framework.

    111 Auditor-General Report No. 36 2017–18, Corporate Planning in the Australian Public Sector 2017–18, paragraph 2.104.

    112 See Table 2.1.

    113 The period of the analysis covered January 2017 to November 2019. Key governance bodies were defined as: ARENA’s Executive Leadership Team and the Executive Committee; ARENA’s accountable authority — the ARENA board; and the ARENA Risk and Audit Committee.

    114 ARENA Act, subsection 8(c).

    115 As noted in Chapter 2, Table 2.3, ARENA’s performance measures include knowledge sharing performance criteria, but do not set specific targets. Setting publically available targets for the number of documents to be published and downloaded would improve stakeholders’ understanding of ARENA’s effectiveness in delivering its knowledge sharing functions.

    116 Not all knowledge sharing material is suitable to be published on ARENA’s website. Some material is commercial in confidence and other knowledge sharing occurs through unstructured methods such as informal networks.

    117 Paragraphs 2.37–2.42.

    118 ARENA, Publications [Internet], available from https://arena.gov.au/about/publications/ [accessed 17 December 2019]. ARENA, Funding [Internet], available from https://arena.gov.au/funding/ [accessed 17 December 2019].

    119 Australian Renewable Energy Agency, Annual Report 2018–19, pp. 186–188.

    120 Table 2.3 presents a summary of ARENA 2018–19 corporate plan performance framework.

    121 ARENA, Annual Report 2018–19, pp. 111–123.

    122 Auditor-General Report No. 17 2018–19, Implementation of the Annual Performance Statements Requirements 2017–18, paragraph 3.51.

    123 See paragraphs 2.11–2.25.

    124In the spotlight: Australian solar energy research and development outcomes and achievements in a global context: A review of ARENA’s portfolio of solar research and development [internet], July 2018, available from https://arena.gov.au/assets/2018/10/In-the-spotlight-Australian-solar-energy-RD-outcomes-and-achievements-in-a-global-context.pdf [accessed 17 December 2019].

    Distributed Energy Resources (DER) Integration Project Contribution Assessment [internet], March 2019, available from https://arena.gov.au/assets/2019/11/distributed-energy-resources-integration-project-contribution-assessment.pdf [accessed 25 February 2020].

    125 ARENA, ARENA Conflicts of Interests Policy, version 3.0, November 2018, p. 3.

    126 Australian Energy Market Regulator, Residential electricity price trends 2019 [Internet], December 2019, p. 6 and 11, available from https://www.aemc.gov.au/market-reviews-advice/residential-electricity-price-trends-2019 [accessed 12 December 2019]. The analysis supporting this report excluded the Northern Territory, p. 6.

    127 Department of the Environment and Energy, Australia’s emissions projects 2019 [Internet], December 2019, p. 1 and 11, available from https://www.environment.gov.au/climate-change/publications/emissions-projections-2019 [accessed 12 December 2019].

    128 See paragraph 3.36.

    129 See paragraphs 2.66, 3.15–3.19 and Recommendation no.3 at paragraph 3.20.