The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected major projects. The status of the selected major projects is reported in the Statement by the Secretary of Defence and the Project Data Summary Sheets (PDSSs) prepared by Defence. Assurance from the ANAO’s review is conveyed in the Independent Assurance Report by the Auditor-General.

Due to the complexity of material and the multiple sources of information for the 2018–19 Major Projects Report, we are unable to represent the entire document in HTML. You can download the full report in PDF or view selected sections in HTML below. PDF files for individual Project Data Summary Sheets (PDSS) are also available for download.

!Part 1. ANAO Review and Analysis

.

Summary and Review Conclusion

About the Major Projects Report

1. Major Defence equipment acquisition projects (Major Projects) continue to be the subject of parliamentary and public interest. This is due to their high cost and contribution to national security, and the challenges involved in completing them within the specified budget and schedule, and to the required capability.

2. The Australian National Audit Office (ANAO) has reviewed 26 of Defence’s Major Projects in this twelfth annual report (2017–18: 26). The Major Projects Report (MPR) reviews overall issues, risks, challenges and complexities affecting Major Projects and also reviews the status of each of the selected Major Projects, in terms of cost, schedule and forecast capability.1 The objective of the report is ‘to improve the accountability and transparency of Defence acquisitions for the benefit of Parliament and other stakeholders.’2

3. The Capability Acquisition and Sustainment Group (CASG) within the Department of Defence (Defence), manages the process of bringing new capabilities into service.3 As at 30 June 2019, Defence was managing 205 active major and minor capital equipment projects worth $132 billion, with an in-year budget of $8.6 billion.4 Defence capitalised some $8.6 billion from these projects in 2018–19.5

4. The February 2016 Defence White Paper established the Australian Government’s priorities for future capability investment for the next 20 years and provided for additional spending of over $29 billion across the next decade. The 2019–20 Defence Portfolio Budget Statements indicated that the Defence budget would grow to approximately $200 billion over the coming decade, for investing in Defence capability.6 The Government commenced its $89 billion investment in Australia’s future shipbuilding industry in April 20177, and on 29 June 2018 announced Second Pass Approval8 of the $35 billion Future Frigate program.9 Further, on 11 February 2019, the Government announced the signing of the Strategic Partnership Agreement10 for the $50 billion Future Submarine program.11

Major Projects selected for review

5. Major Projects are selected for review based on the criteria included in the 2018–19 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).12 They represent a selection of the most significant Major Projects managed by Defence.

6. The total approved budget for the Major Projects included in this report is approximately $64.1 billion, covering 49 per cent of the total budget of active major and minor capital equipment projects of $132 billion.13 The selected projects are listed in Table 1.

Table 1: 2018–19 MPR projects and approved budgets at 30 June 2019 1, 2

Project Number (Defence Capability Plan)

Project Name (on Defence advice)

Abbreviation (on Defence advice)

Approved Budget $m

AIR 6000 Phase 2A/2B

New Air Combat Capability 3

Joint Strike Fighter

16,522.6

SEA 4000 Phase 3

Air Warfare Destroyer Build 3

AWD Ships

9103.7

AIR 7000 Phase 2B

Maritime Patrol and Response Aircraft System

P-8A Poseidon

5375.7

AIR 9000 Phase 2/4/6

Multi-Role Helicopter 3

MRH90 Helicopters

3771.1

SEA 1180 Phase 1

Offshore Patrol Vessel 3

Offshore Patrol Vessel

3724.3

AIR 5349 Phase 3

EA-18G Growler Airborne Electronic Attack Capability

Growler

3510.3

LAND 121 Phase 3B

Medium Heavy Capability, Field Vehicles, Modules and Trailers 3

Overlander Medium/Heavy

3399.9

AIR 9000 Phase 8

Future Naval Aviation Combat System Helicopter

MH-60R Seahawk

3212.5

JP 2048 Phase 4A/4B

Amphibious Ships (LHD)

LHD Ships

3092.2

LAND 121 Phase 4

Protected Mobility Vehicle – Light (PMV-L) 3

Hawkei

1979.6

AIR 8000 Phase 2

Battlefield Airlift – Caribou Replacement 3

Battlefield Airlifter

1442.1

SEA 1654 Phase 3

Maritime Operational Support Capability

Repl Replenishment Ships

1070.6

AIR 5431 Phase 3

Civil Military Air Management System 3

CMATS

975.8

JP 2072 Phase 2B

Battlespace Communications System Phase 2B

Battle Comm. Sys. (Land) 2B

942.6

AIR 7403 Phase 3

Additional KC-30A Multi-role Tanker Transport

Additional MRTT

894.3

SEA 1448 Phase 2B

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2B

678.7

SEA 1439 Phase 5B2

Collins Class Communications and Electronic Warfare Improvement Program

Collins Comms and EW

607.8

SEA 3036 Phase 1

Pacific Patrol Boat Replacement

Pacific Patrol Boat Repl

504.0

JP 9000 Phase 7

Helicopter Aircrew Training System

HATS

481.6

SEA 1439 Phase 3

Collins Class Submarine Reliability and Sustainability 4

Collins R&S

445.3

LAND 53 Phase 1BR

Night Fighting Equipment Replacement

Night Fighting Equip Repl

442.6

SEA 1442 Phase 4

Maritime Communications Modernisation

Maritime Comms

440.0

JP 2072 Phase 2A

Battlespace Communications System Phase 2A

Battle Comm. Sys. (Land) 2A

438.1

SEA 1448 Phase 4B

ANZAC Air Search Radar Replacement

ANZAC Air Search Radar Repl

428.7

JP 2008 Phase 5A

Indian Ocean Region UHF SATCOM

UHF SATCOM

421.8

JP 2048 Phase 3

Amphibious Watercraft Replacement

LHD Landing Craft

236.7

Total

26

 

 

64,142.6

     

Note 1: Once a project is selected for review, it remains within the portfolio of projects under review until the JCPAA endorses its removal, normally once it has met the capability requirements of Defence.

Note 2: SEA 1180 Phase 1 Offshore Patrol Vessel, SEA 1439 Phase 5B2 Collins Class Communications and Electronic Warfare Improvement Program, LAND 53 Phase 1BR Night Fighting Equipment Replacement and SEA 1448 Phase 4B ANZAC Air Search Radar Replacement are included in the MPR for the first time in 2018–‍19.

Note 3: These projects have been the subject of individual performance audits. See Table 8 for more information.

Note 4: SEA 1439 Phase 3 Collins Class Submarine Reliability and Sustainability is a group of 24 activities primarily sustainment in nature. While not an acquisition project, it has been included at the JCPAA’s request.

Source: The Project Data Summary Sheets in Part 3 of this report.

7. In September 2019, the JCPAA endorsed project selection for the 2019–20 MPR, including the entry of five new projects.14 The ANAO advised the JCPAA that it proposed to consider the exit of seven projects from the 2018–19 MPR that were expected to reach FOC by the end of 2019 and/or to confirm that only low risk deliverables were remaining.15 Six of these projects are now considered suitable to exit on the basis that they:

  • have achieved FOC (SEA 1448 Phase 2B ANZAC ASMD 2B);
  • will achieve FOC in December 2019 (JP 2072 Phase 2A Battle Comm. Sys. (Land) 2A); or
  • have provided a pre-FOC risk assessment supporting the timely declaration of FOC (JP 2048 Phase 4A/4B LHD Ships, AIR 7403 Phase 3 Additional MRTT, JP 9000 Phase 7 HATS, and JP 2048 Phase 3 LHD Landing Craft).

8. In the case of one project, AIR 8000 Phase 2 Battlefield Airlifter, the project’s FOC has been delayed and it is not considered to be suitable for exit yet (see paragraph 2.54).

Report objective and scope

9. The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects. The status of the selected Major Projects is reported in the Statement by the Secretary of Defence (see Part 3 of this report) and the Project Data Summary Sheets (PDSSs) prepared by Defence (see Part 3 of this report). Assurance from the ANAO’s review is conveyed in the Independent Assurance Report by the Auditor-General (see Part 3 of this report).

10. The following forecast information found in the PDSSs is excluded from the scope of the ANAO’s review:

  • Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  • Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  • forecast dates where included in each PDSS.

Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information are required to be considered in forming the conclusion.

11. The exclusions to the scope of the review, noted above, are due to a lack of Defence systems from which to provide complete and accurate evidence16 in a sufficiently timely manner to facilitate the review. This has been an area of focus of the JCPAA over a number of years17, and it is intended that all components of the PDSSs will eventually be included within the scope of the ANAO’s review.

12. Separate to the formal review, the ANAO has undertaken an analysis of key elements of the PDSSs — including cost, schedule, progress towards delivery of required capability, project maturity, and risks and issues. Longitudinal analysis across these key elements of projects has also been undertaken.

13. Defence provides further insights and context in its commentary and analysis. This commentary and analysis is not included within the scope of the ANAO’s review.

Review methodology

14. The ANAO has reviewed the PDSSs prepared by Defence as a priority assurance review under subsection 19A(5) of the Auditor-General Act 1997. The criteria to conduct the review are provided by the Guidelines approved by the JCPAA, and include whether Defence has procedures in place designed to ensure that project information and data was recorded in a complete and accurate manner, for all 26 projects.

15. The review included an assessment of Defence’s systems and controls, including the governance and oversight in place, to ensure appropriate project management. The ANAO also sought representations and confirmations from Defence senior management and industry in relation to the status of the Major Projects in this report.

Report structure

16. The report is organised into four parts:

  • Part 1 comprises the ANAO’s review and analysis (pp. 1–66);
  • Part 2 comprises Defence’s commentary, analysis and appendices (not included within the scope of the Independent Assurance Report by the Auditor-General) (pp. 67–122);
  • Part 3 incorporates the Independent Assurance Report by the Auditor-General, the Statement by the Secretary of Defence, and the PDSSs prepared by Defence as part of the assurance review process (pp. 123–398); and
  • Part 4 reproduces the 2018–19 Major Projects Report Guidelines endorsed by the JCPAA, which provide the criteria for the compilation of the PDSSs by Defence and the ANAO’s review (pp. 399–426).

Figure 1, below, depicts the four parts of this report.

Figure 1: 2018–19 Report structure

Figure 1

Note: To assist in conducting inter-report analysis, the presentation of data in the PDSSs remains largely consistent and comparable with the 2017–18 MPR.

Project Data Summary Sheets

17. The PDSSs include unclassified information on project performance, prepared by Defence.18 As projects appear in the MPR for multiple years, changes to the PDSS from the previous year are depicted in bold orange text in the PDSS.

18. Each PDSS comprises:

  • Project Header: including name; capability and acquisition type; Capability Manager; approval dates; total approved and in-year budgets; stage; complexity; and an image;
  • Section 1—Project Summary: including description; current status including financial assurance and contingency statement; and context, including background, uniqueness, major risks and issues, and other current sub-projects;
  • Section 2—Financial Performance: including budgets and expenditure; variances; and major contracts in place (in addition to quantities delivered as at 30 June 2019);
  • Section 3—Schedule Performance: providing information on design development; test and evaluation; and forecasts and achievements against key project milestones, including Initial Materiel Release (IMR), Final Materiel Release (FMR)19, Initial Operational Capability (IOC) and Final Operational Capability (FOC)20;
  • Section 4—Materiel Capability Delivery Performance: provides a summary of Defence’s assessment of its expected delivery of key capabilities, the extent to which milestones were achieved (particularly where caveats are placed on the Capability Manager’s declaration of significant milestones), and a description of the constitution of each key milestone;
  • Section 5—Major Risks and Issues: outlines the major risks and issues of the project and remedial actions undertaken for each;
  • Section 6—Project Maturity: provides a summary of the project’s maturity, as defined by Defence21, and a comparison against the benchmark score;
  • Section 7—Lessons Learned: outlines the key lessons that have been learned at the project level (further information on lessons learned by Defence are included in Defence’s Appendix 2 in Part 2 of this report); and
  • Section 8—Project Line Management: details current project management responsibilities within Defence.

Overall outcomes

Statement by the Secretary of Defence

19. The Statement by the Secretary of Defence was signed on 10 December 2019. The Acting Secretary’s statement provides her opinion that the PDSSs for the 26 selected projects ‘comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2019’.

20. In addition, the Statement by the Secretary of Defence details significant events occurring post 30 June 2019, which materially impact the projects included in the report, and which should be read in conjunction with the individual PDSSs. These include: Joint Strike Fighter, P-8A Poseidon, MRH90 Helicopters, Offshore Patrol Vessel, Growler, LHD Ships, Hawkei, Battlefield Airlifter, Repl Replenishment Ships, CMATS, Additional MRTT, Collins Comms and EW, Night Fighting Equip Repl, Maritime Comms, ANZAC Air Search Radar Repl, UHF SATCOM, and LHD Landing Craft.

21. The 2018–19 MPR Guidelines require Defence to report in the Statement by the Secretary of Defence on projects which have been removed from the MPR which still have outstanding caveats. Defence has not reported any outstanding caveats in the 2018–19 Statement by the Secretary of Defence.

Conclusion by the Auditor-General

22. The Auditor-General has concluded in the Independent Assurance Report for 2018–19 that ‘nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2018–19 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.’

23. Additionally, in 2018–19, a number of observations were made in the course of the ANAO’s review, as summarised below:

  • Defence is unable to provide point-in-time data on project personnel numbers and costs. See further explanation in paragraphs 1.53 to 1.55;
  • instances of a lack of oversight, non-compliance with corporate guidance and the use of spreadsheets in the management of risks and issues (Section 5 of the PDSS). See further explanation in paragraphs 1.56 to 1.6222;
  • outdated policy guidance for the project maturity framework (Section 6 of the PDSS). See further explanation in paragraphs 1.63 to 1.6723; and
  • an increase in the number of MPR projects which have achieved significant milestones with caveats. See further explanation in paragraphs 1.68 to 1.73.

ANAO’s analysis of project performance

24. As discussed, the ANAO has undertaken an analysis of key elements of the Defence PDSSs — cost, schedule, progress towards delivery of required capability, project maturity, risks and issues, and longitudinal analysis across these key elements of projects. Table 2 provides: summary data on Defence’s progress toward delivering the capabilities for the Major Projects covered in this report; and compares current data against that reported in previous editions of the MPR. This section also contains a summary analysis of the three principal components of project performance: cost, schedule and capability.

Table 2:  Summary longitudinal analysis1

 

2016–17 MPR

2017–18 MPR

2018–19 MPR

Number of Projects

27

26

26

Total Approved Budget at 30 June

$62.0 billion

$59.4 billion

$64.1 billion

Total Approved Budget at final Second Pass Approval

$53.5 billion

$50.2 billion

$53.9 billion

Total Expenditure Against Total Approved Budget

$32.1 billion (51.7%)

$32.4 billion (54.5%)

$36.3 billion (56.6%)

Total In-year Expenditure Against In-year Budget

$4.1 billion (96.6%)

$4.6 billion (98.6%)

$4.8 billion (93.4%)

Total Budget Variation since initial Second Pass Approval 2

$22.3 billion (36.0%)

$23.0 billion (38.7%)

$24.4 billion (38.0%)

Total Budget Variation since final Second Pass Approval 3

$8.5 billion (13.7%)

$9.2 billion (15.5%)

$10.2 billion (15.9%)

In-year Approved Budget Variation

-$1.6 billion (-2.6%)

-$0.3 billion (-0.5%)

$1.2 billion (1.9%)

Total Schedule Slippage 4

793 months (29%)

801 months (32%)

691 months (27%)

Average Schedule Slippage per Project

30 months

32 months

27 months

In-year Schedule Slippage 5

149 months (6%)

104 months (5%)

108 months (5%)

Total Project Maturity 6

1531 / 1890 (81%)

1484 / 1820 (82%)

1485 / 1820 (82%)

Total Reported Risks and Issues 7, 8

136

138

138

Expected Capability (Defence Reporting)

  • High level of confidence of delivery (Green)

98%

99%

99%

  • Under threat, considered manageable (Amber)

2%

1%

1%

  • Unlikely to be met (Red)

0% 9

0%

10

    

Refer to paragraphs 24 to 39 in Part 1 of this report.

Note 1: The data for the 26 Major Projects in the 2018–19 MPR compares the data from projects in the 2017–18 MPR and 2016–17 MPR. The Major Projects included within each MPR are based on entry and exit criteria in the Guidelines, which have been included in Part 4 of this report. The entry and exit of projects should be considered when comparing data across years.

Note 2: Where a project has multiple Second Pass Approvals (see footnote 8), the MPR has historically reported budget variations from the initial Second Pass Approval. The figures in this row are consistent with prior year reporting. See Table 3 for a breakdown of the major components of this variance, and Table 8 for all real variations.

Note 3: In the 2017–18 and 2018–19 PDSSs, where a project has multiple Second Pass Approvals, the budget at Second Pass Approval reported in the Header refers to the total budget as at the final Second Pass Approval. The figures in this row use this methodology.

Note 4: Slippage refers to the difference between the original government approved date and the current forecast date. Slippage can occur due to late delivery, increases in scope or at times can be a deliberate management decision. These figures exclude schedule reductions over the life of the project. However, paragraph 2.43 reports total schedule reductions over the life of the projects.

Note 5: Based on the 25 repeat projects from the 2015–16 MPR plus one new project (CMATS) that had slippage in 2016–17, 23 repeat projects from the 2016–17 MPR, and 22 repeat projects from the 2017–18 MPR respectively.

Note 6: The figures represent the total of the reported maturity scores divided by the total benchmark maturity score, in the PDSSs across all projects.

Note 7: The grey section of the table is excluded from the scope of the ANAO’s priority assurance review, due to a lack of systems from which to obtain complete and accurate evidence in a sufficiently timely manner to facilitate the review.

Note 8: The figures represent the combined number of open high and extreme risks and issues reported in the PDSSs across all projects. Risks and issues may be aggregated at a strategic level.

Note 9: Defence advised in this year that Joint Strike Fighter would not deliver one element of capability at FOC (which equated to approximately one per cent). However, across all the Major Projects this percentage rounded to zero per cent.

Note 10: Defence advised in this year that AWD Ships would not deliver one element of capability at FOC (which equated to approximately one per cent). However, across all the Major Projects this percentage rounded to zero per cent.

Cost

25. Cost management is an ongoing process in Defence’s administration of the Major Projects. While all projects reported that they could continue to operate within the total approved budget of $64.1 billion, MRH90 Helicopters, Repl Replenishment Ships, and Battle Comm. Sys (Land) 2B were required to draw upon contingency funds to complete project activities.

26. The approved budget for Major Projects included in this MPR has increased by $24.4 billion (38.0 per cent) since initial Second Pass Approval. Budget variations greater than $500 million are detailed in Table 3, below.24 However, as the MPR predominantly focusses on the approved capital budget for acquisition, the ongoing costs of Project Offices25, training, replacement capability, etc., are not reported here.

Table 3: Budget variation over $500m post initial Second Pass Approval by variation type 1, 2

Project

Variation Type

Explanation

Year

Amount $b

Scope Increases

14.1

MRH90 Helicopters

 

34 additional aircraft at Phase 4/6 Second Pass Approval

2005–06

2.3

 

Joint Strike Fighter

 

58 additional aircraft at Stage 2 Second Pass Approval

2013–14

10.5

 

P-8A Poseidon

 

Four additional aircraft

2015–16

1.3

 

Real Cost and other Increases

1.8

AWD Ships

 

Real Cost Increase of $1.2b offset by $0.1b transfer for facilities in 2014

2013–14 and 2015–16

1.1

 

Overlander Medium/Heavy

 

Project supplementation3 ($684.2m) and additional vehicles, trailers and equipment ($28.0m) at Revised Second Pass Approval

2013–14

0.7

 

Other budget movements

1.3

Other

Scope increase/budget transfers (net)

Other scope changes and transfers

Various

1.3

 

Price Indexation – materials and labour (net) (to July 2010) 4

 

2.8

Exchange Variation – foreign exchange (net) (to 30 June 2019)

 

4.4

Total

24.4

        

Note 1: For the variations related to all projects and values refer to Table 8 of this report. For the breakdown of in-year variation, refer to Table 9 of this report.

Note 2: For projects with multiple Second Pass Approvals, this table shows variations from the initial approval.

Note 3: Defence has advised that ‘project supplementation’ is a unique term used to describe the approvals history of this project as follows: ‘The original amount of $2549.2, was the Government decision to split Phase 3 into Phase 3A and 3B. In 2011, Government approved Second Pass approval of Phase 3A and the ‘Interim Pass’ Government approval for Phase 3B. The decision to grant Phase 3B ‘Interim Pass’ was to allow greater bargaining power for Defence while negotiating Phase 3A. Phase 3B was always going to return to Government for formal Second Pass approval, which occurred in July 2013, once contract negotiations were complete.’

Note 4: Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.

Source: ANAO analysis of the 2018–19 PDSSs.

Schedule

27. Delivering Major Projects on schedule continues to present challenges for Defence, affecting when the capability is made available for operational release and deployment by the Australian Defence Force, as well as the cost of delivery.

28. The total schedule slippage for the 26 selected Major Projects, as at 30 June 2019, is 691 months when compared to the initial schedule.26 This represents a 27 per cent increase since Second Pass Approval. Across MPR projects that have experienced slippage (21 of 26 projects), the average slippage is 32.9 months (2.7 years). Table 4 below includes details of in-year and total schedule slippage by project. The table shows an increase of 108 months of in-year slippage during 2018–19.

29. The total slippage of 691 months in 2018–19 is 110 months lower than the total in 2017–18 of 801 months. This is due to:

  • the removal of completed projects (Collins RCS, Hw Torpedo, ANZAC ASMD 2A and Battle Management System) removing 254 months of slippage from the total reported in 2017–18 (see Table 5);
  • the addition of 108 months of in-year slippage described above; and
  • the Collins Comms and EW project adding 36 months of slippage to the total of 691 months; the slippage occurred in 2016–17 but the project was reported in the MPR for the first time in 2018–19.

Table 4: Schedule slippage from original planned Final Operational Capability 1

Project

In-year (months)

Total (months)

Project

In-year (months)

Total (months)

Joint Strike Fighter 2

0

2

Battle Comm. Sys. (Land) 2B

24

24

AWD Ships

5

40

Additional MRTT

2

23

P-8A Poseidon

1

29

ANZAC ASMD 2B

10

77

MRH90 Helicopters

0

89

Collins Comms and EW

0

36

Offshore Patrol Vessel

0

0

Pacific Patrol Boat Repl

0

2

Growler

1

2

HATS

3

3

Overlander Medium/Heavy

6

11

Collins R&S

0

112

MH-60R Seahawk

0

0

Night Fighting Equip Repl

0

0

LHD Ships 2

0

37

Maritime Comms

6

13

Hawkei 2

0

0

Battle Comm. Sys. (Land) 2A

9

39

Battlefield Airlifter

0

24

ANZAC Air Search Radar Repl

0

0

Repl Replenishment Ships 2,3

7

7

UHF SATCOM 2

21

42

CMATS 2

0

28

LHD Landing Craft

13

51

Total (months)

108

691

Total (%)

5

27

      

Note 1: Refer to footnote 25.

Note 2: These projects have been identified by Defence as Projects of Interest (see paragraph 1.23 in Part 1).

Note 3: Slippage for this project is reflective of movement from a forecast of May 2022 in 2017–18, to a forecast of December 2022 in 2018–19. While the forecast of December 2022 remains within the window for FOC achievement approved by Government, the previous forecast of May 2022 was approved by this project’s line management. See paragraph 2.45.

Source: ANAO analysis of the 2018–19 PDSSs.

30. Platform availability has contributed to the slippage experienced within some projects. For example, Maritime Comms and Collins R&S have been impacted by changes to docking schedules of the Anzac Class frigates and Collins Class submarines respectively. Significant delays have also been experienced by those projects with the most developmental content: AWD Ships, MRH90 Helicopters, CMATS and Battle Comm. Sys. (Land) 2B. Additionally, delays to operational test and evaluation activities have led to delays to the LHD Ships and LHD Landing Craft projects.

31. Table 5, below, provides details of total schedule slippage by project, for projects which have exited the MPR. Compared to the 691 months total schedule slippage for the current 26 Major Projects, the 22 projects which have exited the MPR have reported accumulated schedule slippage of 953 months, as at their respective exit dates. Table 5 indicates that schedule slippage for projects which have exited the MPR was more pronounced in projects with the most developmental content.

Table 5: Schedule slippage for projects which have exited the MPR 1

Project

Total (months)

Project

Total (months)

Wedgetail (Developmental)

78

HF Modernisation (Developmental)

147

Super Hornet (MOTS)

0

Armidales (Australianised MOTS)

45

Hornet Upgrade (Australianised MOTS)

39

Collins RCS (Australianised MOTS)

109

ARH Tiger Helicopters (Australianised MOTS)

82

Hw Torpedo (MOTS)

63

C-17 Heavy Airlift (MOTS)

0

SM-2 Missile (Australianised MOTS)

26

Air to Air Refuel (Developmental)

64

ANZAC ASMD 2A (Australianised MOTS)

82

FFG Upgrade (Developmental)

132

155mm Howitzer (MOTS)

7

Bushmaster Vehicles (Australianised MOTS)

1

Stand Off Weapon (Australianised MOTS)

37

Overlander Light (Australianised MOTS)

9

Battle Comm. Sys. (Australianised MOTS)

24

Next Gen Satellite 2 (MOTS)

0

C-RAM (MOTS)

2

Additional Chinook (MOTS)

6

 

 

Total 

 953

    

Note 1: The Hornet Refurb and Battle Management System (BMS) projects are not included in this table as they did not have FOC milestones.

Note 2: Next Gen Satellite shows slippage in Figure 8, which related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced and slippage was reduced.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

32. Additional ANAO analysis (refer to Figure 7, on page 54) has compared project slippage against the Defence classification of projects as Military Off-The-Shelf (MOTS), Australianised MOTS or developmental. These classifications are a general indicator of the difficulty associated with the procurement process.

33. Figure 8 (on page 55) provides analysis of projects either completed, or removed from the MPR review, and shows that a focus on MOTS27 acquisitions has assisted in reducing schedule slippage. Prima facie, the more developmental in nature a project is, the more likely it will result in a greater degree of project slippage. Figure 8 was requested by the JCPAA in May 2014.28

34. Longitudinal analysis indicates that while the reasons for schedule slippage vary, it primarily reflects the underestimation of both the scope and complexity of work, particularly for Australianised MOTS and developmental projects (see paragraphs 2.31 to 2.37).

Capability

35. The third principal component of project performance examined in this report is progress towards the delivery of capability required by government. While the assessment of expected capability delivery by Defence is outside the scope of the Auditor-General’s formal review conclusion, it is included in the analysis to provide an overall perspective of the three principal components of project performance.

36. The Defence PDSSs report that 20 projects in this year’s report will deliver all of their key capability requirements. Defence’s assessment indicates that some elements of the capability required may be ‘under threat’, but the risk is assessed as ‘manageable’. The five project offices experiencing challenges with expected capability delivery (2017–18: three) are Joint Strike Fighter, MRH90 Helicopters, Hawkei, Battlefield Airlifter and LHD Landing Craft. One project office (AWD Ships) reports that it is unable to deliver all of the required capability by FOC.

37. Defence’s presentation of capability delivery performance in the PDSSs is a forecast and therefore has an element of uncertainty. In 2015–16, the ANAO developed an additional measure of the status of current capability delivery progress to assist the Parliament — Capability Delivery Progress — which is a tally of the capability delivered as at 30 June 2019, as reported by Defence. Table 6 below provides a worked example of the ANAO’s methodology, utilising the performance information provided in the relevant PDSS.

Table 6: Capability Delivery Progress assessment — Additional MRTT (multi-role tanker transport)

Capability elements

as per Section 4.2 of the PDSS

No. of elements approved

No. of elements delivered at 30 June 2019

Comments

Delivery of first aircraft, and delivery of initial spares and support equipment (IMR)

2

2

First aircraft and initial spares delivery completed.

Delivery of second aircraft, delivery of remaining spares and support equipment, and delivery of Aircraft Stores Replenishment Vehicle (FMR)

3

2

Second aircraft and remaining spares and support equipment delivery completed. Aircraft Stores Replenishment Vehicle yet to be delivered.

Total (number)

5

4

 

Total (%)

100

80

    

Source: PDSSs in published Major Projects Reports and ANAO analysis.

38. Table 7 below, summarises expected capability delivery as at 30 June 2019 — as reported by Defence and using the ANAO’s Capability Delivery Progress measure.

Table 7: Capability delivery

Expected Capability

(Defence Reporting)

2016–17 MPR (%)

2017–18 MPR (%)

2018–19 MPR (%)

Capability Delivery Progress

(ANAO Analysis)

2018–19 MPR (%)

2018–19 MPR (%) Adjusted 3

High Confidence (Green)

98

99

99

Delivered

67

54

Under Threat, considered manageable (Amber)

2

1

1

Not yet delivered

33

46

Unlikely (Red)

1

0

2

Not delivered at FOC

2

2

Total

100

100

100

Total

100

100

       

Note 1: Defence advised in this year that Joint Strike Fighter would not deliver one element of capability at FOC (which equated to approximately one per cent of elements required). However, across all the Major Projects this percentage rounded to zero.

Note 2: Defence advised in this year that AWD Ships would not deliver one element of capability at FOC (which equates to approximately one per cent of elements required). However, across all the Major Projects this percentage rounds to zero.

Note 3: While the left-hand column reports the total percentage of elements delivered across all 26 Major Projects, the right-hand adjusted column reports the average percentage of elements delivered per project. This adjustment results in an analysis where all projects have equal weight and the percentage is not affected by the numbers of deliverables per project.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

39. In addition to reporting on expected capability delivery, Defence has continued the practice of including in the PDSSs declassified information on settlement actions for projects, including stop payments and liquidated damages. During 2018–19, Hawkei, Pacific Patrol Boat Repl, and UHF SATCOM had negotiated contractual settlements involving stop payments or liquidated damages. Prior settlements for projects within this report related to MRH90 Helicopters, LHD Ships and Maritime Comms.

1. The Major Projects Review

1.1 This chapter provides an overview of the scope and approach adopted by the Australian National Audit Office (ANAO) for the review of the 26 Project Data Summary Sheets (PDSSs) prepared by the Department of Defence (Defence). This chapter also provides the results of the Major Projects Report (MPR) review.

Review scope and approach

1.2 In 2012 the Joint Committee of Public Accounts and Audit (JCPAA) identified the review of the PDSSs as a priority assurance review, under subsection 19A(5) of the Auditor-General Act 1997 (the Act). This provided the ANAO with full access to the information gathering powers under the Act. The ANAO’s review of the individual project PDSSs, which are reproduced in Part 3 of this report, was conducted in accordance with the auditing standards set by the Auditor-General under section 24 of the Auditor-General Act 1997 through its incorporation of the Australian Standard on Assurance Engagements (ASAE) 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Australian Auditing and Assurance Standards Board.

1.3 The following forecast information is excluded from the scope of the ANAO’s review: capability delivery, risks and issues, and forecast dates. These exclusions are due to the lack of Defence systems from which to provide complete and accurate evidence29, in a sufficiently timely manner to complete the review. Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.

1.4 The ANAO’s work is appropriate for the purpose of providing an Independent Assurance Report in accordance with the above auditing standard. However, the review of individual PDSSs is not as extensive as individual performance and financial statement audits conducted by the ANAO, in terms of the nature and scope of issues covered, and the extent to which evidence is required by the ANAO. Consequently, the level of assurance provided by this review, in relation to the 26 major Defence equipment acquisition projects (Major Projects), is less than that provided by the ANAO’s program of audits.

1.5 Separately, the ANAO undertakes analysis of key elements of the PDSSs and examines systemic issues and provides longitudinal analysis for the 26 projects reviewed.

1.6 The review was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $2.2 million.30

Review methodology

1.7 The ANAO’s review of the information presented in the individual PDSSs included:

  • examination and assessment of the governance and oversight in place to ensure appropriate project management;
  • an assessment of the systems and controls that support project financial management, risk management and project status reporting, within Defence;
  • an examination of each PDSS and the documents and information relevant to them;
  • a review of relevant processes and procedures used by Defence in the preparation of the PDSSs;
  • discussions with persons responsible for the preparation of the PDSSs and management of the projects;
  • analysis of project information, for example, cost and schedule variances;
  • taking account of industry contractor comments provided on draft PDSS information;
  • assessing the assurance by Defence managers attesting to the accuracy and completeness of the PDSSs;
  • examination of the representations by the Chief Finance Officer supporting the project financial assurance and contingency statements;
  • examination of confirmations, provided by the Capability Managers, relating to each project’s progress toward Initial Materiel Release (IMR), Final Materiel Release (FMR), Initial Operational Capability (IOC) and Final Operational Capability (FOC); and
  • examination of the Statement by the Secretary of Defence, including significant events occurring post 30 June, and management representations by the Secretary of Defence.

1.8 The ANAO’s review of PDSSs also focused on project management and reporting arrangements contributing to the overall governance of the Major Projects. The ANAO considered:

  • developments in acquisition governance (see paragraphs 1.10 to 1.32, below);
  • the financial framework, particularly as it applies to the project financial assurance and contingency statements, and Defence’s advice that project financial reporting during 2018–19 would be prepared on the same basis as project approvals and expenditure represented in the Portfolio Budget Statements and the Defence Annual Report (i.e. on a cash basis) (Section 2 of the PDSSs);
  • schedule management and test and evaluation processes (Section 3 of the PDSSs);
  • capability assessments, including Defence statements of the likelihood of delivering key capabilities, particularly where caveats are placed on the Capability Manager’s declaration of significant milestones (Section 4 of the PDSSs);
  • the ongoing reform process for Defence’s Enterprise Risk Management Framework, and the completeness and accuracy of major risk and issue data (Section 5 of the PDSSs);
  • the project maturity framework along with its related reporting and the systems in place to support the consistent and accurate application and the provision of this data (Section 6 of the PDSSs); and

40. the impact of acquisition issues on sustainment to ensure the PDSS is a complete and accurate representation of the acquisition project.

1.9 This review informed the ANAO’s understanding of the systems and processes supporting the PDSSs for the 2018–19 review period. It also highlighted issues in those systems and processes that warrant attention.

Acquisition governance

1.10 Consistent with previous years, the ANAO considered Defence’s acquisition governance processes when planning and conducting review for the 2018–19 MPR. While some of these processes are now established, others continue to mature or require further development to achieve their intended impact.

Defence Independent Assurance Reviews

1.11 The Defence Independent Assurance Review process provides the Defence Senior Executive with assurance that projects and products will deliver approved objectives and are prepared to progress to the next stage of activity. Reviews allow early identification of problem projects and products, facilitating their timely recovery.31, 32

1.12 Formerly called Gate Reviews, Independent Assurance Reviews are intended to be conducted at key acquisition and sustainment ‘gates’ in the Capability Life Cycle.33

1.13 Twenty of the 26 projects included in this report had an Independent Assurance Review conducted during 2018–1934, which formed key corroborative evidence for the ANAO’s review.

Projects of Concern

1.14 The Projects of Concern process is intended to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. The process has continued to play a role across the portfolio of MPR projects.35 As at 30 June 2019, one MPR project, MRH90 Helicopters, was a continuing Project of Concern. The project was placed on the list in November 2011 due to contractor performance relating to significant technical issues preventing the achievement of milestones on schedule.36

1.15 Auditor-General Report No.31 2018–19, Defence’s Management of its Projects of Concern, assessed whether the Department of Defence’s Projects of Concern regime was effective in managing the recovery of underperforming projects. It concluded that, while the regime is an appropriate mechanism for escalating troubled projects to the attention of senior managers and ministers, Defence was not able to demonstrate the effectiveness of its regime in managing recovery of underperforming projects. Moreover, the audit observed that the transparency and rigour of the framework’s application has declined in recent years. The audit recommended that:

  • Recommendation no.1: Defence introduce, as part of its formal policy and procedures, a consistent approach to managing entry to, and exit from, its Projects of Interest and Projects of Concern lists. This should reflect Defence’s risk appetite and be made consistent with the new Capability Acquisition and Sustainment Group Risk Model and other, Defence-wide, frameworks for managing risk. To aid transparency, the policy and the list should be made public.
  • Recommendation no.2: Defence evaluates its Projects of Concern regime.37

Defence has agreed to implement these recommendations. It has advised that the Project Risk Management System (PRMS) being developed as part of the risk reform should provide a consistent approach for entry into and exit from the Projects of Interest and Projects of Concern Lists. Defence advised that it intends for this recommendation to be implemented by March 2020 but notes that this will depend on progress in implementing the CASG Risk reform (see paragraphs 1.56 to 1.62)

1.16 In relation to Recommendation 2, Defence has advised that CASG’s Project Management Branch will analyse Projects of Concern to understand the root cause of project performance. Defence has also advised that lessons will be published, communicated and used to treat systemic issues in project practice and performance, including the training and skilling of project management. The CASG Project Management Branch will assess the contribution of the Projects of Concern regime to managing the recovery of underperforming projects.

1.17 The ANAO will continue to monitor Defence’s implementation of the recommendations and report on progress in the next MPR.

Quarterly Performance Report

1.18 The Defence Quarterly Performance Report (QPR) aims to provide senior stakeholders within government and Defence with insight into the delivery of major capability to the Australian Defence Force.38 The report is provided to the Minister for Defence and the Minister for Defence Industry on a quarterly basis.39

1.19 In July 2019, the ANAO completed an audit on the effectiveness of the QPR in providing senior stakeholders with accurate and timely information on the status and emerging risks and issues. It found the June 2018 QPR, reviewed by the ANAO, to be largely effective, contained mostly accurate information, and was valued by senior stakeholders.40 The ANAO recommended that Defence improve the QPR as a tool for senior leaders by reporting on:

  1. trend performance data for sustainment products; and
  2. emerging candidates for the Projects/Products of Concern list and Products/Projects of Interest list that have been recommended by an Independent Assurance Review or which are under active consideration by senior management.41

1.20 In the course of its review for the 2018–19 MPR, the ANAO observed that Defence’s June 2019 QPR reported on both improved and deteriorated performance for both acquisition and sustainment products since the previous QPR.42 This reflects a change in trend reporting consistent with the agreed ANAO recommendation.

1.21 The ANAO also observed as part of its review that Defence’s June 2019 QPR reported the emerging candidates for the Projects/Products of Concern list and Products/Projects of Interest list which had been recommended either by an IAR or which were under active consideration. This change was also consistent with the agreed ANAO recommendation.43

1.22 The ANAO examines QPRs as part of the procedures for its limited assurance review of Defence’s PDSSs.44 For the 2018–19 review, the ANAO examined the five QPRs from September 2018 to September 2019.45

1.23 The June 2019 QPR identified six MPR projects as Projects of Interest46:

  • Joint Strike Fighter, noting risks related to affordability and IOC and FOC deliverables47;
  • LHD Ships, due to ongoing operational testing, a large number of outstanding requirements, defects and deficiencies, and an immature support system48;
  • Hawkei, due to risks to capability and schedule that relate to ongoing reliability issues, design maturity, and production delays caused by the voluntary administration of the engine manufacturer (Steyr Motors)49;
  • Repl Replenishment Ships (Maritime Operational Support Capability), due to a change in delivery strategy with the prime contractor, which caused significant schedule changes, and deficiencies in the Integrated Logistics Support system50;
  • Civil Military Air Traffic Management System (CMATS), noting risks to schedule due to execution of design milestones and poor scope definition, and ongoing contract negotiations51; and
  • UHF SATCOM, due to delays in contract negotiation, software development and US Government certification. In the March 2019 QPR, the entire JP2008 program was identified as a Program of Interest, which is inclusive of UHF SATCOM.52

1.24 These ongoing issues for the Joint Strike Fighter, LHD Ships, Hawkei, Repl Replenishment Ships, CMATS, and UHF SATCOM projects align with the results of the ANAO’s review of the PDSSs. Delays to progress have impacted the delivery schedule of Hawkei, Repl Replenishment Ships, and UHF SATCOM during 2018–1953 (see Table 4, on page 15).

Project Directives and Materiel Acquisition Agreements

1.25 Project Directives (previously known as Joint Project Directives) state the terms of government approval, reflecting the approved scope and timeframes for activities, responsibilities and resources allocated, and key risks and issues.54 Project Directives are used to inform internal Defence documentation such as Materiel Acquisition Agreements (MAAs) between Capability Acquisition and Sustainment Group (CASG) and the Service Chiefs.55, 56 Project Directives are a key governance document under the Capability Life Cycle57, intended to ensure that all parties in Defence are informed of Government decisions. The ANAO has previously highlighted the importance of ensuring that Project Directives properly reflect the relevant Government decision, and that MAAs are appropriately aligned with the relevant Project Directive.58

1.26 In some cases Project Directives have been finalised after the MAAs they are intended to inform and, as a result, care is required to ensure that Project Directives properly reflect the relevant government decision, and that MAAs are appropriately aligned with the relevant Project Directive. The mechanism for providing the directive is via the Capability Lifecycle (CLC) Management Tool, which records the Government decision in relation to a project. In some cases, the Joint Force Authority may provide a specific documented directive. For all four new projects entering the 2018–19 MPR, it was not clear whether the projects had their Project Directive signed prior to the MAA. These projects either did not have a specific documented Project Directive, or were unable to access the CLC Management Tool at the time of the ANAO site visits. Project Directives are a requirement of the Interim CLC Manual59, and their production increases the likelihood of complying with government decisions.

1.27 The ANAO requires access to original approval documents to validate the requirements of projects. At this time, validation based on internal Defence documentation is not always possible.

1.28 The ANAO will continue to take Project Directives into account as part of its review. The extent to which they can be relied upon will be dependent on the completeness and accuracy of Project Directives, in relation to recording the detail of government approvals.

1.29 Product Delivery Agreements (PDAs)60 were being developed to replace the existing MAAs and Materiel Sustainment Agreements (MSAs). Defence has advised the ANAO that this initiative has not progressed.

Business systems

1.30 Defence continues to review its business systems with the aim of consolidating processes and systems in order to provide a more manageable system environment. As reported to the JCPAA on 31 March 2017, Defence stated that there was a ‘need to get a single unified system of accountability and reporting inside the organisation’.61 During 2018–19, and at the time of this report, the Monthly Reporting System (MRS), which provides much of the data for the PDSSs, remains in place. Defence has advised that an update to the MRS, which may be a new system, will commence pilot testing in April 2020.

1.31 In October 2018, Defence advised that it had concluded its trial of the Project Performance Review (PPR) in November 2017. At that time, PPR was a spreadsheet; now, the PPR is a bespoke Information Communications Technology (ICT) platform which draws project performance data from Defence systems. It is intended for use by project managers to inform discussions with Project Directors and Branch Heads. PPR has not been developed to replace the MRS.

1.32 In January 2018, Defence initiated a plan to implement the PPR across CASG. Defence has now commenced Phase 2 of PPR Release, and its ICT platform is currently being used by 60 projects. Defence has advised that six MPR projects are using PPR.62 The information in these documents is largely consistent with the projects’ PDSSs. The ANAO observed inconsistencies relating to:

  • project approval dates;
  • the number of High- or Extreme-rated project risks; and
  • project maturity scores.
Results of the review

1.33 The following sections outline the results of the ANAO’s review, which inform the overall conclusion in the Independent Assurance Report by the Auditor-General for 2018–19.

Financial framework

1.34 The project financial assurance statements were introduced in the 2011–12 Major Projects Report and have been included within the scope of the Independent Assurance Report by the Auditor-General since 2014–15. The contingency statements were introduced for the first time in the 2013–14 report and these describe the use of contingency funding to mitigate project risks. Together, they are aimed at providing greater transparency over projects’ financial status.

1.35 A project’s total approved budget comprises:

  • the allocated budget, which covers the project’s approved activities, as indicated in the MAA; and
  • the contingency budget, which is set aside for the eventuality of risks occurring and includes unforeseen work that arises within the delivery of the planned scope of work.63

1.36 In 2018–19, the ANAO reviewed the financial framework as it applied to managing project budgets and expenditure, including: project financial assurance, contingency, the reporting environment, and reporting cost variations and personnel costs.

Project financial assurance statement

1.37 The project financial assurance statement’s objective is to enhance transparency by providing readers with information on each project’s financial position (in relation to delivering project capability) and whether there is ‘sufficient remaining budget for the project to be completed’.64

1.38 In September and November 2018, due to cost pressures, the Joint Strike Fighter project received government approval to transfer project scope of $1.5 billion to other phases of the Joint Strike Fighter program (none of which have been approved by government). There was no corresponding transfer of funds out of the project budget.65 The PDSS states that ‘there is sufficient budget, including contingency, remaining for the project to deliver the revised scope’.

1.39 The Joint Strike Fighter PDSS reports a risk that project capability may be affected by overall funding or programming issues arising from internal cost growth, forecasting accuracy and external budget constraints. The remedial actions to address this risk reported in the PDSS include:

  • ‘Conduct ongoing engagement with the F-35 Joint Program Office and major project suppliers to facilitate improved cost data to allow the F-35 project to meet budgeting and programming expectations’ — i.e. clarifying potential cost pressures on the project;
  • ‘Proactive management of cost risk identification and engagement with the Capability Manager to prioritize requirements to deliver project capability within the approved project budget’ — i.e. actively identify cost risk and engage with senior leaders; and
  • ‘Options may be developed for Capability Manager consideration to achieve project affordability by aligning project expenditure with the Defence Integrated Investment Program capacity in any specific year’ — i.e. consider options for scheduling project expenditure to align with Defence’s available funding.

1.40 The Chief Finance Officer’s representation letter to the Secretary of Defence on the 2018–‍19 MPR’s project financial assurance statements was unqualified. The project financial assurance statement is restricted to the current financial contractual obligations of Defence for these projects, including the result of settlement actions and the receipt of any liquidated damages, and current known risks and estimated future expenditure as at 30 June 2019.

Contingency statements and contingency management

1.41 The purpose of the project contingency budget is to estimate the inherent cost, schedule and technical uncertainties of projects’ in-scope work.66 Defence policy requires project managers to ensure that all decisions in regards to a project’s contingency budget are included in the project’s contingency budget log to ensure ongoing transparency and traceability.67

1.42 PDSSs are required to include a statement regarding the application of contingency funds during the year, if applicable, as well as disclosing the risks mitigated by the application of those contingency funds. Defence’s Project Risk Management Manual (PRMM version 2.4, page 110)68 requires that contingency be applied for identified risk mitigation activities which have been assessed as being cost effective and representing value for money.

1.43 Contingency provisions for projects are not programmed or funded in cash terms.69 As such, projects are encouraged to meet contingency funding requirements from within their currently programmed cash funding. If this cannot be achieved, a project may then propose to access contingency funding from the relevant capital program (Approved Major Capital Investment Program (AMCIP), Facilities and Infrastructure Program (FIP) and ICT Capital Program). If this cannot be achieved, the contingency call will be presented to the Defence Investment Committee, which if agreed will potentially be met by budget offsets across the whole Integrated Investment Program.70

1.44 Three projects in the MPR had contingency funds applied in 2018–19: MRH90 Helicopters (supportability and performance risks), Repl Replenishment Ships (implementation of capability requirements such as spares and identification of friend or foe, and delivery from Spain to Australia), and Battle Comm. Sys. (Land) 2B (Interface Control Integration).

1.45 In 2017–18, the remaining scope of two projects was transferred into Collins R&S (SEA 1114 Phase 3 and SEA 1439 Phase 5B1). In 2018–19, the associated budgets were transferred to Collins R&S. A contingency allocation was transferred along with the budget of SEA 1439 Phase 5B1. Previous to this, Collins R&S did not have a contingency allocation. As a result, all 26 projects in the 2018–19 MPR now have a contingency allocation.

1.46 The ANAO’s examination of projects’ contingency logs as at 30 June 2019 highlighted that:

  • the clarity of the relationship between contingency allocation and identified risks continues to be an issue. Five projects (LHD Ships, Repl Replenishment Ships, CMATS, ANZAC ASMD 2B, and Pacific Patrol Boat Repl) did not explicitly align their contingency log with their risk log, by including risk identification numbers as required by PRMM version 2.4; and
  • there were two project offices (CMATS and ANZAC Air Search Radar Repl) that did not meet all the requirements of PRMM version 2.4 in terms of keeping a record of review of contingency logs. However, the ANAO observed that the information required could be located in other documents.

1.47 Non-compliance with PRMM version 2.4 has resulted in inconsistent approaches between projects in the management of contingency logs, with some projects advising that they will not remedy these non-compliances until the outcomes of the risk management reform within CASG are known (see para 1.59).

Reporting environment

1.48 On 4 April 2018, Defence advised project offices that project financial reporting for 2017–‍18 PDSSs would be prepared on the same basis as project approvals and expenditure represented in the Portfolio Budget Statements and the Defence Annual Report (i.e. on a cash basis).71

1.49 Defence obtains cash expenditure data using a management reporting tool called BORIS. Prior to the 2017–18 MPR, accrual expenditure data was extracted from the Financial Management Information System known as ROMAN. Given the change in the extraction method, the ANAO requested that Defence perform reconciliations of the cash expenditure figures from BORIS to ROMAN for each project.

1.50 In the 2018–19 MPR, Defence continued to report the projects’ financial information on a cash basis and therefore continued to perform these reconciliations. This activity concluded in October 2019 and enabled the ANAO to obtain assurance over the cash expenditure. The Defence Chief Finance Officer has determined that, from the 2020–21 MPR onwards, Defence would report expenditure data on an accrual basis. The 2019–20 MPR should, therefore, be the last year that these reconciliations need to be performed.

Reporting cost variations since Second Pass Approval and personnel costs

1.51 In May 2018, the JCPAA wrote to the Auditor-General to request that the ANAO report back to it ‘on how Defence major project cost variations and the costs of retaining project staff over time might be reported annually in future Major Projects Reports.’72

1.52 A new table was included in the 2017–18 MPR showing all budget variations post initial Second Pass Approval for projects. Refer to Table 8 on page 42.

Project Personnel Numbers and Costs

1.53 In terms of calculating the cost of retaining project staff, Defence advised the ANAO in November 2018 that its current IT systems do not provide a direct mapping of personnel to projects. It noted that personnel often work on multiple projects and sustainment activities at any given time.

1.54 The ANAO observed during fieldwork in 2019 that several MPR projects had staff who worked concurrently on other projects, which included shared corporate staff. Some of these projects did not have systems in place to record accurately the proportion of time these shared staff attributed to the project. Moreover, the ANAO observed that MPR projects used different methods to record personnel data.

1.55 Defence has advised the ANAO that it is not yet in a position to provide the staff cost component of projects and its systems are not capable of calculating the cost of retaining project staff over time. Accordingly, Defence has not provided any data on the costs of project staff for projects in the MPR. The ANAO will continue to monitor Defence’s progress in recording project personnel numbers and costs in future reports.

Enterprise Risk Management Framework

1.56 While major risks and issues data in the PDSSs remains excluded from the formal scope of the Auditor-General’s Independent Assurance Report73, material inconsistencies identified in relation to this information are required to be detailed in the report. The following information is included to provide an overall perspective of how risks and issues are managed within Defence and the selected Major Projects.

1.57 Risk management has been a focus of the MPR since its inception. The CASG risk management environment consists of multiple policies and varying implementation mechanisms and documentation. There are multiple group-level (i.e. CASG), sub-group (i.e. Divisional) and project-level risk management documents. The primary focus of the ANAO’s examination of risk management is at the project level, in order to provide assurance over the PDSSs.

1.58 At the Group level, Deputy Secretary CASG issued a directive in May 2017 establishing a CASG Risk Management Reform Program to implement a risk management model that is situated within Defence’s risk management framework. To date, Defence has delivered the first two phases of the reform (establishment of the CASG Risk Management Group Model, and the associated support activities such as training and consultation). The third phase of the reform — which includes the development of risk management policies and toolsets for use by projects — was initially planned to be concluded in June 2019. Defence has advised that it has experienced delays in completing certain deliverables, and that the contract for the completion of the Risk Reform Program has been extended to February 2020. Defence expects that, once the Reform has concluded, implementation will take a number of annual cycles to reach maturity.74

1.59 The ANAO has observed that some projects chose not to review risk and issues management procedures until this stage has been completed, as noted at paragraph 1.47. The ANAO will continue to monitor the implementation of the reform as part of future reviews, but will not be able to consider including risks and issues in the scope of the MPR until the reform is sufficiently progressed.

1.60 In 2018–19, the ANAO again examined project offices’ risk and issue logs at the Group and Service level, which are predominantly created and maintained utilising spreadsheets and/or Predict! software.75 Overall, the issues with risk management that the ANAO observed related to:

  • variable compliance with corporate guidance, for example all projects had a Risk Management Plan, however; 10 out of 26 Major Projects did not validate the currency of the Risk Management Plan in line with PRMM version 2.476;
  • the visibility of risks and issues when a project is transitioning to sustainment;
  • for one project (Joint Strike Fighter), sustainment and acquisition risks are managed together, despite Defence risk management policy for acquisition and sustainment providing inconsistent guidance77;
  • the frequency with which risk and issue logs are reviewed to ensure risks and issues are appropriately managed in a timely manner, and accurately reported to senior management;
  • risk management logs and supporting documentation of variable quality, particularly where spreadsheets are being used78; and
  • lack of quality control resulting in inconsistent approaches in the recording of issues within Predict!

1.61 The ANAO has previously observed that Defence’s use of spreadsheets as a primary form of record for risk management is a high risk approach. Spreadsheets lack formalised change/version control and reporting, thereby increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by whom, and what change was made. As a result, a significant amount of quality assurance is necessary to obtain confidence that spreadsheets are complete and accurate at 30 June, which is not an efficient approach. The ANAO’s review of CASG’s 26 project offices indicates that 15 utilise spreadsheets79 as their primary risk management tool, five utilise Predict!80, one (Maritime Comms) utilises both Microsoft Excel and Predict!, two (Joint Strike Fighter and CMATS) utilise a bespoke SharePoint based tool, one (MH-60R Seahawk) utilises Microsoft Word, one (Night Fighting Equip Repl) utilises the Project Performance Review (see paragraph 1.31), and one (HATS) does not currently manage any risks given the delivery of all primary project elements. Defence has advised that a risk management system will not be mandated until the outcomes of the CASG risk reform are known (see paragraph 1.58).

1.62 The JCPAA recommended in September 2018 that Defence plan and report a methodology to the Committee showing how acquisition projects can transition from the use of spreadsheet risk registers to tools with better version control.81 In response, Defence advised that the Risk Reform Program was developing a revised methodology for managing project risk and intended to commence prioritised transition of projects into the remodelled risk management approach from the first quarter of 2019. However, the ANAO observed in the course of its 2019 site visits that MPR projects had not received project specific guidance. Defence advised the ANAO in October 2019 that it is looking to mandate a standardised ICT tool for the management of risk across projects. The decision on this tool is expected in November 2019. The 2019–20 MPR will report on MPR projects’ use of the mandated tool.

Project maturity framework

1.63 Project Maturity Scores have been a feature of the Major Projects Report since its inception in 2007–08. The DMO Project Management Manual 2012 defined a maturity score as:

The quantification, in a simple and communicable manner, of the relative maturity of capital investment projects as they progress through the capability development and acquisition life cycle.82

1.64 Maturity scores are a composite indicator, cumulatively constructed through the assessment and summation of seven different attributes. The attributes are: Schedule, Cost, Requirement, Technical Understanding, Technical Difficulty, Commercial, and Operations and Support, which are assessed on a scale of one to 10.83 Comparing the maturity score against its expected life cycle gate benchmark provides internal and external stakeholders with a useful indication of a project’s progress.

1.65 The ANAO has previously identified that the policy guidance underpinning the attribution of maturity scores would benefit from a review for internal consistency and the relationship to Defence’s contemporary business. For example, allocating approximately 50 per cent of the maturity score at Second Pass Approval, regardless of acquisition type, is often inconsistent with the proportion of project budget expended, and the remaining work required to deliver the project. Further, the existing project maturity score model does not always reflect a project’s progress during the often protracted build phase, particularly for developmental projects. During this phase it can be expected that maximum expenditure will occur, and that many risks will be realised, some of which will only emerge as test and evaluation activities are pursued through to acceptance into operational service.

1.66 In 201684 and again in 201785, the JCPAA recommended that Defence update the policy on Project Maturity Scores. At the JCPAA hearing held on 23 March 2018, Defence undertook to update the framework by mid-2018 with a two-stage process: first to remediate inconsistencies in the policy and accommodate Interim Capability Life Cycle terminology; then to undertake a more substantial amendment of the policy.86 In September 2018, the JCPAA sought a written update from Defence outlining progress towards updating the policy.87 Also in September 2018, Defence advised the ANAO that the maturity score process was being re-considered within the CASG risk reform context. In December 2018, Defence reported this to the JCPAA in its response to the 2018 recommendation.

1.67 In October 2019, Defence advised the ANAO that a draft Project Maturity Score policy has been developed (renamed the ‘Project Progress Score’) and will be trialled and evaluated in late 2019, for implementation in the 2020–21 MPR. Defence further advised that while the progress score has been decoupled from the risk reform work, it will be informed by existing project risk polices and enhanced by improvements arising out of the risk reform framework. The ANAO was provided the draft Project Progress Score policy in November 2019. The draft indicates that the Project Progress Score will be calculated based on 10 project attributes, leading to a total score out of 100, instead of the seven attributes and total score out of 70 under the current Project Maturity Score policy. The draft policy also notes that ‘The [Project Progress Score] is not designed as an assessment of project risk’.

Caveats and deficiencies

1.68 Defence has not defined the terms ‘caveat’ or ‘deficiency’ to the declaration of significant milestones in its internal policies and procedures. The ANAO has observed use of these terms by Defence to represent exceptions to the achievement of significant milestones declared by Defence such as IMR, IOC, FMR and FOC.

1.69 The 2017–18 MPR noted a ‘reduced trend of Major Projects which had achieved significant milestones with caveats’ and Defence’s advice that it discourages Independent Assurance Reviews recommending caveats at FOC.88 Only one project (Growler) achieved a major milestone with caveats in 2017–18.89

1.70 In 2018–19, Defence declared more milestones with caveats than in 2017–18, as follows:

  • P-8A Poseidon — Defence declared two caveats to the achievement of the Operational Capability 2 (OC2) milestone in February 2019, related to deficiencies of spares (Fly Away Kits) and Operational Flight Trainer (pilot simulator) qualification; and
  • Growler — Defence declared one caveat to the achievement of the IOC milestone in February 2019, related to in-country aircrew training not yet possible due to delays in delivery of the Mobile Threat Training Emitter System.

1.71 The Chief of Air Force acknowledged achievement of FMR for the Additional MRTT project in October 2019, with an ‘accepted deficiency’ relating to the non-delivery of a minor piece of support equipment.

1.72 In addition, the Chief of Navy declared FOC for the LHD Ships project in November 2019, with seven ‘notable deficiencies’. Key deficiencies referenced in the PDSS relate to technical issues and defects, primarily affecting the propulsion pods, and Integrated Logistic Support. Remediation is underway through the Transition and Remediation Program, and the prime contract has been extended to allow for closure of the outstanding contractual requirements.90

1.73 The ANAO will continue to monitor Defence’s declaration and resolution of caveats (or exceptions) to the achievement of significant Capability Milestones in future reviews. This will include projects which have been removed from the MPR with outstanding caveats which are required to be reported by Defence in the Statement by the Secretary of Defence until their final status is accepted by the Capability Manager.91

2. Analysis of Projects’ Performance

2.1 Performance information is important in the management and delivery of major Defence equipment acquisition projects (Major Projects). It informs decisions about the allocation of resources, supports advice to government, and enables stakeholders to assess project progress.

2.2 Project performance has been the subject of many of the reviews of the Department of Defence (Defence), and a consistent area of focus of the Joint Committee of Public Accounts and Audit (JCPAA) since the first Major Projects Report (MPR). This chapter progresses previous Australian National Audit Office (ANAO) analysis over project performance.

Project performance analysis by the ANAO

2.3 The major dimensions of projects’ performance are:

  • Cost performance (pp. 37–49) — this includes the percentage of budget expended (Budget Expended), changes in budget since Second Pass Approval, in-year changes to budget, and in-year expenditure;
  • Schedule performance (pp. 50–61) — this includes the percentage of time elapsed (Time Elapsed), total schedule slippage, and in-year changes to schedule; and
  • Capability performance (pp. 62–66) — this includes the percentage of key materiel capabilities delivered (Capability Delivery Progress).

2.4 The ANAO has previously utilised Defence’s prediction of expected final capability, as reported in Section 4.1 of each Project Data Summary Sheet (PDSS). In 2015–16, the ANAO derived an indicator for ‘Capability Delivery Progress’, which aims to show the current capability delivered, in terms of capability elements included within the agreed Materiel Acquisition Agreements (MAAs). These performance indicators are measured in percentage terms, to enable comparisons between projects of differing scope, and to provide a view across the selected projects of progress and performance.

2.5 The following sections of this chapter provide analysis relating to the three principal dimensions of project performance noted above. This includes in-year information, longitudinal analysis and the results of project progress for the year-ended 30 June 2019. Figure 2 directly compares cost performance with schedule performance through two metrics, Budget Expended and Time Elapsed.92

Figure 2: Budget Expended and Time Elapsed

Figure 2

Source: ANAO analysis of the 2018–19 PDSSs.

2.6 Figure 2 shows that for nearly all projects (25 of 26), Budget Expended is broadly in line with (within 10 per cent), or lagging, Time Elapsed. This relationship is generally expected in an acquisition environment predominantly based on milestone payments. However, due to the varying complexity, stages and acquisition approaches across the portfolio of projects, further analysis of these simple performance measures is required to provide a better understanding of key variances.

2.7 Where Budget Expended is significantly lagging Time Elapsed, the project schedule may be at risk — i.e. expenditure lags may indicate delays in milestone achievement. In 2018–19, the Budget Expended for three projects lagged Time Elapsed by at least 20 per cent. For two of these three projects, milestones have been delayed, as detailed below:

  • Hawkei (Budget Expended 28 per cent, Time Elapsed 49 per cent) — the project’s achievement of milestones has been delayed by reliability issues, design maturity, and production delays. The project is expected to complete its Production Reliability Acceptance Test 21 months later than originally contracted; and
  • Battlefield Airlifter (Budget Expended 58 per cent, Time Elapsed 93 per cent) — the project has not signed contracts for the acquisition of training devices, and some other equipment is also outstanding. Delivery of this equipment has been significantly delayed; as at 30 June 2019, the project planned to deliver these items after the achievement of the FOC milestone.

2.8 For the third project, Joint Strike Fighter (Budget Expended 28 per cent, Time Elapsed 69 per cent), the expenditure lag reflects the recent transition from the aircraft development stage, where relatively little budget was expended. The project is now entering into main production contracts for aircraft, with in-year expenditure increasing compared to prior years.

2.9 Where Budget Expended leads Time Elapsed, the project budget may be at risk — i.e. expenditure increases may indicate real cost increases. However, for the one project where Budget Expended leads Time Elapsed by 10 per cent or more, the cause of the variance relates to work being performed prior to Second Pass Approval. For ANZAC Air Search Radar Repl (Budget Expended 46 per cent, Time Elapsed 29 per cent), this project spent approximately nine per cent of its budget prior to Second Pass, to conduct a Risk Reduction Program and make early purchases of equipment to ensure the schedule would be met.93

2.10 In each case of significant variance between Budget Expended and Time Elapsed, the performance information highlights projects that may require further attention. This is to ensure that unspent funds are returned to the Defence budget for re-allocation in a timely manner, the timing of key deliverables remains in focus, or planning focuses on bringing together all elements in a timely manner, as equipment is delivered.

Cost performance analysis

Budget Expended and Project Maturity

2.11 Figure 3, below, sets out each project’s Budget Expended against Project Maturity94 and shows that Budget Expended lags Project Maturity for the majority of projects (22 of 26). This relationship is typical of acquisition projects for two reasons:

  • in an acquisition environment predominantly based on milestone payments, projects will typically develop confidence in delivering their scope through design reviews, testing and demonstration, ahead of formal acceptance of milestone achievement or equipment deliveries (and expenditure of budget); and
  • more generally, Budget Expended will often lag Project Maturity as the result of Defence’s project maturity framework attributing approximately 50 per cent (35 out of 70 points) of total Project Maturity at Second Pass Approval (the main investment decision by government)95 prior to any significant expenditure of budget.

2.12 In both cases, the Budget Expended is expected to catch up to Project Maturity over the course of the project’s life, with projects approaching closure expected to show Budget Expended and Project Maturity broadly in line with each other.

2.13 Budget Expended lags Project Maturity with a variance of 20 per cent or more in 12 projects. As expected, the majority of these projects are at a relatively early stage and have expended minimal budget while progressing through design and testing phases, or are waiting on significant amounts of equipment to be delivered. The exceptions to this are projects that have delivered the majority of their major equipment, leading to an advanced maturity score, while the budget expended is lagging as items such as training equipment or weapons are yet to be delivered and paid for. Projects fitting this pattern are Battlefield Airlifter (all aircraft have been delivered while some training devices and other equipment are outstanding), and Night Fighting Equip Repl (most Tranche 1 equipment has been delivered while the evaluation of future Tranche 2 equipment is still in its early stages).

2.14 Where Budget Expended leads Project Maturity by a significant amount, this may indicate that the project is behind in development or achievement of its scope, or that the required scope is not affordable. There are no instances where Budget Expended leads Project Maturity by 20 per cent or more. The largest variance is for UHF SATCOM, where Budget Expended leads Project Maturity by 11 per cent. The project’s maturity score has been affected by delays in software development, while the majority of budget has been expended and the project has funded further development with contingency.

Figure 3: Budget Expended and Project Maturity

Figure 3

Source: ANAO analysis of the 2018–19 PDSSs.

Second Pass Approval and 30 June 2019 approved budget

2.15 Figure 4, below, compares each project’s approved budget at initial Second Pass Approval and its approved budget at 30 June 2019.

2.16 The total budget for the 26 projects at 30 June 2019 was $64.1 billion, a net increase of $24.4 billion, when compared to the approved budget at initial Second Pass Approval of $39.7 billion.

2.17 Figure 4 indicates all budget variations from initial Second Pass Approval. Six projects have variations of $500 million or more. The list below describes the components of these variations:

  • Joint Strike Fighter — increase of $13.8 billion, comprising $10.5 billion for 58 additional aircraft in 2013–14, $2.9 billion for exchange rate variation and $0.4 billion for price indexation96;
  • AWD Ships — increase of $1.9 billion, comprising $1.2 billion for a Real Cost Increase97 in July 2015 to complete the project, $1.2 billion for price indexation, offset by a $0.4 billion decrease for exchange rate variation and a $0.1 billion decrease for transfers to facilities projects in 2013–14;
  • P-8A Poseidon — increase of $1.8 billion, comprising $1.3 billion for four additional aircraft in 2015–16 and $0.5 billion for exchange rate variation;
  • MRH90 Helicopters — increase of $2.8 billion, comprising $2.6 billion for 34 additional aircraft in 2005–06 and other minor scope changes, and $0.7 billion for price indexation, offset by a $0.3 billion decrease due to scope transfers for facilities, and a $0.1 billion decrease for exchange rate variation;
  • Growler — increase of $0.8 billion, comprising $0.9 billion for exchange rate variation, $0.3 billion for the Mobile Threat Training Emitter System and weapons, offset in 2015–‍16 by a $0.2 billion decrease for transfers to facilities projects and $0.2 billion for the return to the Defence budget of surplus funds and contingency for reallocation; and
  • Overlander Medium/Heavy — increase of $0.8 billion, comprising $0.7 billion ‘project supplementation’ to reduce cost pressures and $0.1 billion exchange rate variation.

Figure 4: Projects’ initial Second Pass Approval and 30 June 2019 approved budget ($m)

Figure 4

Note 1: Figure marker indicates that the budget for the project at 30 June 2019 is less than the original budgeted cost.

Note 2: On 22 May 2015, the Minister for Defence and the Minister for Finance announced there would be further delays to the delivery of the Air Warfare Destroyers and an additional $1.2 billion would be required to complete the project. The budget increase was incorporated into the approved project budget as at 30 June 2016.

Source: ANAO analysis of the 2018–19 PDSSs.

2.18 Budget variances since initial Second Pass Approval may result from: increasing the scope of a project via revised Second Pass Approvals, programmatic decisions, Real Cost Increases/Decreases, transfers to/from other projects and budgetary adjustments. A summary of budget variations is at Table 3 (page 13) and a more detailed analysis of this variance is included in Table 8, below.

Table 8: Budget variation post initial Second Pass Approval by variation type as at 30 June 2019 and Performance Audits 1

Project

Initial Second Pass Approval Budget $m

Variation

Explanation of Variation

Year/s of Variation

Total Amount of Variation $m

Performance Audits

AIR 6000 Phase 2A/2B New Air Combat Capability 2

2751.6

(Stage 1)

Scope increase/Budgetary Adjustments/Transfer

58 additional aircraft (Stage 2 Second Pass Approval) offset by minor transfers

2013–14

2017–18

10,504.1

Auditor-General Report No.14 of 2018–19: Joint Strike Fighter — introduction into service and sustainment planning

Auditor-General Report No.6 of 2012–13: Management of Australia’s Air Combat Capability – F-35A Joint Strike Fighter Acquisition

SEA 4000 Phase 3 Air Warfare Destroyer Build

7207.4

Real Cost Increase/Budget transfers

Real Cost Increase of $1.2b offset by minor transfers for facilities in 2014

2013–14

and

2015–16

1089.6

Auditor-General Report No.22 of 2013–14: Air Warfare Destroyer Program

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

AIR 7000 Phase 2B Maritime Patrol and Response Aircraft System

3577.7

Scope increase

Four additional aircraft

$1m transferred from DST Group from 2017-18 surplus funds

2015–16

2017-18

1296.4

N/A

AIR 9000 Phase 2/4/6 Multi-Role Helicopter 3

957.2

(Phase 2)

Scope increase/Budget transfers

34 additional aircraft (Phase 4/6 Second Pass Approval), offset by minor transfers

2005–06

2018-19

 

2270.5

Auditor-General Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions (paragraph 4.54)

Auditor-General Report No.52 of 2013–14: Multi-Role Helicopter Program

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

SEA 1180 Phase 1 Offshore Patrol Vessel

3632.8

N/A

N/A

N/A

0.0

Auditor-General Report No.39 of 2017–18: Naval Construction Programs – Mobilisation

AIR 5349 Phase 3 EA-18G Growler Airborne Electronic Attack Capability

2641.4

Scope increase/Real Cost Decrease

Additional training devices offset by return of surplus funds and other minor transfers

2014–15,

2015–16

and 2016–17

(91.6)

N/A

LAND 121 Phase 3B Medium Heavy Capability, Field Vehicles, Modules and Trailers 2

2549.2

Real Cost Increase 4 /Scope/Budgetary

adjustment

 

Project supplementation ($684.2m) and additional vehicles, trailers and equipment ($28.0m) at Revised Second Pass Approval

Budgetary Adjustment (-$30.0m)

2013–14

2018–19

682.2

Auditor-General Report No.52 of 2014–15: Australian Defence Force’s Medium and Heavy Vehicle Fleet Replacement (LAND 121 Phase 3B)

AIR 9000 Phase 8 Future Naval Aviation Combat System Helicopter

3029.6

Budget transfer

Transfer to Defence Support and Reform Group

2013–14

(39.2)

N/A

JP 2048 Phase 4A/4B Amphibious Ships (LHD)

2958.3

Budget transfer

Transfer from Defence Science and Technology Group

2008–09

9.3

Auditor-General Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

LAND 121 Phase 4 Protected Mobility Vehicle – Light

1945.0

N/A

N/A

N/A

0.0

Auditor-General Report No. 6 of 2018–19: Army’s Protected Mobility Vehicle – Light

AIR 8000 Phase 2 Battlefield Airlift – Caribou Replacement

1156.5

N/A

N/A

N/A

0.0

Auditor-General Report No.3 of 2013–14: AIR 8000 Phase 2 – C-27J Spartan Battlefield Airlift Aircraft

SEA 1654 Phase 3

Maritime Operational Support Capability

1004.6

Budget Transfer

Transfer for training offset by minor transfers

2015–16

2018–19

69.4

N/A

AIR 5341 Phase 3 Civil Military Air Management System

731.4

Real Cost Increase/ Budgetary Adjustment

Real Cost Increase offset by minor transfers

2017–18

240.7

Auditor-General Report No.4 of 2019–20: OneSky: Contractual Arrangements

Auditor-General Report No.46 of 2016–17: Conduct of the OneSKY Tender

Auditor-General Report No.1 of 2016–17: Procurement of the International Centre for Complex Project Management to Assist on the OneSKY Australia Program

AIR 7403 Phase 3 Additional KC-30A Multi-role Tanker Transport

681.9

Scope increase/Budgetary adjustment

Additional capability (Government Transport and Communication) offset by minor transfers

2015–16

182.9

N/A

SEA 1448 Phase 2B ANZAC Anti-Ship Missile Defence

248.8

Budget transfers/Scope increase

Scope increases offset by minor transfers

2005–06 

2011–12

363.4

Auditor-General Report No.30 of 2018-19: ANZAC Class Frigates - Sustainment

SEA 1439 Phase 5B2

Collins Class Communications and Electronic Warfare Improvement Program

247.7 (Stage 1)

Scope increase

Additional capability (Stage 2 Second Pass Approval)

2016–17

351.5

Auditor-General Report No.23 of 2008–09: Management of the Collins-class Operations Sustainment

JP 9000 Phase 7

Helicopter Aircrew Training System

483.8

Budget transfers

Transfer of budget to Estate and Infrastructure Group for Facilities Activities

2018-19

(0.1)

N/A

SEA 1439 Phase 3 Collins Class Submarine Reliability and Sustainability

72.0

Scope increase/ Budget transfers/ Budgetary adjustments

Implementation of full scope, offset by minor transfers

2000–01 

2001–02 

2002–03 

2004–05 

2005–06 

2006–07

2018-19

305.0

Auditor-General Report No.23 of 2008–09: Management of the Collins-class Operations Sustainment

SEA 1442 Phase 4 Maritime Communications Modernisation

385.7

N/A

N/A

N/A

0.0

Auditor-General Report No.30 of 2018-19: ANZAC Class Frigates - Sustainment

JP 2072 Phase 2A

Battlespace Communications System Phase 2A

436.4

Real Cost Decrease

Real Cost Decrease

2017–18

(25.6)

N/A

SEA 1448 Phase 4B

ANZAC Air Search Radar Replacement

427.8

 

N/A

N/A

N/A

0.0

Auditor-General Report No.30 of 2018-19: ANZAC Class Frigates - Sustainment

JP 2008 Phase 5A Indian Ocean Region UHF SATCOM

460.9

Real Cost Decrease

Real Cost Decrease

2013–14

(18.0)

N/A

JP 2048 Phase 3 Amphibious Watercraft Replacement

235.7

Budget transfer

Correction to transfer price

2013–14

(7.7)

Auditor-General Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions (paragraph 4.68)

Total

37,823.4

 

 

 

17,182.8

 

             

Note 1: Some projects have multiple Second Pass Approvals. This table reports on variations since the first, i.e. initial, Second Pass Approval.

Note 2: Three projects have had multiple Second Pass Approvals. For the purposes of this table, the ANAO has used the earliest Second Pass Approval.

Note 3: Projects that have had no Real Variations to their budget, and have not appeared in any performance audits, do not appear in this table. They are: Battle Comm. Sys. (Land) 2B, Pacific Patrol Boat Repl, and Night Fighting Equip Repl. For a definition of ‘Real Variations’ see page 410 in the 2018–19 MPR Guidelines in Part 4 of this report.

Note 4: Described by Defence as ‘project supplementation’. Refer to Note 3 of Table 3.

Budget performance

2.19 The following figures and tables illustrate the budget performance of the 26 selected projects by way of:

  • in-year budget variations by project (see Table 9, below); and
  • expenditure forecasting performance against actual expenditure for 2018–19 (see Figure 5, on page 49).
In-year budget variance analysis

2.20 Table 9, below, sets out the in-year budget variations for each project. Overall, the approved budget for the projects as at 30 June 2019 increased by $1225.1 million, or 2.1 per cent, compared to their approved budget as at 30 June 2018. This was driven by exchange rate variation increases of $1221.6 million and net real increases of $3.7 million.

2.21 Exchange rate variations result from projects’ exposure to foreign currencies and movements in foreign exchange rates against the Australian dollar.98 Budget adjustments aim to maintain the relative buying power of the project budget. Movements in the US dollar and the Euro are the main influences. Projects with larger movements in foreign exchange in 2018–19 included:

  • Joint Strike Fighter — movement of $1018.6 million, or 6.6 per cent increase in budget; and
  • MH-60R Seahawk — movement of -$217.8 million, or 6.3 per cent decrease in budget.

2.22 Real Variations99 primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability and budgetary adjustments such as administrative savings decisions. In 2018–19, the two projects with more significant Real Variations were:

  • Collins R&S — variation of $33.7 million reflecting budget transfers associated with a transfer of scope from two related projects; and
  • Overlander Medium/Heavy — variation of -$30.0 million reflecting the return of funds to the Integrated Investment Program.

Table 9: In-year (2018–19) budget variations by project

Project

Approved Budget 2017–18 $m

Approved Budget 2018–19 $m

In-year Exchange Variation $m

In-year Real Variation $m

Total Variance $m

Total Variance (per cent)

Joint Strike Fighter

15,504.0

16,522.6

1018.6

-

1018.6

6.6

AWD Ships

9089.3

9103.7

14.4

-

14.4

0.2

P-8A Poseidon 1

5212.0

5375.7

163.8

-

163.7

3.1

MRH90 Helicopters

3771.1

3771.1

0.2

(0.2)

0.0

0.0

Offshore Patrol Vessel

-

3724.3

91.5

-

91.5

2.5

Growler

3430.4

3510.3

79.9

-

79.9

2.3

Overlander Medium/Heavy

3428.9

3399.9

1.0

(30.0)

(29.0)

(0.8)

MH-60R Seahawk

3430.3

3212.5

(217.8)

-

(217.8)

(6.3)

LHD Ships

3091.7

3092.2

0.5

-

0.5

0.0

Hawkei

1952.0

1979.6

27.6

-

27.6

1.4

Battlefield Airlifter

1433.3

1442.1

8.8

-

8.8

0.6

Repl Replenishment Ships

1066.8

1070.6

3.5

0.3

3.8

0.4

CMATS

974.2

975.8

1.6

-

1.6

0.2

Battle Comm. Sys. (Land) 2B

920.1

942.6

22.5

-

22.5

2.1

Additional MRTT

887.8

894.3

6.5

-

6.5

0.7

ANZAC ASMD 2B

678.7

678.7

-

-

0.0

0.0

Collins Comms and EW

-

607.8

8.3

-

8.3

1.4

Pacific Patrol Boat Repl

501.2

504.0

2.8

-

2.8

0.6

HATS

481.5

481.6

0.2

(0.1)

0.1

0.0

Collins R&S

411.6

445.3

-

33.7

33.7

6.7

Night Fighting Equip Repl

-

442.6

(17.7)

-

(17.7)

(4.0)

Maritime Comms 1

437.7

440.0

2.4

-

2.3

0.5

Battle Comm. Sys. (Land) 2A

438.0

438.1

0.1

-

0.1

0.0

ANZAC Air Search Radar Repl

-

428.7

1.0

-

1.0

0.2

UHF SATCOM

419.9

421.8

1.9

-

1.9

0.5

LHD Landing Craft

236.7

236.7

-

-

-

0.0

Total 2

57,797.2

64,142.6

1221.6

3.7

1225.1

2.1

             

Note 1: The Total Variance and components for this project do not add up due to rounding differences.

Note 2: The difference between the total approved budgets for 2017–18 and 2018–19 is partly due to the projects entering the MPR in 2018–19 (Offshore Patrol Vessel, Collins Comms and EW, Night Fighting Equip Repl, and ANZAC Air Search Radar Repl) not contributing to the total budget figure for 2017–18.

Source: ANAO analysis of the 2017–18 and 2018–19 PDSSs.

In-year forecast and actual expenditure

2.23 Accurately forecasting and managing budget expenditure is an important element in the management of a portfolio of projects. Figure 5, below, sets out the expenditure forecasting performance of each project against actual expenditure in 2018–19. In total, actual expenditure for the 26 projects at 30 June 2019 was $4831.4 million. This is compared against an initial Portfolio Budget Statements (PBS) forecast expenditure of $5809.4 million, a mid-year Portfolio Additional Estimates Statements (PAES) forecast of $5382.3 million, and a final forecast of $5173.0 million (Final Plan, approved during May 2019).

2.24 Figure 5 highlights that notable in-year underspends occurred in the following projects:

  • AWD Ships (expenditure of $198.9 million compared to $375.9 million PBS, $226.2 million PAES and $226.6 million Final Plan estimates) — the variance is due to deferring planned expenditure to subsequent years and Defence Finance Group deferring payment of invoices incurred during 2018–19;
  • P-8A Poseidon (expenditure of $472.4 million compared to $592.3 million PBS, $408.9 million PAES and $472.6 million Final Plan estimates) — the variance is due to having paid some planned 2018–19 expenditure in 2017–18 and deferring some planned 2018–19 expenditure to subsequent years; and
  • Hawkei (expenditure of $89.3 million compared to $395.6 million PBS, $202.8 million PAES and $117.5 million Final Plan estimates) — the variance is due to ongoing vehicle reliability issues, design maturity, and delays in the delivery of engine components delaying payment of milestones, as well as Defence Finance Group deferring payment of invoices incurred during 2018–19.

2.25 Figure 5 also highlights that a notable in-year overspend occurred in the following project:

  • Joint Strike Fighter (expenditure of $1942.0 million compared to $1821.1 million PBS, $1933.3 million PAES and $1977.6 million Final Plan estimates) — the variance is due to foreign exchange updates.

Figure 5: In-year (2018–19) projects’ forecast expenditure performance compared to actual expenditure ($m)

Figure 5

Sources: ANAO analysis of the 2018–19 PDSSs and Defence Portfolio Budget Statements.

Schedule performance analysis

2.26 Defence data continues to show that schedule performance is a key issue in delivering and sustaining equipment.100 Project schedule slippage can have the effect of introducing or exacerbating a capability gap, or requiring an extension to the planned withdrawal date for those platforms being replaced.101

Time Elapsed and Project Maturity

2.27 Based on the findings of the Defence Procurement Review 2003102, in 2005 Defence began to increase the proportion of MOTS acquisitions, which are generally lower risk projects and therefore more likely to meet schedule timelines. Analysis of the available performance information highlights that the selection of MOTS projects assists in reducing risk during project acquisition, where Project Maturity is more advanced at Second Pass Approval than developmental projects. For example, CMATS is a developmental project that has experienced significant schedule slippage; its maturity score at Second Pass Approval was 31 points, below the expected benchmark of 35 points for projects at Second Pass Approval. In contrast, MH-60R Seahawk is a MOTS project that has not experienced any slippage to date; its maturity score at Second Pass Approval was 37 points.

2.28 Figure 6, below, sets out each project’s Time Elapsed against Project Maturity.103 Time Elapsed lags Project Maturity for 19 of 26 projects. Similar to the analysis of Budget Expended and Project Maturity, at paragraphs 2.11 to 2.14, this pattern is expected as projects will generally score 50 per cent of their Project Maturity at Second Pass Approval, when Time Elapsed is zero (for the purposes of the ANAO’s analysis in this report). The lag is most pronounced in MOTS and Australianised MOTS acquisitions, including Offshore Patrol Vessel, Collins Comms and EW, Pacific Patrol Boat Repl, and Night Fighting Equip Repl. The exception is ANZAC Air Search Radar Repl, a developmental project where the lag in Time Elapsed against Project Maturity reflects the project’s extensive schedule to FOC, required to upgrade all eight ships in the Anzac class.

2.29 For the 10 projects where Time Elapsed lags Project Maturity by 20 per cent or more, this generally reflects projects at relatively early stages of acquisition processes, including proceeding through design activities, or awaiting significant amounts of their major equipment to be constructed and delivered. There are two significant exceptions to this:

  • MH-60R Seahawk, where the majority of equipment has been delivered but the project needs to test and integrate a number of ADF Mission System Options and modify Navy ships to operate with the helicopter; and
  • HATS, where all helicopters and training devices have been delivered, but more time is required to prove the full capability of the equipment in operational service.

2.30 For the seven projects where Time Elapsed leads Project Maturity, there were no instances where this difference was significant (20 per cent or more). The greatest variance was for Battlefield Airlifter, where Time Elapsed leads Project Maturity by 15 per cent. At 30 June 2019 this project was forecasting to achieve FOC in December 2019, despite major contracts for the acquisition of training devices not yet being signed and some other equipment also not yet acquired.

Figure 6: Time Elapsed and Project Maturity

Figure 6

Source: ANAO analysis of the 2018–19 PDSSs.

Schedule slippage and acquisition type by approval date

2.31 Figure 7, below, illustrates the total schedule slippage104 since Second Pass Approval for the 26 selected projects. It also depicts the acquisition type and places projects in order of government approval. Figure 8 illustrates the total schedule slippage for the 22 projects that have exited the review.

2.32 Following implementation of the recommendations of the Defence Procurement Review 2003105, in 2005 Defence began focusing on MOTS and Australianised MOTS acquisitions.106 Figure 8 shows that the inclusion of MOTS acquisitions contributed, prima facie, to a reduction in schedule slippage in the Major Projects portfolio. For projects that have exited the MPR, MOTS projects report an average of 11 months of slippage per project, while Australianised MOTS projects report an average of 45 months and developmental projects report an average of 105 months. Decisions on whether to undertake developmental projects should be considered on a risk basis.107 In this context, the consideration of risk should be holistic and weigh up the level of capability to be acquired against potential risks relating to cost and schedule.

2.33 The 2015 First Principles Review recommended the construction of a ‘smart buyer’ framework, with the aim of ‘[ensuring] Defence can make strategic decisions regarding the most appropriate procurement and contracting methodologies’.108 Defence has begun to conduct Smart Buyer assessments for acquisition projects at different stages of approval. None of the projects currently in the Major Projects portfolio have been approved under the Smart Buyer processes. The ANAO will continue to report on the outcomes of the First Principles Review and the Smart Buyer framework in subsequent years.

2.34 The First Principles Review also identified technical risk as the major cause of post Second Pass Approval schedule slippage, and observed that schedule slippage causes cost escalation.109 The challenge of gaining a full understanding of the complexities of developmental aspects of projects at Second Pass Approval is evident by the extent of slippage over time.

2.35 Figures 7 and 8 illustrate that older projects have experienced the most slippage. These projects tended to be more developmental (complex) in nature and typically experienced schedule slippage in the past, and have often continued to do so. This demonstrates an ongoing trend of slippage in historically late projects, which is more pronounced in older projects. This trend is also visible, but less prominent, in newer projects.

2.36 The more recent developmental projects, Hawkei and ANZAC Air Search Radar Repl, are yet to experience slippage to their FOC dates. However, these projects have experienced slippage to design reviews, test programs, or material release milestones; in the case of Hawkei, 21 months slippage to the Production Reliability Acceptance Test, leading to 17 months slippage to Initial Materiel Release. In contrast, recent MOTS projects, Pacific Patrol Boat Repl and Night Fighting Equip Repl, have adhered more closely to their design and materiel release schedules with only minor variances. This indicates that although developmental projects currently in the MPR are not reporting significantly more slippage to FOC than MOTS projects, developmental projects still appear to carry a higher level of technical risk.

2.37 While it is not possible to predict the full extent of slippage a project will experience, Figure 8 analysis has been provided to highlight changes since the Kinnaird Review. Nine post Kinnaird and 12 pre Kinnaird projects have exited the MPR. Total slippage of the nine post Kinnaird projects is 7.1 years. Total slippage of the 12 pre Kinnaird projects is 72.3 years. Six of the nine post Kinnaird projects were MOTS acquisitions and 11 out of the 12 pre Kinnaird acquisitions were Australianised MOTS or developmental.

Figure 7: Current Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Figure 7

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: Additional scope approved following Second Pass Approval has caused slippage for: P-8A Poseidon (24 months), Additional MRTT (21 months), Collins Comms and EW (36 months) and Collins R&S (13 months). The additional scope for these projects explains 94 months of the 691 months of total slippage reported in 2018–19.

Note 3: Only one project in the 2018–19 MPR, Collins R&S, was approved prior to the Kinnaird reforms in 2005.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Figure 8: Exited Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Figure 8

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates. The Hornet Refurb and BMS projects did not have FOC dates.

Note 2: The slippage shown for Next Gen Satellite related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced which reduced this slippage.

Note 3: Projects to the left of the dotted line were approved following implementation of the Kinnaird reforms in 2005. Projects to the right were approved prior to the reforms being implemented.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Schedule performance

2.38 The figures and tables that follow illustrate:

  • the original and 30 June 2019 forecasts for achieving FOC;
  • in-year schedule changes to achieving FOC; and
  • total schedule slippage across the Major Projects.
Original and 30 June 2019 Final Operational Capability forecasts

2.39 Figure 9, below, presents information on the selected projects’ original and 30 June 2019 forecasts for achieving FOC. The total schedule slippage110 for the 26 Major Projects to date is 691 months compared to the initial prediction when approved by government. This represents a 27 per cent increase on the approved schedule. Of the 26 projects in the 2018–19 report, 21 have experienced schedule slippage.

2.40 Total schedule slippage across the Major Projects was 691 months in 2018–19. This is 110 months lower than the figure of 801 months reported in the 2017–18 report. The difference is mainly due to the exit of projects with significant slippage — Collins RCS, Hw Torpedo, ANZAC ASMD 2A and BMS — which reduced the total accumulated slippage by 254 months. This was offset by in-year slippage for Battle Comm. Sys. (Land) 2B (significant technical issues requiring a change in technical approach), ANZAC ASMD 2B (administrative delays to declaring FOC), UHF SATCOM (further delays in software development), and LHD Landing Craft (rescheduling of heavy load carriage sea trials). These projects, combined, added 68 months of the 108 months schedule slippage in 2018–19. Additionally, Collins Comms and EW added 36 months of slippage to the total of 691 months; the slippage occurred in 2016–17 but the project was reported in the MPR for the first time in 2018–19.

2.41 The reasons for schedule slippage often require a deep understanding of project technical elements and a realistic assessment of the capacity of the private sector to deliver in the expected timeframe. A project office’s ability to gain access to the platform for upgrading can also result in schedule delay (for example, Maritime Comms and Collins R&S).111

2.42 A closer examination of the reasons for schedule slippage demonstrates the importance of initial assessments of project complexity. A key factor is whether a project is MOTS, Australianised MOTS or developmental.112 One project, MRH90 Helicopters113, was originally misclassified as MOTS. The project was reclassified by Defence to Australianised MOTS (i.e. more developmental) subsequent to Second Pass Approval.114 This project has experienced extended schedule slippage. Another project, UHF SATCOM, is still classified as MOTS but includes the development of significant amounts of software. Delays in software development have led to 42 months of slippage to the FOC milestone.

2.43 Figure 9 further indicates that one project (Joint Strike Fighter) is currently forecasting an FOC date earlier than originally approved. However, Joint Strike Fighter has previously forecast an earlier date than its 30 June 2019 forecast, and has experienced slippage from that previous forecast, partially offsetting its schedule recovery. Other projects with schedule recovery offset by slippage are AWD Ships, Growler, Overlander Medium/Heavy, Additional MRTT, ANZAC ASMD 2B, HATS, Collins R&S, and LHD Landing Craft. In total, these projects have contributed 35 months of schedule recovery to the Major Projects; however, ANAO analysis (for example, in Table 2 and Figure 11) excludes this effect to portray the complete amount of slippage experienced by the Major Projects.

Figure 9: Projects’ original and 30 June 2019 FOC forecasts

Figure 9

Note 1: Figure marker indicates that the forecast FOC date for the project at 30 June 2019 is earlier than the original FOC date.

Source: ANAO analysis of the 2018–19 PDSSs.

In-year schedule performance

2.44 In 2018–19, there was schedule slippage of 108 months in the forecast achievement of FOC across the 26 Major Projects, as shown in Figure 10, below. In-year project performance, measured by slippage over the last 12 months, may not reflect the project trend.

2.45 In-year schedule slippage occurred for the following 13 projects115 (the explanation provided, drawn from the 2018–19 PDSSs, may also include the reasons for prior slippage):

  • AWD Ships — the variance reflects rescheduling of the FOC milestone until after completion of Combat System Ship Qualification Trials;
  • P-8A Poseidon — the variance reflects minor rescheduling of the FOC milestone;
  • Growler — the variance reflects minor rescheduling of the FOC milestone;
  • Overlander Medium/Heavy — the variance reflects a more accurate forecast of the duration of activities leading up to FOC;
  • Repl Replenishment Ships — the project’s understanding of FOC requirements has matured and more time will be required to achieve them;
  • Battle Comm. Sys. (Land) 2B — delays in interfacing projects required a change in this project’s technical solution. More time is required to conduct design, testing and evaluation of the new solution;
  • Additional MRTT — minor schedule savings were predicted in 2017–18, but were not realised;
  • ANZAC ASMD 2B — administrative processes delayed the declaration of FOC;
  • HATS — minor delays to align with the completion time frame agreed to in the project’s MAA;
  • Maritime Comms — delay to FOC to allow for Navy to conduct processes following completion of FMR;
  • Battle Comm. Sys. (Land) 2A — administrative processes are still required to declare FOC;
  • UHF SATCOM — the project has experienced further delay to FOC due to contractor delays and additional security certification requirements by the US Government; and
  • LHD Landing Craft — final operational test and evaluation trials were rescheduled for Quarter 3, 2019.

Figure 10: In-year (2018–19) schedule changes to achieving FOC

Figure 10

Note: Defence’s PDSSs indicate that 13 of the 26 Major Projects Report projects did not record changes to their Final Operational Capability dates this year.

Source: ANAO analysis of the 2018–19 PDSSs.

Schedule performance by year of entry to MPR

2.46 Figure 11, below, shows the accumulated schedule slippage of the Major Projects included in the MPR reports from 2008–09 to 2018–19.116 Table 10 provides the details of the specific projects included in the analysis. The figure shows that over half of the total schedule slippage across the Major Projects covered in the 2018–19 report (57.6 years or 691 months) is made up of the slippage from the oldest projects, added in 2008–09 and 2009–10.

Figure 11: Schedule slippage by year of entry to the MPR in 2018–19 (in years)

Figure 11

Note: The total schedule slippage in 2018–19 across the 26 projects is 691 months.

Source: ANAO analysis of the PDSSs in Major Projects Reports.

Table 10: Year of entry for projects included in Figure 11 analysis

Project

Year added to MPR

Project

Year added to MPR

AWD Ships

2008–09

P-8A Poseidon

2014–15

MRH90 Helicopters

2008–09

Maritime Comms

2014–15

LHD Ships

2008–09

Additional MRTT

2015–16

ANZAC ASMD 2B

2009–10

HATS

2015–16

Collins R&S

2009–10

Hawkei

2016–17

Joint Strike Fighter

2010–11

CMATS

2016–17

UHF SATCOM

2010–11

Repl Replenishment Ships

2017–18

MH-60R Seahawk

2011–12

Battle Comm. Sys. (Land) 2B

2017–18

Battle Comm. Sys. (Land) 2A

2012–13

Pacific Patrol Boat Repl

2017–18

Growler

2013–14

Offshore Patrol Vessel

2018–19

Overlander Medium/Heavy

2013–14

Collins Comms and EW

2018–19

Battlefield Airlifter

2013–14

Night Fighting Equip Repl

2018–19

LHD Landing Craft

2013–14

ANZAC Air Search Radar Repl

2018–19

       

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

2.47 Table 11 shows that 16 per cent (112 of 691 months) of the total schedule slippage across the 2018–19 Major Projects is attributed to the sole remaining project approved prior to the Kinnaird reforms, Collins R&S.

Table 11: Project slippage

Project

No. of months between Approval and Original FOC date

No. of months between Approval and 30/6/19 FOC date

No. of months slippage between Original FOC and 30/6/19 FOC date

Joint Strike Fighter 1

169

167

2

AWD Ships 1

131

168

40

P-8A Poseidon

71

100

29

MRH90 Helicopters

119

208

89

Offshore Patrol Vessel

151

151

0

Growler 1

111

112

2

Overlander Medium/Heavy 1

125

125

11

MH-60R Seahawk

150

150

0

LHD Ships

113

150

37

Hawkei

94

94

0

Battlefield Airlifter

68

92

24

Repl Replenishment Ships

73

80

7

CMATS

102

130

28

Battle Comm. Sys. (Land) 2B

65

89

24

Additional MRTT 1

33

54

23

ANZAC ASMD 2B 1

90

165

77

Collins Comms and EW

114

150

36

Pacific Patrol Boat Repl

89

91

2

HATS

76

76

3

Collins R&S 1,2

165

273

112

Night Fighting Equip Repl

61

61

0

Maritime Comms

125

138

13

Battle Comm. Sys. (Land) 2A

55

94

39

ANZAC Air Search Radar Repl

84

84

0

UHF SATCOM

111

153

42

LHD Landing Craft 1

53

99

51

Total - All Projects With Slippage

2,598

3,254

691

       

Note 1: These figures do not add horizontally due to the exclusion of schedule reductions over the life of the project. Refer to footnote 26.

Note 2: Collins R&S was approved prior to the implementation of the Kinnaird reforms. It accounts for 16% of the total schedule slippage.

Source: ANAO analysis of the 2018–19 PDSSs.

Capability performance analysis

2.48 Defence defines capability as the power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.117 An operational effect is achieved by combining the nine Fundamental Inputs to Capability — organisation, command and management, personnel, collective training, major systems, facilities and training areas, supplies, support, and industry118 — and undertaking designated operations.

2.49 In acquiring Defence platforms and systems, a range of documentation (including capability definition, operational concept, function and performance specification, and Test and Evaluation Master Plans) is developed, which establishes the detailed requirements/performance attributes to be achieved.

2.50 Since the 2009–10 MPR, capability reporting119 has been based on Defence’s prediction of the final capability that would be achieved on the basis of deliverables and/or activities completed. This assessment of capability performance (Expected Capability) is measured against the Materiel Release Milestones (MRMs) and Completion Criteria specified in each project’s Materiel Acquisition Agreement (MAA). This is distinct from an assessment of whether milestones will be achieved on schedule. As the ANAO has previously noted, this data involves ‘making certain assumptions in forecasting achievements and is therefore subjective in approach …’.120

2.51 For example, for the LHD Landing Craft project, in the 2015–16 MPR Defence predicted and reported that 99 per cent of elements of capability had a ‘high level of confidence of delivery’, with the capability to transport heavy loads (of up to 65 tonnes) still requiring trials prior to declaration of capability achievement; this capability was assessed as ‘under threat, considered manageable’.

2.52 These trials have been delayed on multiple occasions since 30 June 2016, as reported to the JCPAA by Defence at public hearings.121 The 2018–19 PDSS continues to report a one per cent Amber rating corresponding to the outstanding trials. This indicates that the subjective 2015–16 forecast of the capability achievement may not have been accurate, with the capability proving harder to achieve than expected.

2.53 The Chief of Navy declared FOC for the LHD Landing Craft project in November 2019 on the basis of trials conducted in July 2019 demonstrating heavy load carriage of up to 62 tonnes, and a desktop analysis extrapolating the results of the trials to 65 tonnes.122

2.54 A further example is the Battlefield Airlifter project which reported a 100 per cent Green capability prediction at its inclusion in the MPR in 2013–14. However, the 2013–14 PDSS also reported major risks relating to capability deficiencies arising from the US Government divesting from the program, with Australia no longer able to rely on the US Air Force processes. These risks have continued to affect the project, with a mature training system and a number of baseline capability requirements now not expected to be delivered until after FOC. These capability issues were reported in the PDSS Pie Chart for the first time in 2018–19, indicating that the earlier confidence in the ability to achieve the required capability may have been overly optimistic.123

2.55 Over time, the JCPAA has sought the use of a more robust measure of capability performance.124

2.56 In October 2017, the JCPAA recommended ‘that the Department of Defence review the procedure for the development of expected capability estimates for future Major Projects Reports. The outcomes of this review should be provided to the Committee within six months of the tabling of this report. Further, the Committee requests that Defence provide a progress report within three months of the tabling of this report.’125

2.57 Defence made a submission to the Committee in March 2018 regarding this recommendation.

Defence will conduct a schedule baseline validation activity for the Major Projects Report projects to drive greater consistency in schedule reporting.

Once this activity is complete, Defence should be in a better position to investigate a more robust approach to measuring Capability estimates. Utilising the validated baseline data could inform:

  • A simple percentage of schedule milestones achieved to measure progress to date. This is a quantitative assessment that relies on the maintenance of a robust project baseline, which is not dissimilar to the approach proposed by ANAO previously;
  • CASG working with Force Design to identify how to measure capability, that considers all elements of Fundamental Inputs to Capability, and that is suitable for unclassified publication; and
  • Defence is working towards a new whole of organisational reporting system (the Enterprise Resource Planning (ERP) System) which is expected to roll-out in Financial Year 2020-21. CASG will endeavour to incorporate the work conducted with Force Design on measuring capability.126

2.58 In September 2018, the JCPAA noted that ‘Materiel Capability Delivery Performance charts continue to be ambiguous in displaying actual current capability levels.’127

2.59 Defence advised the ANAO in November 2018 that partial progress had been made on its schedule baseline validation activity discussed with the JCPAA. The ANAO notes that a measurement of schedule milestones will not necessarily reflect a measurement of capability delivered.

Modified method of capability reporting

2.60 In light of the above, in 2015–16 the ANAO developed a measure of key materiel capabilities delivered (Capability Delivery Progress). This presents a current assessment of the capability delivered, which differs from Defence’s prediction of final capability. The information used in forming the ANAO’s assessment is primarily based on Section 4.2 of the PDSS, which sets out the capability elements required to achieve Initial Materiel Release and Final Materiel Release, combined with other information in the PDSS reporting the delivery of equipment/achievement of these requirements toward FOC.

2.61 Noting that a system of capability reporting with a robust methodology applicable to materiel acquisition does not exist within Defence, the information presented below is a more meaningful reflection of current project progress than an end-state prediction.

Capability Delivery Progress and Project Maturity

2.62 Figure 12, below, sets out each project’s Capability Delivery Progress against Project Maturity.128 It shows that Capability Delivery Progress lags Project Maturity for the majority of projects (19 of 26). This relationship is expected as projects will typically develop confidence in the ability to deliver their scope and capability through testing and demonstration of capability components (for example, design reviews and acceptance tests) prior to delivery of the majority of equipment.

2.63 Figure 12 also shows that Capability Delivery Progress lags Project Maturity by 20 per cent or more in 12 projects, and for 10 of these, Capability Delivery Progress lags by 50 per cent or more.

Figure 12: Project snapshot — Capability Delivery Progress and Project Maturity

Figure 12

Source: ANAO analysis of the 2018–19 PDSSs.

2.64 As noted in paragraph 2.11, Defence’s project maturity framework attributes approximately 50 per cent of total project maturity at Second Pass Approval.129 As a result, Defence’s project maturity framework is not appropriately structured to assign project maturity progress throughout the project life cycle, particularly within the acquisition phase, which is predominantly the longest and most expensive component.

2.65 Figure 12 also highlights a continuing issue with the level of specification of capability elements. For the projects that show little or no Capability Delivery Progress, this can be attributed to Defence’s high level description of requirements in the capability elements. This indicates that it would be worthwhile for Defence to undertake additional work to track project progress. In respect of the four projects that show no capability delivery at 30 June 2019, progress was as follows:

  • Offshore Patrol Vessel — this project had completed almost all major design review activities;
  • Repl Replenishment Ships — the first ship had been launched and was being outfitted prior to acceptance, while 70 per cent of blocks had been erected for the second ship;
  • CMATS — this project was in early stages of procurement, and was progressing through early design processes; and
  • Maritime Comms — this project had progressed through design reviews and commenced ship installations.

2.66 Further, Figure 12 indicates that:

  • one project, ANZAC ASMD 2B, has delivered all of the required capability;
  • three projects, HATS, Battle Comm. Sys. (Land) 2A, and LHD Landing Craft, have delivered essentially all of their capability with only minor items of capability or administrative processes remaining prior to declaration of FOC;
  • 18 projects are still to deliver part of their capability; and
  • one project, AWD Ships, will not deliver all of the required capability.

!Part 2. Defence Major Projects Report

Part 2. Defence Major Projects Report is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

!Part 3. Assurance by the Auditor-General and the Secretary of Defence

.

Assurance by the Auditor-General and the Secretary of Defence

PRIORITY ASSURANCE REVIEW – SECTION 19A(5) OF THE AUDITOR-GENERAL ACT 1997

Independent Assurance Report

Department of Defence Project Data Summary Sheets

To the President of the Senate
To the Speaker of the House of Representatives

Conclusion

Based on the procedures I have performed and the evidence I have obtained, nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2018–19 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.

The purpose of the Major Projects Report is to report on the performance of selected major Department of Defence (Defence) equipment acquisition projects (Major Projects), since Second Pass Approval, and associated sustainment activities (where applicable), managed by Defence.

I have undertaken a limited assurance review of the PDSSs, reporting on the status of the projects selected by the Joint Committee of Public Accounts and Audit, and the Statement by the Secretary of Defence, for the year-ended 30 June 2019. The following forecast information was excluded from the scope of this engagement:

  1. Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  2. Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  3. forecast dates where included in each PDSS.

The forecast information has not been included in the scope of the engagement, due to the lack of Defence systems from which to provide complete and accurate evidence, in a sufficiently timely manner to facilitate the review. Accordingly, my conclusion does not provide any assurance in relation to this forecast information. However, material inconsistencies identified in relation to the forecast information are required to be considered in forming my conclusion.

Basis for Conclusion

I have undertaken a limited assurance review in accordance with the ANAO Auditing Standards, which include the relevant Standard on Assurance Engagements ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Auditing and Assurance Standards Board.

I believe that the evidence I have obtained is sufficient and appropriate to provide a basis for my conclusion.

Responsibilities of the Secretary of Defence for the Project Data Summary Sheets

The Secretary of Defence is responsible for the preparation and presentation of the PDSSs for the 26 selected projects, and the Statement by the Secretary of Defence, in accordance with the Guidelines. This responsibility includes the design, implementation and maintenance of internal control that the Secretary determines is necessary to enable the preparation of PDSSs that are free from material misstatement, whether due to fraud or error. The Guidelines provide that the PDSSs and supporting evidence, provided to the ANAO for review, are complete and accurate.

Independence and Quality Control

I have complied with the independence and other relevant ethical requirements relating to assurance engagements, and applied Auditing Standard ASQC 1 Quality Control for Firms that Perform Audits and Reviews of Financial Reports and Other Financial Information, Other Assurance Engagements and Related Services Engagements in undertaking this assurance review.

Responsibilities of the Auditor-General

My responsibility is to express an independent limited assurance conclusion on the PDSSs and Statement by the Secretary of Defence, based on the procedures I have performed and the evidence I have obtained. ASAE 3000 requires that I plan and perform my procedures to obtain limited assurance about whether anything has come to my attention that the PDSSs and the Statement by the Secretary of Defence have not, in all material respects, been prepared in accordance with the Guidelines.

In a limited assurance engagement, the assurance practitioner performs procedures, primarily consisting of: making enquiries of managers and others within the entity, as appropriate; the examination of documentation; and the evaluation of the evidence obtained. The procedures selected depend on my judgement, including identifying areas where the risks of material misstatement are likely to arise. The procedures performed are detailed at paragraph 1.7 of Part 1 of this report.

The procedures performed in a limited assurance engagement vary in nature and timing from, and are less in extent than those performed for, a reasonable assurance engagement. Consequently the level of assurance obtained in a limited assurance engagement is substantially lower than the assurance that would have been obtained had a reasonable assurance engagement been performed. Accordingly I do not express a reasonable assurance opinion on whether the PDSSs and the Statement by the Secretary of Defence are prepared in all material respects in accordance with the Guidelines.

Grant Hehir
Auditor-General
Canberra
10 December 2019

Statement by the Secretary of Defence

The attached Project Data Summary Sheets (PDSS) for the 26 major projects included in this report have been prepared in accordance with the Guidelines developed by Defence in consultation with the Australian National Audit Office and endorsed by the Joint Committee of Public Accounts and Audit. 

Project Status as at 30 June 2019

In my opinion, the Project Data Summary Sheets comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2019.

Significant Events Occurring Post 30 June 2019

In stating this opinion that the PDSSs comply in all material respects with the Guidelines, I acknowledge the following material events have occurred post‑30 June 2019:

AIR 6000 Phase 2A/2B - Joint Strike Fighter

Aircraft 17 and 18 were delivered in the United States of America during October 2019 and will be ferried to Australia in December 2019.

In July 2019 the United States Government suspended Turkey's involvement in the global F-35 Joint Strike Fighter Partnership in response to Turkey's acquisition of the Russian S-400 Missile Defence System.

AIR 7000 Phase 2B – Maritime Patrol and Response Aircraft System

Acceptance of the 3rd Mobile Tactical Operations Centre is expected in December.

As at November 2019, 11 aircraft have been accepted with the remaining aircraft to be delivered early in 2020, on time or ahead of schedule.

AIR 9000 Phases 2, 4, 6 – Multi-Role Helicopter

Testing of the Taipan Gun Mount has been partially conducted with further testing due by end 2019. This is one of the activities under the Projects of Concern remediation plan.

Service release of the Enhance Cargo Hook System and subsequent declaration of Operational Capability Milestone 2 has been delayed until mid-2020.

A Projects of Concern Summit was held on 3 December 2019.

SEA 1180 Phase 1 – Offshore Patrol Vessel

The Whole of Ship Detailed Design Review was completed on 18 November 2019.

AIR 5349 Phase 3 – EA-18 Growler Airborne Electronic Attack Capability

Milestone Release 5 is now expected to be achieved in July 2020 due to delays in the US Navy flight clearance of certain specific EA-18G Growler weapons configurations and delays in the supply of some components of CEA threat emulation systems. Air Force has been informed and sufficient mitigations are in place to minimise impact to capability.

JP 2048 Phase 3 – Amphibious Watercraft Replacement

The successful testing of heavy loads during Sea Series exercises was completed in July 2019.

Final Operational Capability was declared by Chief of Navy on 4 November 2019.

JP 2048 Phase 4A/4B – Amphibious Ships

Final Materiel Release was achieved on 18 October 2019, and Final Operational Capability declared by Chief of Navy on 4 November 2019.

LAND 121 Phase 4 – Protected Mobility Vehicle – Light

Thales Australia advised Defence that it had acquired Steyr Motors, with the sale finalised on 23 August 2019. Thales’ procurement of Steyr Motors will ensure the continuity of engine supply and the long-term sustainability of the Hawkei program.

The Production Reliability Acceptance Test continues to inform vehicle reliability, with over 50 per cent of the total test distance completed.

Hawkei Maintainer Training commenced in the fourth quarter 2019 at the Army School of Electrical and Mechanical Engineers. The Integral Computing System Maintainer Training is expected to commence in early 2020.

LAND 121 Phase 4 has taken delivery of the first 138 Hawkei vehicles and trailers required for Initial Materiel Release.

AIR 8000 Phase 2 – Battlefield Airlifter

FMR was not achieved as scheduled in October 2019. Declaration of Final Operational Capability, scheduled for December 2019, is likely to be impacted by the delay to Final Material Release and is under review by the Chief of Air Force.

SEA 1654 Phase 3 – Maritime Operational Support Capability

NUSHIP Stalwart was launched on schedule on 30 August 2019.

AIR 5431 Phase 3 – Civil Military Air Management System

Contract Change Proposal 4 was executed in July 19 resulting in a delay to Initial Operational Capability from November 2022 to July 2023.

Final Operational Capability has been delayed from October 2025 to April 2026 due to previously undisclosed impacts associated with Contract Change Proposal 2.

Contract Change Proposal 5 was executed in October 2019 and incorporates the remaining Defence Collaboration initiatives. This has not introduced any Milestone impact.

AIR 7403 Phase 3 – Additional KC-30A Multi-role Tanker Transport

Contract Final Acceptance was achieved in September 2019.

Final Materiel Release was achieved in October 2019.

Final Operational Capability is expected to be achieved in December 2019.

SEA 1442 Phase 4 – Maritime Communications Modernisation

Initial Materiel Release and Initial Operational Capability is now planned to be achieved in quarter 3 of 2020.

SEA 1448 Phase 4B – ANZAC Air Search Radar Replacement

SEA1448 Phase 4B Initial Operational Capability is at high schedule risk primarily due to complexities in completing the United States Identification Friend or Foe (IFF) certification requirements.

LAND 53 Phase 1BR – Night Fighting Equipment Replacement

Material Release 3 was achieved on 21 November 2019.

JP 2008 Phase 5A – Indian Ocean Region UHF SATCOM

The risk of reaching Network Control System acceptance in December 2019 has increased significantly, and the milestone is now expected to be achieved in March 2020. This will delay Final Materiel Release to no later than September 2020. Final Operational Capability remains the same at December 2021.

SEA 1439 Phase 5B2 – Collins Class Communication and Electronic Warfare Improvement Program

SEA 1439 5B2 Initial Materiel Release and Initial Operational Capability claims were delayed due to finalising all applicable objective quality evidence and availability of safety assessment report and hazard safety controls to support an Initial Materiel Release and Initial Operational Capability claim. It is anticipated that Initial Operational Capability will now be achieved in quarter 1 of 2020.

Rebecca Skinner
Acting Secretary
Department of Defence
10 December 2019

!Part 4. JCPAA 2018–19 Major Projects Report Guidelines

The JCPAA 2018–19 Major Projects Report Guidelines part is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

Footnotes

1 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), Executive Summary, p. 1.

2 ibid.

3 Defence states that CASG ‘purchases and maintains military equipment and supplies in the quantities and to the service levels that are required by Defence and approved by Government’. Department of Defence, About CASG [Internet], Defence, available from http://www.defence.gov.au/casg/AboutCASG/ [accessed 30 October 2019].

4 Department of Defence, Defence Annual Report 2018–19, Defence, Canberra, 2019, Chapter 3, Annual Performance Statements, p. 34.

5 ibid, Appendix A: Financial Statements, p. 176.

6 Department of Defence, Defence Portfolio Budget Statements 2019–20, Defence, Canberra, 2019, p. 5.

7 Department of Defence, Naval Shipbuilding Plan, Defence, Canberra, 2017, p.11. See also Auditor-General Report No.39 2017–18 Naval Construction Programs—Mobilisation.

8 Second pass approval is the Government decision to acquire a fully defined and costed capability. See Defence’s Appendix 5 in Part 2 of this report.

9 M Turnbull (Prime Minister), C Pyne (Minister for Defence Industry), M Payne (Minister for Defence), M Cormann (Minister for Finance), ‘The Hunter Class – defending Australia and securing our shipbuilding sovereignty’, media release, Parliament House, Canberra, 29 June 2018.

10S Morrison (Prime Minister), C Pyne (Minister for Defence), S Ciobo (Minister for Defence Industry), ‘Government Delivers on Future Submarine Program’, media release, Parliament House, Canberra, 11 February 2019.

11 The Future Submarines Design Acquisition (SEA 1000 Phase 1B) and Future Frigates (SEA 5000 Phase 1) projects will be included in the 2019–20 MPR as per the Guidelines endorsed by the JCPAA on 23 September 2019. A performance audit, Future Submarine – transition to design is currently underway by the ANAO, with an expected tabling date of December 2019.

12 The 2018–19 Major Projects Report Guidelines were endorsed by the JCPAA in September 2018 and are included in Part 4 of this report.

13 Department of Defence, Defence Annual Report 2018–19, Defence, Canberra, Chapter 3, Annual Performance Statements, 2019, p. 34.

14 SEA 5000 Phase 1 Future Frigates, SEA 1000 Phase 1B Future Subs, LAND 400 Phase 2 Combat Recon. Vehicles, AIR 7000 Phase 1B Triton Drones, and LAND 200 Tranche 2 Battlefield Command System.

15 The removal of projects from the MPR is generally based on declaration of Final Operational Capability (FOC), or on a pre-FOC risk assessment of the timely declaration of FOC where a significant portion of the project’s deliverables are complete.

16 For example, Defence project risk management records can be managed in spreadsheets, where the risk to the completeness and accuracy of records is too high to be included within the scope of the review.

17 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), Recommendation 2, p. vii.

18 The MPR Guidelines provide that data of a classified nature is to be prepared in such a way as to allow for unclassified publication.

19 IMR and FMR are milestones that Defence utilises to mark the completion and release of acquisition project supplies required to support the achievement of IOC and FOC respectively. They are defined in the relevant MAA (Materiel Acquisition Agreement). See Department of Defence, Defence Materiel Standard Procedure (Project Management) DMSP (PROJ) 11-0-008, Initial Materiel Release And Final Materiel Release Across The Project Lifecycle, Defence, Canberra, 2013, p. 2.

20 Initial Operational Capability is the capability state relating to the in-service realisation of the first subset of a capability system that can be employed operationally. Final Operational Capability is the capability state relating to the in-service realisation of the final subset of a capability system that can be employed operationally. Declaration of IOC and FOC is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group(s) that the fundamental inputs to capability have been delivered. See Defence’s Appendix 5 in Part 2 of this report.

21 The project maturity framework — outlined in the Department of Defence’s Defence Materiel Standard Procedure (Project Management), DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, Defence, Canberra, 2010 — is a methodology used to quantify the maturity of projects as they progress through the acquisition life cycle. See further explanation in paragraphs 1.63–1.67.

22 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. See paragraph 1.61 for further detail.

23 See footnote 21.

24 Individual PDSSs also report on budget variations.

25 The JCPAA requested in May 2018 that the ANAO report back to the Committee on how Defence Major Projects cost variations and the costs of retaining project staff over time might be reported in future MPRs. See paragraphs 1.53 to 1.55 for the outcomes of this consideration.

26 As noted in Note 4 of Table 2, slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project. In November 2017, Defence raised with the ANAO, for the purposes of calculating total schedule slippage, the feasibility of identifying what the proportion of slippage represented by the expanded scope of projects is (for example with respect to the P-8A Poseidon and Additional MRTT projects). See Note 2 of Figure 7 of this report which shows that the slippage attributable to increases in project scope is 94 months.

27 Off-The-Shelf: Systems, hardware or software that already exists or is confirmed in service for an equivalent purpose and requires no, or minimal, change. Sometimes expressed as commercial off-the-shelf or military off-the-shelf. Department of Defence, Interim Defence Test and Evaluation Manual, Defence, Canberra, 2016, Annex 1A, Definitions, p. iii.

28 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, (2014), Recommendation 5, p. 31.

29 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), Recommendation 2, p. vii.

30 Defence has reported that its estimated cost of producing each MPR is $2.4 million on page 74 in Part 2 of this report.

31 Department of Defence, Independent Assurance Reviews for Projects and Sustainment Products, Defence, Canberra, 2016, pp. 3 and 9.

32 Although referred to by Defence as ‘assurance’ reviews, these administrative reviews are not carried out within frameworks issued by the Australian Auditing and Assurance Standards Board.

33 Defence advised in November 2017 that ‘Gate Review’ is now a description for a separate process that leads to Gate submission (to the Investment Committee) including the CASG Independent Assurance Review and the Capability Manager Gate Review.

34 Independent Assurance Reviews were conducted for: Joint Strike Fighter, AWD Ships, P-8A Poseidon, Offshore Patrol Vessels, Growler, Overlander Medium/Heavy, MH-60R Seahawk, Hawkei, Battlefield Airlifter, Repl Replenishment Ships, Battle Comm. Sys. (Land) 2B, CMATS, Additional MRTT, Collins Comms and EW, Pacific Patrol Boat Repl, HATS, Maritime Comms, Battle Comm. Sys. (Land) 2A, ANZAC Air Search Radar Repl, and UHF SATCOM.

35 Department of Defence, Defence Annual Report 2018–19, Chapter 7, Asset Management, Defence, Canberra, 2019, p. 131.

36 Issues in the project were discussed in Auditor-General Report No. 52, 2013–14 Multi-Role Helicopter Program.

37 Auditor-General Report No. 31 2018–19 Defence’s Management of its Projects of Concern, p. 10.

38 Department of Defence, Quarterly Performance Report June 2019, Defence, Canberra, 2019, p. 4.

39 Auditor-General Report No. 3 2019–20 Defence’s Quarterly Performance Report on Acquisition and Sustainment, p. 7.

40 Auditor-General Report No.3 2019–20 Defence’s Quarterly Performance Report on Acquisition and Sustainment, pp. 8–9.

41 ibid, p. 11.

42Department of Defence, Quarterly Performance Report June 2019, Defence, Canberra, 2019, pp. 19-21.

43ibid, pp. 14-15.

44 Similar to the PDSSs, the QPR provides a summary of projects’ performance in the areas of cost, schedule and capability. However, there are some differences between the measures used, and the level of detail provided. For example, both the PDSSs and the QPR use a ‘traffic light indicator’ to reflect capability delivery, but the indicators are defined differently between the two products. In the PDSSs, Amber capability is defined as ‘under threat but still considered able to be met’, whereas the QPR defines Amber capability as ‘major elements of scope about to fail against the baseline’. In addition, the QPR allows for only one indicator to be used in the assessment, i.e. ‘all Green’, ‘all Amber’ or ‘all Red’. In contrast, the Pie Chart in the PDSSs allows for a breakdown of capability, with individual components assessed as Green, Amber or Red, providing a more detailed assessment (see paragraphs 2.48–2.59).

45 The September 2019 QPR has been examined in the context of reviewing the Statement by the Secretary of Defence relating to significant events occurring post 30 June 2019.

46These are CASG acquisition projects that have variances significant enough — in the areas of schedule, cost, and/or capability performance — to warrant attention from senior management. Department of Defence, Quarterly Performance Report June 2019, Defence, Canberra, 2019, p. 36.

47Department of Defence, Quarterly Performance Report June 2019, Defence, Canberra, 2019, p. 38.

48ibid, p. 37.

49Department of Defence, Quarterly Performance Report June 2019, Defence, Canberra, 2019, p. 41.

50ibid, p. 51. See Note 2, Table 4.

51ibid, p. 39.

52 Department of Defence, Quarterly Performance Report March 2019, Defence, Canberra, 2019, p. 27.

53 Repl Replenishment Ships and UHF SATCOM had 7 and 21 months of in-year slippage to their FOC dates respectively. Hawkei did not experience slippage to FOC, but its IMR and IOC milestones slipped by 12 months.

54 Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, pp. 14 and 93.

55 The Project Directive defines the Project, in terms of fundamental inputs to capability, together with the resources necessary to deliver the project and is developed in accordance with the parameters agreed by government. Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, p. 93.

56 The Interim Capability Life Cycle Manual does not describe MAAs and instead refers to Product Delivery Agreements (PDAs) (see paragraph 1.29). Projects in this MPR have an approved MAA.

57 Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, pp. 14 and 93.

58 Auditor-General Report No.6 2013–14 Capability Development Reform, p. 232.

59 Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, p. 14.

60 A PDA is an agreement between the Sponsor and Lead Delivery Group which specifies the scope, resourcing, priorities and performance and preparedness requirements for support of a capability system throughout its life, to support performance measurement. Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, p. 92.

61 Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 31 March 2017, Mr. K Gillis, Deputy Secretary, Capability Acquisition and Sustainment Group, Department of Defence, p. 11.

62 Joint Strike Fighter, Overlander Medium/Heavy, Hawkei, Battlefield Airlifter, CMATS, and Night Fighting Equip Repl.

63 Department of Defence, (PM) 003, CASG Project Controls Manual, Acronyms, Abbreviations and Definitions, 2017, p. 8.

64 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 436: Review of the 2011–12 Defence Materiel Organisation Major Projects Report, (2013), paragraph 3.4, p. 14.

65 Defence advised the ANAO in November 2019 that scope elements (principally the Engine Maintenance Repair Overhaul and Upgrade facility, worth $0.08 billion) were brought forward from later phases into the Joint Strike Fighter project without commensurate funding transfer.

66Department of Defence, Management of Contingency Budgets in Defence Acquisition Projects, Defence, Canberra, 2019, p. 2.

67ibid, p. 5.

68In August 2019, Defence issued version 2.5 of the PRMM document. This latest version will be used to review projects’ contingency statement and management in the 2019–20 MPR.

69Department of Defence, Management of Contingency Budgets in Defence Acquisition Projects, Defence, Canberra, 2019, p. 3.

70Department of Defence, Management of Contingency Budgets in Defence Acquisition Projects, Defence, Canberra, 2019, p. 4.

71 Auditor-General Report No.26 2017–18 2016–17 Major Projects Report, p. 41.

72 The reporting of cost variations was also raised at the JCPAA’s public hearing into the 2016–17 MPR on 23 March 2018 and at estimates hearings of the Finance and Public Administration Legislation Committee on 27 February 2018.

73 See paragraph 1.3 in Part 1 of this report for more information.

74 See Part 2 of this report.

75 Predict! is a risk management tool used by Defence to manage risks and issues.

76 The Defence Project Risk Management Manual version 2.4, Business Rule 2 requires the project manager to validate the currency of the RMP on transition from one stage of the Materiel Life Cycle to the next stage and, for any stage that is longer than six months, every six months within that stage.

77 As at 30 June 2019, Defence risk management guidance for acquisition projects was the DMM (PROJ) 11-0-002 Project Risk Management Manual, version 2.4, 2013; and guidance for sustainment products was the DMM (LOG) 04-0-003, Defence Materiel Manual (Logistics Management), which provide different consequence and likelihood descriptors.

78 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error.

79 The 15 projects are: MRH90 Helicopters, Growler, Overlander Medium/Heavy, LHD Ships, Hawkei, Battlefield Airlifter, Repl Replenishment Ships, Additional MRTT, ANZAC ASMD 2B, Collins Comms and EW, Collins R&S, Battle Comm. Sys. (Land) 2A, ANZAC Air Search Radar Repl, UHF SATCOM, and LHD Landing Craft.

80 The five projects are: AWD Ships, P-8A Poseidon, Offshore Patrol Vessel, Battle Comm. Sys. (Land) 2B, and Pacific Patrol Boat Repl.

81 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), List of Recommendations, p. vii.

82 Department of Defence, DMM (PROJ) 1-0-001, DMO Project Management Manual 2012, Defence, Canberra, 2012, p. 75. This manual has since been superseded by PM 002 CASG Project Management Manual which does not refer to project maturity.

83 See Appendix 4 in Part 2 of this report and footnote 19 for further detail.

84 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014–15), (2016), Recommendation 3, p. 50. The JCPAA sought an update from Defence in the course of public hearings on 31 March 2017.

85 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), Recommendation 2, p. vii.

86 Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 23 March 2018, Mr Greg Divall, Group Business Manager, Capability Acquisition and Sustainment Group, Department of Defence, p. 11.

87 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), Recommendation 1, p. vii.

88 Auditor-General Report No.20 of 2017–18, 2017–18 Major Projects Report, paragraphs 1.61–1.62, p. 32.

89 The following projects which have exited the MPR, had also achieved FOC with caveats: Wedgetail (achieved FOC with caveats in 2015), Overlander Light (achieved FOC with caveats in 2016), ARH Tiger Helicopters (achieved FOC with caveats in 2016).

90 See the LHD Ships PDSS in Part 3 of this report.

91 This requirement was included in the 2018–19 Major Projects Report Guidelines endorsed by the JCPAA in September 2018 which are included in Part 4 of this report.

92 A project’s budgeted cost and schedule data is at 30 June 2019, and may differ from originally approved budgets and schedules.

93 Between First Pass Approval in May 2015 and Second Pass Approval in August 2017, Government approved $65.6 million of funding to undertake these activities.

94 The JCPAA has previously recommended that Defence work with the ANAO to review and revise its policy regarding Project Maturity Scores. Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014–15), (2016), pp. 49–50, and Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), pp. 9–10. In Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016-17), (2018), the JCPAA recommended that Defence advise the Committee of progress in updating Project Maturity Scores.

95 The JCPAA has observed that ‘Defence remains behind the Committee’s expectations on working to update Project Maturity Scores – the Committee recommended reform in this area several years ago, and changes remain slow and uncertain.’ Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016-17), (2018), p. 4. Refer to paragraphs 1.63 to 1.67.

96 See also paragraphs 1.38 to 1.39.

97 See Note 2 of Figure 4, below, for further information.

98 Australian Government arrangements for foreign exchange variation involve ‘no win/no loss’ supplementation. As a matter of policy, unless specifically approved, individual entities are not permitted to ‘hedge’ against foreign exchange risk.

99 Real Variations include ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; ‘Budgetary Adjustments’ made to account for corrections resulting from foreign exchange or indexation accounting estimation errors; ‘Real Cost Increases’, where funds have been approved by government to increase the Project’s budget (generally without a change in scope); and ‘Real Cost Decreases’, where funds have been handed back to the Defence portfolio.

100 See Table 2 in Part 1 of this report.

101 Extensions to planned withdrawal dates may involve additional costs relating to the maintenance and servicing of equipment.

102 M Kinnaird, Defence Procurement Review 2003, Department of Defence, Canberra, 2003.

103 Refer to footnote 21 for more detail.

104 Refer to footnote 26.

105 M Kinnaird, Defence Procurement Review 2003, Department of Defence, Canberra, 2003.

106 Refer to paragraph 32 for a discussion of definitions for these different acquisition types.

107 Of the nine projects added to the MPR since 2016–17, four have been developmental (Hawkei, CMATS, Battle Comm. Sys. (Land) 2B) and ANZAC Air Search Radar Repl). Of these projects, CMATS has experienced 28 months of schedule slippage and Battle Comm. Sys. (Land) 2B has experienced 24 months of schedule slippage, while Hawkei and ANZAC Air Search Radar Repl are yet to experience slippage to FOC milestones.

108 D Peever, First Principles Review: Creating One Defence, Department of Defence, Canberra, 2015, p. 35. Defence’s implementation of the First Principles Review was examined in Auditor-General Report No.34 of 2017–‍18 Defence’s Implementation of the First Principles Review.

109 ibid pp. 34 and 92.

110 Schedule slippage is defined in footnote 25.

111 See the ANZAC ASMD 2B and Collins R&S PDSSs in Part 3 of this report.

112 Auditor-General Report No.6 2013–14 Capability Development Reform, paragraphs 9.1 to 9.4, pp. 198–199.

113 Further information on MRH90 Helicopters can be found in Auditor-General Reports No.48 2008–09, Planning and Approval of Defence Major Capital Equipment Projects, pp. 84, 90 and 133; No.52 2011–12 Gate Reviews for Defence Capital Acquisition Projects, pp. 86–87 and pp. 130–133; and No.52 2013–14 Multi-Role Helicopter Program.

114Similarly, the ARH Tiger Helicopters project, which exited the Major Projects Report in 2016–17, was originally misidentified as MOTS by Defence and was subsequently reclassified as being more developmental. See Auditor-General Report No.11 2016–17 Tiger—Army’s Armed Reconnaissance Helicopter, paragraph 1.7 and paragraph 2.3.

115 In the Statement by the Secretary of Defence in Part 3 of this report, the Acting Secretary also makes reference to additional information on achieved milestone dates for Offshore Patrol Vessel, LHD Ships, Repl Replenishment Ships, Additional MRTT, Night Fighting Equip Repl, and LHD Landing Craft.

116 Tables 4 and 5, on pages 15 and 16 respectively, report on the slippage for each project that has been in the MPR since 2007–08.

117 Department of Defence, Interim Capability Life Cycle Manual, Defence, Canberra, 2017, p. 85.

118 ibid p. 13.

119 As per the 2018–19 MPR Guidelines, a project is defined as the acquisition or upgrade of Specialist Military Equipment, which normally excludes facilities and other Fundamental Inputs to Capability. The 2018–19 MPR Guidelines also note that the MPR may report on associated sustainment activities (where applicable).

120 Auditor-General Report No.17 2010–11 2009–10 Major Projects Report, p. 35.

121 Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 17 March 2016, Mr. P Croser, Director General, Specialist Ships Acquisition, Capability Acquisition and Sustainment Group, Department of Defence, p. 5;

Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 31 March 2017, Mr K Gillis, Deputy Secretary, Capability Acquisition and Sustainment Group, Department of Defence, p. 14;

Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 23 March 2018, CDRE S Hughes, Deputy General, Littoral, Department of Defence, p. 13.

122 The desktop analysis considered 65 tonnes to be the weight of an M1A1 Main Battle Tank.

123 The PDSS references issues relating to the design and build quality of the aircraft that are expected to have an ongoing effect on sustainment. The aircraft has been affected by structural fatigue, difficulty obtaining spares, low availability, poor build quality and design limitations. See the Battlefield Airlifter PDSS in Part 3 of this report.

124 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, (2014), pp. 37–39;

Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014-15), (2016), pp. 48–49.

125 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), Recommendation 1, p. vii.

126 Department of Defence, written submission to the Joint Committee of Public Accounts and Audit, Inquiry into the 2016-17 Defence Major Projects Report, p. 1.

127 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), p. 2.

128 Refer to footnote 95 for more detail.

129 Refer to footnote 96 for more detail.