The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects. The status of the selected Major Projects is reported in the Statement by the Secretary of Defence and the Project Data Summary Sheets (PDSSs) prepared by Defence. Assurance from the ANAO’s review is conveyed in the Independent Assurance Report by the Auditor-General.

Due to the complexity of material and the multiple sources of information for the 2017-18 Major Projects Report, we are unable to represent the entire document in HTML. You can download the full report in PDF or view selected sections in HTML below. PDF files for individual Project Data Summary Sheets (PDSS) are also available for download.

!Part 1. ANAO Review and Analysis

Summary and Review Conclusion

About the Major Projects Report

1. Major Defence equipment acquisition projects (Major Projects) continue to be the subject of parliamentary and public interest. This is due to their high cost and contribution to national security, and the challenges involved in completing them within the specified budget and schedule, and to the required capability.

2. The Australian National Audit Office (ANAO) has reviewed 26 of Defence’s Major Projects in this eleventh annual report (2016–17: 27). The Major Projects Report (MPR) reviews overall issues, risks, challenges and complexities affecting Major Projects and also reviews the status of each of the selected Major Projects, in terms of cost, schedule and forecast capability.1 The objective of the report is ‘to improve the accountability and transparency of Defence acquisitions for the benefit of Parliament and other stakeholders.’2

3. The Capability Acquisition and Sustainment Group (CASG) within the Department of Defence (Defence), manages the process of bringing new capabilities into service.3 In 2017–18 Defence was managing 198 active major and minor capital equipment projects worth $103.5 billion, with an in-year budget of $6.9 billion.4 Defence capitalised some $7.5 billion from these projects in 2017–18.5

4. The February 2016 Defence White Paper established the Government’s priorities for future capability investment for the next 20 years and provided for additional spending of over $29 billion across the next decade. The 2018–19 Defence Portfolio Budget Statements indicated that the Defence budget would grow approximately $200 billion over the coming decade, for investing in Defence capability.6 The Government commenced its $89 billion investment in Australia’s future shipbuilding industry in April 20177, and on 29 June 2018 announced Second Pass Approval of the $35 billion Future Frigate program.8

Major Projects selected for review

5. Major Projects are selected for review based on the criteria included in the 2017–18 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).9 They represent a selection of the most significant Major Projects managed by Defence.

6. The total approved budget for the Major Projects included in this report is approximately $59.4 billion, covering 57 per cent of the total budget of active major and minor capital equipment projects of $103.5 billion.10 The selected projects are listed in Table 1.

Table 1: 2017–18 MPR projects and approved budgets at 30 June 2018 1, 2, 3

Project Number (Defence Capability Plan)

Project Name (on Defence advice)

Abbreviation (on Defence advice)

Approved Budget $m

AIR 6000 Phase 2A/2B

New Air Combat Capability

Joint Strike Fighter

15,504.0

SEA 4000 Phase 3

Air Warfare Destroyer Build

AWD Ships

9089.3

AIR 7000 Phase 2B

Maritime Patrol and Response Aircraft System

P-8A Poseidon

5212.0

AIR 9000 Phase 2/4/6

Multi-Role Helicopter

MRH90 Helicopters

3771.1

AIR 5349 Phase 3

EA-18G Growler Airborne Electronic Attack Capability

Growler

3430.4

AIR 9000 Phase 8

Future Naval Aviation Combat System Helicopter

MH-60R Seahawk

3430.3

LAND 121 Phase 3B

Medium Heavy Capability, Field Vehicles, Modules and Trailers

Overlander Medium/Heavy

3428.9

JP 2048 Phase 4A/4B

Amphibious Ships (LHD)

LHD Ships

3091.7

LAND 121 Phase 4

Protected Mobility Vehicle – Light

Hawkei

1952.0

AIR 8000 Phase 2

Battlefield Airlift – Caribou Replacement

Battlefield Airlifter

1433.3

SEA 1654 Phase 3

Maritime Operational Support Capability

Repl Replenishment Ships

1066.8

 

AIR 5431 Phase 3

Civil Military Air Traffic Management System

CMATS

974.2

JP 2072 Phase 2B

Battlespace Communications Systems

Battle Comm. Sys. (Land) 2B

920.1

AIR 7403 Phase 3

Additional KC-30A Multi-role Tanker Transport

Additional MRTT

887.8

SEA 1448 Phase 2B

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2B

678.7

SEA 3036 Phase 1

Pacific Patrol Boat Replacement

Pacific Patrol Boat Repl

501.2

AIR 9000 Phase 7

Helicopter Aircrew Training System

HATS

481.5

SEA 1439 Phase 4A

Collins Replacement Combat System

Collins RCS

450.5

JP 2072 Phase 2A

Battlespace Communications System

Battle Comm. Sys. (Land) 2A

438.0

SEA 1442 Phase 4

Maritime Communications Modernisation

Maritime Comms.

437.7

SEA 1429 Phase 2

Replacement Heavyweight Torpedo

Hw Torpedo

427.6

JP 2008 Phase 5A

Indian Ocean Region UHF SATCOM

UHF SATCOM

419.9

SEA 1439 Phase 3

Collins Class Submarine Reliability and Sustainability

Collins R&S

411.6

SEA 1448 Phase 2A

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2A

386.8

LAND 75 Phase 4

Battle Management System

BMS

367.9

JP 2048 Phase 3

Amphibious Watercraft Replacement

LHD Landing Craft

236.7

Total 26

 

 

59,430.0

    

Note 1: Once a project is selected for review, it remains within the portfolio of projects under review until the JCPAA endorses its removal, normally once it has met the capability requirements of Defence.

Note 2: SEA 1654 Phase 3 Maritime Operational Support Capability (Repl Replenishment Ships), JP 2072 Phase 2B Battlespace Communications System (Battle Comm. Sys. (Land) 2B) and SEA 3036 Phase 1 Pacific Patrol Boat Replacement (Pacific Patrol Boat Repl) are included in the MPR for the first time in 2017–18.

Note 3: SEA 1439 Phase 3 Collins Class Submarine Reliability and Sustainability is a group of 22 activities primarily sustainment in nature. While not an acquisition project, it has been included at the JCPAA’s request.

Source: The Project Data Summary Sheets in Part 3 of this report.

Report objective and scope

7. The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects. The status of the selected Major Projects is reported in the Statement by the Secretary of Defence and the Project Data Summary Sheets (PDSSs) prepared by Defence. Assurance from the ANAO’s review is conveyed in the Independent Assurance Report by the Auditor-General.

8. The following forecast information is excluded from the scope of the ANAO’s review:

  • Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  • Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  • forecast dates where included in each PDSS.
  • Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.

9. The exclusions to the scope of the review noted above are due to a lack of Defence systems from which to provide complete and accurate evidence11, in a sufficiently timely manner to facilitate the review. This has been an area of focus of the JCPAA over a number of years12, and it is intended that all components of the PDSSs will eventually be included within the scope of the ANAO’s review.

10. Separate to the formal review, the ANAO has undertaken an analysis of key elements of the PDSSs — including cost, schedule, progress towards delivery of required capability, project maturity, and risks and issues. Longitudinal analysis across these key elements of projects has also been undertaken.

11. Defence provides further insights and context in its commentary and analysis. This commentary and analysis is not included within the scope of the ANAO’s review.

Review methodology

12. The ANAO has reviewed the PDSSs prepared by Defence as a priority assurance review under subsection 19A(5) of the Auditor-General Act 1997. The criteria to conduct the review are provided by the Guidelines approved by the JCPAA, and include whether Defence has procedures in place designed to ensure that project information and data was recorded in a complete and accurate manner, for all 26 projects.

13. The review included an assessment of Defence’s systems and controls, including the governance and oversight in place, to ensure appropriate project management. The ANAO also sought representations and confirmations from Defence senior management and industry in relation to the status of the Major Projects in this report.

Report structure

14. The report is organised into four parts:

  • Part 1 comprises the ANAO’s review and analysis;
  • Part 2 comprises Defence’s commentary, analysis and appendices (not included within the scope of the Independent Assurance Report by the Auditor-General);
  • Part 3 incorporates the Independent Assurance Report by the Auditor-General, the Statement by the Secretary of Defence, and the PDSSs prepared by Defence as part of the assurance review process; and
  • Part 4 reproduces the 2017–18 Major Projects Report Guidelines endorsed by the JCPAA, which provide the criteria for the compilation of the PDSSs by Defence and the ANAO’s review.

Figure 1, below, depicts the four parts of this report.

Figure 1: 2017–18 Report structure

2017–18 Report structure

Note: To assist in conducting inter-report analysis, the presentation of data in the PDSSs remains largely consistent and comparable with the 2016–17 MPR.

Project Data Summary Sheets

15. The PDSSs include unclassified information on project performance, prepared by Defence.13 As projects appear in the MPR for multiple years, changes to the PDSS from the previous year are depicted in bold orange text in the PDSS.

16. Each PDSS comprises:

  • Project Header: including name; capability and acquisition type; Capability Manager; approval dates; total approved and in-year budgets; stage; complexity; and an image;
  • Section 1—Project Summary: including description; current status including financial assurance and contingency statement; and context, including background, uniqueness, major risks and issues, and other current sub-projects;
  • Section 2—Financial Performance: including budgets and expenditure; variances; and major contracts in place (in addition to quantities delivered as at 30 June 2018);
  • Section 3—Schedule Performance: providing information on design development; test and evaluation; and forecasts and achievements against key project milestones, including Initial Materiel Release (IMR), Final Materiel Release (FMR)14, Initial Operational Capability (IOC) and Final Operational Capability (FOC)15;
  • Section 4—Materiel Capability Delivery Performance: provides a summary of Defence’s assessment of its expected delivery of key capabilities, the extent to which milestones were achieved (particularly where caveats are placed on the Capability Manager’s declaration of significant milestones), and a description of the constitution of each key milestone;
  • Section 5—Major Risks and Issues: outlines the major risks and issues of the project and remedial actions undertaken for each;
  • Section 6—Project Maturity: provides a summary of the project’s maturity, as defined by Defence16, and a comparison against the benchmark score;
  • Section 7—Lessons Learned: outlines the key lessons that have been learned at the project level (further information on lessons learned by Defence are included in Defence’s Appendix 2); and
  • Section 8—Project Line Management: details current project management responsibilities within Defence.

Overall outcomes

Statement by the Secretary of Defence

17. The Statement by the Secretary of Defence was signed on 11 December 2018. The Secretary’s statement provides his opinion that the PDSSs for the 26 selected projects ‘comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2018’.

18. In addition, the Statement by the Secretary of Defence details significant events occurring post 30 June 2018, which materially impact the projects included in the report, and which should be read in conjunction with the individual PDSSs. These include: Joint Strike Fighter, AWD Ships, P-8A Poseidon, Growler, Overlander Medium/Heavy, LHD Ships, Hawkei, Repl Replenishment Ships, Battle Comm. Sys. (Land) 2B, ANZAC ASMD 2A and 2B, Pacific Patrol Boat Repl, HATS, Battle Comm. Sys. (Land) 2A, Maritime Comms, Collins RCS, Hw Torpedo and LHD Landing Craft.

19. The 2017–18 MPR Guidelines require Defence to report in the Statement by the Secretary of Defence on projects which have been removed from the MPR which still have outstanding caveats. The status of the caveats of the ARH Tiger Helicopter Project, which achieved FOC in 2016 with caveats, has been reported in the Statement in Part 3 of this report.

Conclusion by the Auditor-General

20. The Auditor-General has concluded in the Independent Assurance Report for 2017–18 that ‘nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2017–18 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.’

21. Additionally, in 2017–18, a number of observations were made in the course of the ANAO’s review, as summarised below:

  • non-compliance with corporate guidance resulting in inconsistent approaches taken to contingency allocation (Section 1 of the PDSS). See further explanation in paragraphs 1.34 to 1.41;
  • a change to the basis of financial reporting (Section 2 of the PDSS). See further explanation in paragraphs 1.42 to 1.44;
  • enhanced transparency by reporting cost variations since Second Pass Approval and personnel costs. See further explanation in paragraphs 1.45 to 1.49;
  • a lack of oversight, non-compliance with corporate guidance and the use of spreadsheets in the management of risks and issues (Section 5 of the PDSS). See further explanation in paragraphs 1.50 to 1.5617;
  • outdated policy guidance for the project maturity framework (Section 6 of the PDSS). See further explanation in paragraphs 1.57 to 1.6018; and
  • a decrease in the number of MPR projects which have achieved significant milestones with caveats. See further explanation in paragraphs 1.61 to 1.62.

ANAO’s analysis of project performance

22. As discussed, the ANAO has undertaken an analysis of key elements of the Defence PDSSs — cost, schedule, progress towards delivery of required capability, project maturity, risks and issues, and longitudinal analysis across these key elements of projects. Table 2 provides: summary data on Defence’s progress toward delivering the capabilities for the Major Projects covered in this report; and compares current data against that reported in previous editions of the MPR. This section also contains a summary analysis of the three principal components of project performance: cost, schedule and capability.

Table 2: Summary longitudinal analysis1

 

2015–16 MPR

2016–17 MPR

2017–18 MPR

Number of Projects

26

27

26

Total Approved Budget

$62.7 billion

$62.0 billion

$59.4 billion

Total Expenditure Against Total Approved Budget

$29.4 billion (46.9%)

$32.1 billion (51.7%)

$32.4 billion (54.5%)

Total In-year Expenditure Against In-year Budget

$3.9 billion (91.2%)

$4.1 billion (96.6%)

$4.6 billion (98.6%)

Total Budget Variation since initial Second Pass Approval 2

$23.6 billion (37.6%) 3

$22.3 billion (36.0%) 3

$23.0 billion (38.7%)

Total Budget Variation since final Second Pass Approval 4

$9.8 billion

(15.7%)

$8.5 billion

(13.7%)

$9.2 billion

(15.5%)

In-year Approved Budget Variation

$4.9 billion (7.8%)

-$1.6 billion (-2.6%)

-$0.3 billion (-0.5%)

Total Schedule Slippage 5

708 months (26%)

793 months (29%)

801 months (32%)

Average Schedule Slippage per Project

28 months

30 months

32 months

In-year Schedule Slippage 6

42 months (1%)

149 months (6%)

104 months (5%)

Total Project Maturity 7

1479 / 1820 (81%)

1531 / 1890 (81%)

1484 / 1820 (82%)

Total Reported Risks and Issues 8, 9

123

136

138

Expected Capability (Defence Reporting)

High level of confidence of delivery (Green)

99%

98%

99%

Under threat, considered manageable (Amber)

1%

2%

1%

Unlikely to be met (Red)

0% 10

0% 10

0%

    

Refer to paragraphs 22 to 36 in Part 1 of this report.

Note 1: The data for the 26 Major Projects in the 2017–18 MPR compares the data from projects in the 2016–17 MPR and 2015–16 MPR. The Major Projects included within each MPR are based on entry and exit criteria in the Guidelines, which have been included in Part 4 of this report. The entry and exit of projects should be considered when comparing data across years.

Note 2: Where a project has multiple Second Pass Approvals, the MPR has historically reported budget variations from the initial Second Pass Approval. The figures in this row are consistent with prior year reporting. See Table 3 for a breakdown of the major components of this variance.

Note 3: These figures include a $0.8 billion correction to an error in prior year data.

Note 4: In the 2017–18 PDSSs, where a project has multiple Second Pass Approvals, the budget at Second Pass Approval reported in the Header refers to the total budget as at the final Second Pass Approval. The figures in this row use this methodology.

Note 5: Slippage refers to the difference between the original government approved date and the current forecast date. Slippage can occur due to late delivery, increases in scope or at times can be a deliberate management decision. These figures exclude schedule reductions over the life of the project. However, Figure 10 reports in-year schedule reductions.

Note 6: Based on the 23 repeat projects from the 2014–15 MPR, 25 repeat projects from the 2015–16 MPR plus one new project (CMATS) that had slippage in 2016–17, and 23 repeat projects from the 2016–17 MPR respectively.

Note 7: The figures represent the total of the reported maturity scores divided by the total benchmark maturity score, in the PDSSs across all projects.

Note 8: The figures represent the combined number of open high and extreme risks and issues reported in the PDSSs across all projects. Risks and issues may be aggregated at a strategic level.

Note 9: The grey section of the table is excluded from the scope of the ANAO’s priority assurance review, due to a lack of systems from which to obtain complete and accurate evidence in a sufficiently timely manner to facilitate the review.

Note 10: Defence advised in these years that Joint Strike Fighter would not deliver one element of capability at FOC (which equated to approximately one per cent). However, across all the Major Projects this percentage rounded to zero per cent.

Cost

23. Cost management is an ongoing process in Defence’s administration of the Major Projects. While all projects reported that they could continue to operate within the total approved budget of $59.4 billion, CMATS was granted a Real Cost Increase of $240.7 million by government in February 2018.19,20 In addition, MRH90 Helicopters, Battle Comm. Sys (Land) 2B, UHF SATCOM and BMS have been required to draw upon contingency funds to complete project activities.

24. The approved budget for Major Projects included in this MPR has increased by $23.0 billion (38.7 per cent) since initial Second Pass Approval. Budget variations greater than $500 million are detailed in Table 3, below. However, as the MPR predominantly focusses on the approved capital budget, the ongoing costs of Project Offices21 (acquisition), training, replacement capability, etc., are not reported here.

Table 3: Budget variation over $500m post initial Second Pass Approval by variation type 1, 2

Project

Variation

Explanation

Year

Amount $b

 

Scope Increases

 

 

 

14.1

MRH90 Helicopters

 

34 additional aircraft at Phase 4/6 Second Pass Approval

2005–06

2.3

 

Joint Strike Fighter

 

58 additional aircraft at Stage 2 Second Pass Approval

2013–14

10.5

 

P-8A Poseidon

 

Four additional aircraft

2015–16

1.3

 

 

Real Cost and other Increases

 

 

 

1.8

AWD Ships

 

Real Cost Increase of $1.2b offset by $0.1b transfer for facilities in 2014

2013–14 and 2015–16

1.1

 

Overlander Medium/Heavy

 

Project supplementation ($684.2m) and additional vehicles, trailers and equipment ($28.0m) at Revised Second Pass Approval

2013–14

0.7

 

 

Other budget movements

 

 

 

1.1

Other

Scope increase/budget transfers (net)

Other scope changes and transfers

Various

1.1

 

 

Price Indexation – materials and labour (net) (to July 2010) 3

 

3.0

Exchange Variation – foreign exchange (net) (to 30 June 2018)

 

3.0

 

Total

 

 

 

23.0

      

Note 1: For the variations related to all projects and values refer to Table 8 of this report. For the breakdown of in-year variation, refer to Table 9 of this report.

Note 2: For projects with multiple Second Pass Approvals, this table shows variations from the initial approval.

Note 3: Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.

Source: ANAO analysis of the 2017–18 PDSSs.

Schedule

25. Delivering Major Projects on schedule continues to present challenges for Defence22; affecting when the capability is made available for operational release and deployment by the Australian Defence Force, as well as the cost of delivery.

26. The total schedule slippage for the 26 selected Major Projects, as at 30 June 2018, is 801 months (2016–17: 793 months) when compared to the initial schedule.23 This represents a 32 per cent (2016–17: 29 per cent) increase since Second Pass Approval. Table 4 below includes details of in-year and total schedule slippage by project. While the table shows a five per cent in-year slippage for 2017–18, the removal of completed projects (ARH Tiger Helicopters, Bushmaster Vehicles, Overlander Light, and Additional Chinook) has removed 98 months of slippage. The effect of these projects exiting the review explains the difference between the total schedule slippage in 2017–18 (8 months) and the total in-year slippage amount (104 months). Additionally, the Pacific Patrol Boat Replacement project added two months of slippage to the total of 801 months; the slippage occurred in 2015–16 but the project was reported in the MPR for the first time in 2017–18.

Table 4: Schedule slippage from original planned Final Operational Capability 1

Project

In-year (months)

Total (months)

Joint Strike Fighter 3

0

2

AWD Ships

0

35

P-8A Poseidon

4

28

MRH90 Helicopters

29

89

Growler

1

1

MH-60R Seahawk

0

0

Overlander Medium/Heavy

0

5

LHD Ships 3

0

37

Hawkei

0

0

Battlefield Airlifter

0

24

Repl Replenishment Ships

0

0

CMATS 3

0

28

Battle Comm. Sys. (Land) 2B

0

0

Additional MRTT

0

21

ANZAC ASMD 2B

10

67

Pacific Patrol Boat Repl

0

2

HATS

0

0

Collins RCS

0

109

Battle Comm. Sys. (Land) 2A

13

30

Maritime Comms

7

7

Hw Torpedo

0

63

UHF SATCOM 3

12

21

Collins R&S

13

112

ANZAC ASMD 2A

10

82

BMS 2

N/A

N/A

LHD Landing Craft 4

5

38

Total (months)

104

801

Total (%)

5

32

   

Note 1: Refer to footnote 23.

Note 2: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Note 3: These projects have been identified by Defence as Projects of Interest (see paragraph 1.17 in Part 1).

Note 4: The LHD Landing Craft PDSS shows an FOC forecast of ‘TBA’ (See the PDSS in Part 3 of this report). For the purposes of slippage analysis in this report, the ANAO has reflected the slippage that had occurred for the project at the time of this report. The Statement by the Secretary of Defence in Part 3 of this report notes that since 30 June 2018, FOC for this project has been rescheduled for the second half of 2019.

Source: ANAO analysis of the 2017–18 PDSSs.

27. Platform availability has contributed to the slippage experienced within some projects. For example, the submarine programs have been impacted by changes to docking schedules, following government commissioned reviews. Significant delays have also been experienced by those projects with the most developmental content: AWD Ships, MRH90 Helicopters, CMATS and ANZAC ASMD 2B. Additionally, delays to operational test and evaluation activities have led to delays to the LHD Ships and LHD Landing Craft projects.

28. Table 5, below, provides details of total schedule slippage by project, for projects that have exited the MPR. Compared to the 801 months total schedule slippage for the current 26 Major Projects, the 18 projects which have exited the MPR have reported accumulated schedule slippage of 699 months, as at their respective exit dates. Again, schedule slippage was more pronounced in projects with the most developmental content.

Table 5: Schedule slippage for projects which have exited the MPR

Project

Total (months)

Wedgetail (Developmental)

78

Super Hornet (MOTS)

0

Hornet Upgrade (Australianised MOTS)

39

ARH Tiger Helicopters (Australianised MOTS)

82

C-17 Heavy Airlift (MOTS)

0

Air to Air Refuel (Developmental)

64

FFG Upgrade (Developmental)

132

Bushmaster Vehicles (Australianised MOTS)

1

Overlander Light (Australianised MOTS)

9

Next Gen Satellite 1 (MOTS)

0

Additional Chinook

6

HF Modernisation (Developmental)

147

Armidales (Australianised MOTS)

45

SM-2 Missile (Australianised MOTS)

26

155mm Howitzer (MOTS)

7

Stand Off Weapon (Australianised MOTS)

37

Battle Comm. Sys. (Australianised MOTS)

24

C-RAM (MOTS)

2

Next Gen Satellite 1 (MOTS)

0

Additional Chinook

6

HF Modernisation (Developmental)

147

Armidales (Australianised MOTS)

45

SM-2 Missile (Australianised MOTS)

26

155mm Howitzer (MOTS)

7

Stand Off Weapon (Australianised MOTS)

37

Battle Comm. Sys. (Australianised MOTS)

24

C-RAM (MOTS)

2

Total

699

  

Note 1: Next Gen Satellite shows slippage in Figure 8, which related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced and slippage was reduced.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

29. Additional ANAO analysis (refer to Figure 7) has compared project slippage against the Defence classification of projects as Military Off-The-Shelf (MOTS), Australianised MOTS or developmental.24 These classifications are a general indicator of the difficulty associated with the procurement process. This analysis highlights, prima facie, that the more developmental in nature a project is, the more likely it will result in a greater degree of project slippage, as well as demonstrating one of the advantages of selecting MOTS acquisitions.25

30. Figure 8 provides analysis of projects either completed, or removed from the MPR review, and shows that a focus on MOTS acquisitions has assisted in reducing schedule slippage. Figure 8 was requested by the JCPAA in May 2014.26

31. Longitudinal analysis indicates that while the reasons for schedule slippage vary, it primarily reflects the underestimation of both the scope and complexity of work, particularly for Australianised MOTS and developmental projects (see paragraphs 2.29 to 2.33).

Capability

32. The third principal component of project performance examined in this report is progress towards the delivery of capability required by government. While the assessment of expected capability delivery by Defence is outside the scope of the Auditor-General’s formal review conclusion, it is included in the analysis to provide an overall perspective of the three principal components of project performance.

33. The Defence PDSSs report that 23 projects in this year’s report will deliver all of their key capability requirements. Defence’s assessment indicates that some elements of the capability required may be ‘under threat’, but the risk is assessed as ‘manageable’. The three project offices experiencing challenges with expected capability delivery (2016–17: three) are Joint Strike Fighter, MRH90 Helicopters, and LHD Landing Craft. No project offices report that they are currently unable to deliver all of the required capability by FOC.

34. Defence’s presentation of capability delivery performance in the PDSSs is a forecast and therefore has an element of uncertainty. In 2015–16, the ANAO developed an additional measure of the status of current capability delivery progress to assist the Parliament — Capability Delivery Progress — which is a tally of the capability delivered as at 30 June 2018, as reported by Defence. Table 6 below provides a worked example of the ANAO’s methodology, utilising the performance information provided in the relevant PDSS.

Table 6: Capability Delivery Progress assessment — Additional MRTT (multi-role tanker transport)

Capability elements

as per Section 4.2 of the PDSS

No. of elements approved

No. of elements delivered at 30 June 2018

Comments

Delivery of first aircraft, and delivery of initial spares and support equipment (IMR)

2

2

First aircraft and initial spares delivery completed.

Delivery of second aircraft, delivery of remaining spares and support equipment, and delivery of Aircraft Stores Replenishment Vehicle (FMR)

3

0

Second aircraft, remaining spares and support equipment, and Aircraft Stores Replenishment Vehicle are yet to be delivered.

Total (number)

5

2

Total (%)

100

40

    

Source: PDSSs in published Major Projects Reports and ANAO analysis.

35. Table 7 below, summarises expected capability delivery as at 30 June 2018 — as reported by Defence and using the ANAO’s Capability Delivery Progress measure.

Table 7: Capability delivery

Expected Capability
(Defence Reporting)

2015–16 MPR (%)

2016–17 MPR (%)

2017–18 MPR (%)

Capability Delivery Progress
(ANAO Analysis)

2017–18 MPR (%)

2017–18 MPR (%) Adjusted 2

High Confidence (Green)

99

98

99

Delivered

72

61

Under Threat, considered manageable (Amber)

1

2

1

Not yet delivered

28

39

Unlikely (Red)

1

1

0

Not delivered at FOC

0

0

Total

100

100

100

Total

100

100

       

Note 1: Defence advised that in these years Joint Strike Fighter would not deliver one element of capability at FOC, of a total of 79 elements required for the project (which equated to approximately one per cent). However, across all the Major Projects this percentage rounds to zero.

Note 2: In prior years, ANAO adjusted for projects that disproportionately weighted the calculation of Capability Delivery Progress (those projects with large numbers of deliverables) by excluding them from the analysis. In 2017–18, the ANAO has used a different adjustment method. While the left-hand column reports the total percentage of elements delivered across all 26 Major Projects, the right-hand adjusted column reports the average percentage of elements delivered per project. This adjustment results in an analysis where all projects have equal weight and the percentage is not affected by the numbers of deliverables per project.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

36. In addition to reporting on expected capability delivery, Defence has continued the practice of including declassified information on settlement actions for projects. Prior settlements for projects within this report related to MRH90 Helicopters, LHD Ships and Maritime Comms.

1. The Major Projects Review

1.1 This chapter provides an overview of the review’s scope and approach, as implemented by the Australian National Audit Office (ANAO), for the review of the 26 Project Data Summary Sheets (PDSSs) prepared by the Department of Defence (Defence). This chapter also provides the results of the Major Projects Report (MPR) review.

Review scope and approach

1.2 In 2012 the Joint Committee of Public Accounts and Audit (JCPAA) identified the review of the PDSSs as a priority assurance review, under subsection 19A(5) of the Auditor-General Act 1997 (the Act). This provided the ANAO with full access to the information gathering powers under the Act. The ANAO’s review of the individual project PDSSs, which are reproduced in Part 3 of this report, was conducted in accordance with the auditing standards set by the Auditor-General under section 24 of the Auditor-General Act 1997 through its incorporation of the Australian Standard on Assurance Engagements (ASAE) 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Australian Auditing and Assurance Standards Board.

1.3 The following forecast information is excluded from the scope of the ANAO’s review: capability delivery, risks and issues, and forecast dates. These exclusions are due to the lack of Defence systems from which to provide complete and accurate evidence27, in a sufficiently timely manner to complete the review. Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.

1.4 The ANAO’s work is appropriate for the purpose of providing an Independent Assurance Report in accordance with the above auditing standard. However, the review of individual PDSSs is not as extensive as individual performance and financial statement audits conducted by the ANAO, in terms of the nature and scope of issues covered, and the extent to which evidence is required by the ANAO. Consequently, the level of assurance provided by this review, in relation to the 26 major Defence equipment acquisition projects (Major Projects), is less than that provided by the ANAO’s program of audits.

1.5 Separately, the ANAO undertakes analysis of key elements of the PDSSs and examines systemic issues and provides longitudinal analysis for the 26 projects reviewed.

1.6 The review was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $2.1 million.

Review methodology

1.7 The ANAO’s review of the information presented in the individual PDSSs included:

  • examination and assessment of the governance and oversight in place to ensure appropriate project management;
  • an assessment of the systems and controls that support project financial management, risk management, and project status reporting, within Defence;
  • an examination of each PDSS and the documents and information relevant to them;
  • a review of relevant processes and procedures used by Defence in the preparation of the PDSSs;
  • interviews with persons responsible for the preparation of the PDSSs and management of the projects;
  • analysis of project information, for example, cost and schedule variances;
  • taking account of industry contractor comments provided on draft PDSS information;
  • assessing the assurance by Defence managers attesting to the accuracy and completeness of the PDSSs;
  • examination of the representations by the Chief Finance Officer supporting the project financial assurance and contingency statements;
  • examination of confirmations, provided by the Capability Managers, relating to each project’s progress toward Initial Materiel Release (IMR), Final Materiel Release (FMR), Initial Operational Capability (IOC) and Final Operational Capability (FOC); and
  • examination of the ‘Statement by the Secretary of Defence’, including significant events occurring post 30 June, and management representations by the Secretary of Defence.

1.8 The ANAO’s review of PDSSs also focused on project management and reporting arrangements contributing to the overall governance of the Major Projects. The ANAO considered:

  • resolution of matters described in the Auditor-General’s prior year (2016–17) qualified Independent Assurance Report, relating to the ARH Tiger Helicopter PDSS28;
  • developments in acquisition governance (See paragraphs 1.10 to 1.25, below);
  • the financial framework, particularly as it applies to the project financial assurance and contingency statements, and Defence’s advice that project financial reporting during 2017–18 would be prepared on the same basis as project approvals and expenditure represented in the Portfolio Budget Statements and the Defence Annual Report (i.e. on a cash basis) (Section 2 of the PDSSs);
  • schedule management and test and evaluation processes (Section 3 of the PDSSs);
  • capability assessments, including Defence statements of the likelihood of delivering key capabilities, particularly where caveats are placed on the Capability Manager’s declaration of significant milestones (Section 4 of the PDSSs);
  • the ongoing reform process for the Enterprise Risk Management Framework, and the completeness and accuracy of major risk and issue data (Section 5 of the PDSSs);
  • the project maturity framework along with its related reporting and the systems in place to support the consistent and accurate application and the provision of this data (Section 6 of the PDSSs); and
  • the impact of acquisition issues on sustainment to ensure the PDSS is a complete and accurate representation of the acquisition project.

1.9 This review informed the ANAO’s understanding of the systems and processes supporting the PDSSs for the 2017–18 review period. It also highlighted issues in those systems and processes that warrant attention.

Acquisition governance

1.10 Consistent with previous years, the context of acquisition governance processes are covered in the ANAO’s review and informs the review planning process. While some of these processes are now established, others continue to mature or require further development to achieve their intended impact.

Defence Independent Assurance Reviews

1.11 The Defence Independent Assurance Review process provides the Defence Senior Executive with assurance that projects and products will deliver approved objectives and are prepared to progress to the next stage of activity. Reviews allow early identification of problem projects and products, facilitating their timely recovery.29,30

1.12 Formerly called Gate Reviews, Independent Assurance Reviews are intended to be conducted at key acquisition and sustainment ‘gates’ in the Capability Life Cycle.31

1.13 Seventeen of the 26 projects included in this report had an Independent Assurance Review conducted during 2017–1832, which formed key corroborative evidence for the ANAO’s review.

Projects of Concern

1.14 The Projects of Concern process is intended to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. The process has continued to play a role across the portfolio of MPR projects.33,34 As at 30 June 2018, one MPR project, MRH90 Helicopters was a continuing project of concern. The project was placed on the list in November 2011 due to contractor performance relating to significant technical issues preventing the achievement of milestones on schedule.35

1.15 In August 2017, the Ministers for Defence and Defence Industry announced36 that the Civil Military Air Traffic Management System (CMATS) project was being placed on the list due to substantial challenges getting into contract.37,38 The Ministers for Defence and Defence Industry subsequently announced in February 201839 that the project would be removed from the Projects of Concern list due to having achieved this milestone.40

Quarterly Performance Report

1.16 The Quarterly Performance Report (QPR) aims to provide senior stakeholders within government and Defence with a clear and timely understanding of emerging risks and issues in the delivery of capability to the Australian Defence Force’s end-users.41 Defence has advised that the report is provided to the Minister for Defence and the Minister for Defence Industry on a quarterly basis.42

1.17 In 2017–18, further to the MRH90 Helicopters MPR project of concern noted above, the June 2018 QPR also identified four MPR projects as Projects of Interest43:

  • Joint Strike Fighter, noting risks related to affordability, IOC and FOC deliverables, with consideration being given to de-scoping the project to address these risks;
  • Civil Military Air Traffic Management System (CMATS), noting risks to schedule due to execution of design milestones and poor scope definition, planning and dedicated resources attributed to the Four Alternate Tower Solution;
  • LHD Ships due to the late delivery of ships, a large number of outstanding requirements, defects and deficiencies, and an immature support system; and
  • UHF SATCOM, due to issues with the modification of Commercial Off-The-Shelf software (an element of the project now considered developmental) and delays in the security accreditation process.

1.18 The ongoing issues highlighted above for Joint Strike Fighter, LHD Ships, CMATS, and UHF SATCOM align with the results of the ANAO’s review. Delays to progress have impacted the delivery schedule of UHF SATCOM during 2017-1844 (see Table 4).

Joint Project Directives and Materiel Acquisition Agreements

1.19 Joint Project Directives (JPDs) state the terms of government approval and are used to inform internal documentation such as Materiel Acquisition Agreements (MAAs) between Capability Acquisition and Sustainment Group (CASG) and the Service Chiefs.45,46

1.20 In some cases JPDs have been finalised after the MAAs they are intended to inform and, as a result, care is required to ensure that JPDs properly reflect the relevant government decision, and that MAAs are appropriately aligned with the relevant JPD. For all three new projects entering the 2017–18 MPR, the projects had their JPD signed prior to the MAA which demonstrates improvement in this regard.

1.21 In 2017–18, 16 of the 17 MPR projects approved from 1 March 2010, have completed a JPD.47 However, the ANAO requires access to original approval documents to validate the requirements of projects. At this time, validation based on internal Defence documentation is not always possible.

1.22 The ANAO will continue to take JPDs into account in its review program in future years. The extent to which they can be relied upon will be dependent on the completeness and accuracy of JPDs, in relation to recording the detail of government approvals.

1.23 Product Delivery Agreements (PDAs) were being developed to replace the existing MAAs and Materiel Sustainment Agreements (MSAs)48, however Defence has since advised that this initiative has not progressed.

Business systems

1.24 Defence continues to review its business systems with the aim of consolidating processes and systems in order to provide a more manageable system environment.49 During 2018–19, and at the time of this report, the Monthly Reporting System (MRS), which provides much of the data for the PDSSs, remains in place although replacement of this system is still under consideration. As reported to the JCPAA on 31 March 2017, Defence stated that there was a ‘need to get a single unified system of accountability and reporting inside the organisation’.

1.25 In October 2018, Defence advised that it had concluded its trial of the Project Performance Review in November 2017. The PPR is a spreadsheet with project performance information intended to be used by project managers to inform discussions with Project Directors and Branch Heads. In January 2018 Defence initiated a plan to implement the PPRs across CASG. Defence has advised that seven MPR projects are currently preparing a PPR in Microsoft Excel form.50 Defence planned to then implement a streamlined ICT system to 31 projects commencing in November 2018. Defence has advised that it now expects this implementation to commence in December 2018, however it is not clear whether this will take the form of spreadsheets or a bespoke ICT system.

Results of the review

1.26 The following sections outline the results of the ANAO’s review, which inform the overall conclusion in the Independent Assurance Report by the Auditor-General for 2017–18.

Financial framework

1.27 The project financial assurance statements were introduced in the 2011–12 Major Projects Report and have been included within the scope of the Independent Assurance Report by the Auditor-General since 2014–15. The contingency statements were introduced for the first time in the 2013–14 report and these describe the use of contingency funding to mitigate project risks. Together, they are aimed at providing greater transparency over projects’ financial status.

1.28 A project’s total approved budget comprises:

  • the allocated budget, which covers the project’s approved activities, as indicated in the MAA; and
  • the contingency budget, which is set aside for the eventuality of risks occurring and includes unforeseen work that arises within the delivery of the planned scope of work.51

1.29 In 2017–18, the ANAO reviewed the financial framework as it applied to managing project budgets and expenditure, including: contingency, project financial assurance, the reporting environment, and reporting cost variations and personnel costs.

Project financial assurance statement

1.30 The project financial assurance statement was added to the PDSSs to enhance transparency by providing readers with information on each project’s financial position (in relation to delivering project capability) and whether there is ‘sufficient remaining budget for the project to be completed’.52 Defence advised on 27 October 2017 that the administrative policy supporting this statement was ‘no longer current as it is an artefact of the previous DMO agency’. Defence then advised in January 2018 that there is no Defence policy in place supporting the current administrative practice, but that the previously used policy would represent ‘good practice’.

1.31 In 2017–18 the CMATS project was granted a significant Real Cost Increase of $240.7 million by government in February 201853 which has enabled the project to state that ‘there is sufficient budget remaining for the project to complete against the agreed scope’, which had not been possible in the previous MPR.54

1.32 Unlike in previous years, Defence advised in January 2018 that it would no longer subject a sample of project financial assurance statements to a third-party agreed-upon procedures engagement to support the Chief Finance Officer in determining the appropriateness of the statements. The ANAO agreed with this approach which was endorsed by the JCPAA in the 2017–18 MPR Guidelines.

1.33 In conclusion, for the 2017–18 Major Projects Report, the Chief Finance Officer’s representation letter to the Secretary on the project financial assurance statements was unqualified. The project financial assurance statement is restricted to the current financial contractual obligations of Defence for these projects, including the result of settlement actions and the receipt of any liquidated damages, and current known risks and estimated future expenditure as at 30 June 2018.

Contingency statements and contingency management

1.34 The purpose of the project contingency budget is ‘to provide adequate budget to cover the inherent risk of the in-scope work of the project’.55 Defence policy requires project offices to maintain a contingency budget log to identify and track components of the contingency budget.56

1.35 PDSSs are required to include a statement regarding the application of contingency funds during the year, if applicable, as well as disclosing the risks mitigated by the application of those contingency funds. Defence’s Project Risk Management Manual (PRMM version 2.4, page 110) requires that contingency be applied for identified risk mitigation activities which have been assessed as being cost effective and representing value for money.

1.36 Contingency provisions for projects are not programmed into a project’s cash budget. As such, projects are encouraged to rely on cash budget management and savings to mitigate risks. Where this is not sufficient, projects can submit a request to Defence Finance Group to access contingency. If this cannot be managed within the Approved Major Capital Investment Program cash budget, consideration by the Defence Investment Committee is sought on how best to manage the call within the overall Integrated Investment Program.

1.37 A Defence internal audit, finalised in August 2018, concluded that Defence has been able to fund contingency calls through internal project or program management decisions and project slippage.57 The advent of large scale, high cost and high risk projects has created an environment whereby Defence Senior Executives consulted by the internal audit team expressed concern that it may become more difficult for Defence to redistribute cash funding to manage contingency events without compromising either the timeliness or level of capability delivered.58

1.38 The ANZAC ASMD 2B project PDSS59 notes that access to contingency funding to remediate unplanned obsolescence issues was denied during Defence budget processes.60

1.39 The four project offices which had contingency funds applied in 2017–18 were MRH90 Helicopters (supportability and performance risks), Battle Comm. Sys. (Land) 2B (Interface Control Integration), UHF SATCOM (software review and system security) and BMS (Risk Reduction Activities related relating to the M1A1 Tank Weapons Integrated Battle Management System).

1.40 The ANAO’s examination of the contingency statements as at 30 June 2018 also highlighted that:

  • the clarity of the relationship between contingency application and identified risks continues to be an issue. Of the 25 project offices that have a formal contingency allocation61, seven projects (Joint Strike Fighter, Overlander Medium/Heavy, Repl Replenishment Ships, Battle Comm. Sys. (Land) 2B, CMATS, Pacific Patrol Boat Repl and BMS) did not explicitly align their contingency log with their risk log, by including risk identification numbers as required by PRMM version 2.4;
  • the method for allocating contingency varied, with 22 project offices using the ‘expected costs’ of the risk treatment (as required by PRMM version 2.4), with Pacific Patrol Boat Repl and Repl Replenishment Ships having not yet allocated contingency against risks. The Overlander Medium/Heavy project used the proportionate allocation of the likelihood of the risk eventuating (the method outlined in PRMM version 2.2); and Collins R&S does not have a formal contingency allocation;
  • there were 16 project offices that did not meet all the requirements of PRMM version 2.4 in terms of keeping a record of review of contingency logs, however, the ANAO observed that the information required could be located in other documents.

1.41 Non-compliance with PRMM version 2.4 has resulted in inconsistent approaches taken to the management of contingency, with some projects advising that they will not remedy these non-compliances until the outcomes of the risk management reform within CASG are known (see paragraph 1.53).

Reporting environment

1.42 On April 4 2018, following a submission to the JCPAA hearing held on 23 March 2018, Defence advised project offices that project financial reporting for 2017–18 PDSSs would be prepared on the same basis as project approvals and expenditure represented in the Portfolio Budget Statements and the Defence Annual Report (i.e. on a cash basis). Therefore actual expenditure in the PDSSs may not be consistent with that reported in previous MPRs which had been prepared on an accrual basis. ANAO analysis of the overall variance showed that the difference was approximately 1.5 per cent between accrual and cash expenditure in the 2016–17 MPR.62

1.43 Defence obtains cash expenditure data using a management reporting tool called BORIS. In previous MPRs, accrual expenditure data was extracted from the Financial Management Information System known as ROMAN. Given the change in the extraction method, the ANAO requested evidence from Defence to support that the outputs of the BORIS tool were complete and accurate at the project level.

1.44 Defence was unable to provide sufficient evidence to support this position at the project level, so the ANAO requested that Defence conduct a reconciliation of all cash expenditure data from BORIS to the ROMAN Financial Management Information System which holds the transaction data. This activity concluded in early November 2018 and enabled assurance over the cash expenditure to be obtained by the ANAO.

Reporting cost variations since Second Pass Approval and personnel costs

1.45 In May 2018, the JCPAA wrote to the Auditor-General to request that the ANAO report back to it on how Defence Major Projects cost variations and the cost of retaining project staff might be reported annually in future MPRs.63 The JCPAA further asked the Auditor-General to consider presenting any relevant available data in the 2017–18 MPR.

1.46 The JCPAA indicated that it would consider whether inclusion of such information adds value to the MPR, with a view to amending the associated guidelines for future MPRs if the information proved to be useful in increasing oversight of expenditure.

1.47 In consultation with Defence, the PDSSs in this report have been amended to include the project budget at Second Pass Approval64 in addition to the project’s current approved budget to show any variations between them. In September 2018, the JCPAA endorsed the 2018–19 MPR Guidelines which require that this information is provided in future MPRs.65

1.48 A table of all budget variations post initial Second Pass Approval for the projects is also provided at Table 8.

1.49 The reporting of the costs of project staff has proven to be more challenging. Defence advised the ANAO in November 2018 that its current IT systems do not provide a direct mapping of personnel to projects, with personnel often working on multiple projects and sustainment activities at any given time. Defence has advised the ANAO that while it is not yet in a position to provide the staff cost component of projects, it has begun to collect information on the numbers of staff (including Australian Defence Force and Australian Public Service, but not contractors) for projects.66 Further information on staff numbers will be reported in the 2018–19 MPR.

Enterprise Risk Management Framework

1.50 While major risks and issues data in the PDSSs remains excluded from the formal scope of the Auditor-General’s Independent Assurance Report67, material inconsistencies identified in relation to this information are required to be detailed in the report. The following information is included to provide an overall perspective of how risks and issues are managed within Defence and the selected Major Projects.

1.51 Risk management has been a focus of the MPR since its inception. The CASG risk management environment consists of multiple policies and varying implementation mechanisms and documentation. There are multiple group-level (i.e. CASG), sub-group (i.e. Divisional) and project-level risk management documents. The primary focus of the ANAO’s examination of risk management is at the project level, in order to provide assurance over the PDSSs.

1.52 At the Group level, Deputy Secretary CASG issued a directive in May 2017 establishing a CASG Risk Management Reform Program to implement a risk management model that is situated within Defence’s risk management framework. CASG is part way through the reform initially intended to be completed by June 2019, with Defence advising that completion is now expected to occur by the end of 2019, and then taking a number of annual cycles to reach maturity.68

1.53 The next stage of the reform will provide project specific guidance and tools to support risk management practices of projects. The ANAO has observed that some projects chose not to review risk and issues management procedures until this stage has completed, as noted at paragraph 1.41. The ANAO will continue to monitor the implementation of the reform as part of future reviews, but will not be able to consider including risks and issues in the scope of the MPR until the reform is sufficiently progressed.

1.54 In 2017–18, the ANAO again examined project offices’ risk and issue logs at the Group and Service level, which are predominantly created and maintained utilising spreadsheets and/or Predict! software.69 Overall, the issues with risk management that the ANAO observed related to:

  • variable compliance with corporate guidance, for example all projects had a Risk Management Plan, however; 11 out of 26 Major Projects did not validate the currency of the Risk Management Plan in line with PRMM version 2.470;
  • the visibility of risks and issues when a project is transitioning to sustainment;
  • for four projects (JSF, HATS, Collins RCS and Hw Torpedo), sustainment and acquisition risks are managed together, despite Defence risk management policy for acquisition and sustainment providing inconsistent guidance71;
  • one project (Repl Replenishment Ships) early-adopted draft guidance from Defence intended to be used to prompt discussion as part of the CASG risk reform, only to be advised by the Defence Enterprise Risk Management Branch that this was not compliant with current Defence acquisition risk management guidance;
  • the frequency with which risk and issue logs are reviewed to ensure risks and issues are appropriately managed in a timely manner, and accurately reported to senior management;
  • risk management logs and supporting documentation of variable quality, particularly where spreadsheets are being used72; and
  • lack of quality control resulting in inconsistent approached in the recording of issues within Predict!

1.55 The ANAO has previously observed that Defence’s use of spreadsheets as a primary form of record for risk management is a high risk approach. Spreadsheets lack formalised change/version control and reporting, thereby increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by whom, and what change was made. As a result, a significant amount of quality assurance is necessary to obtain confidence that spreadsheets are complete and accurate at 30 June, which is not an efficient approach. The ANAO’s review of CASG’s 26 project offices indicates that 13 utilise spreadsheets73 as their primary risk management tool, seven utilise Predict!74, one (LHD Ships) utilises both Microsoft Excel and Predict!, two (JSF and CMATS) utilise a bespoke SharePoint based tool, one (MH-60R Seahawk) utilises Microsoft Word and two (Collins RCS and Hw Torpedo) do not currently manage any risks given the delivery of all primary project elements. Defence has advised that a risk management system will not be mandated until the outcomes of the CASG risk reform are known (see paragraph 1.52).

1.56 The JCPAA made a recommendation in September 2018 for Defence to plan and report a methodology to the Committee which shows how acquisition projects can transition from the use of spreadsheet risk registers to tools with better version control.75

Project maturity framework

1.57 Project Maturity Scores have been a feature of the Major Projects Report since its inception in 2007–08. The DMO Project Management Manual 2012, defined a maturity score as:

The quantification, in a simple and communicable manner, of the relative maturity of capital investment projects as they progress through the capability development and acquisition life cycle.76

1.58 Maturity scores are a composite indicator, cumulatively constructed through the assessment and summation of seven different attributes. The attributes are: Schedule, Cost, Requirement, Technical Understanding, Technical Difficulty, Commercial, and Operations and Support, which are assessed on a scale of one to 10.77 Comparing the maturity score against its expected life cycle gate benchmark provides internal and external stakeholders with a useful indication of a project’s progress.

1.59 The ANAO has previously identified that the policy guidance underpinning the attribution of maturity scores would benefit from a review for internal consistency and the relationship to Defence’s contemporary business. For example, allocating approximately 50 per cent of the maturity score at Second Pass Approval, regardless of acquisition type, is often inconsistent with the proportion of project budget expended, and the remaining work required to deliver the project. Further, the existing project maturity score model does not always effectively reflect a project’s progress during the often protracted build phase, particularly for developmental projects. During this phase it can be expected that maximum expenditure will occur, and that many risks will be realised, some of which will only emerge as test and evaluation activities are pursued through to acceptance into operational service.

1.60 In May 2016, the JCPAA recommended ‘that the Department of Defence work with the Australian National Audit Office to review and revise Defence’s policy regarding Project Maturity Scores in time for the new approach to be implemented in the next Major Projects Report.’78 Again in October 2017, the JCPAA recommended ‘that the Department of Defence commence discussions with the Australian National Audit Office on updating Project Maturity Scores.’79 At the JCPAA hearing held on 23 March 2018, Defence undertook to update the framework by mid-2018 with a two-stage process: first to remediate inconsistencies in the policy and accommodate Interim Capability Life Cycle terminology; then to undertake a more substantial amendment of the policy.80 Defence advised the ANAO in September 2018 that the maturity score process is now being re-considered within the CASG risk reform context.

Caveats

1.61 In 2017–18, the ANAO noted a reduced trend of Major Projects which have achieved significant milestones with caveats.81 The ANAO also notes advice from Defence that it discourages Independent Assurance Reviews recommending caveats at FOC. Growler is the only MPR project which has achieved a major milestone with caveats, related to training requirements for IMR in 2017, which have since been resolved.82

1.62 The ANAO will continue to monitor the declaration and resolution of caveats in future reviews, including those related to projects which have been removed from the MPR with outstanding caveats which are required to be reported by Defence in the Statement by the Secretary of Defence until their final status is accepted by the Capability Manager.83 In 2017–18, the ARH Tiger Helicopters project, which has exited the MPR, reported the closure of remaining caveats.84

2. Analysis of Projects’ Performance

2.1 Performance information is important in the management and delivery of major Defence equipment acquisition projects (Major Projects). It informs decisions about the allocation of resources, supports advice to government, and enables stakeholders to assess project progress.

2.2 Project performance has been the subject of many of the reviews of the Department of Defence (Defence), and a consistent area of focus of the Joint Committee of Public Accounts and Audit (JCPAA) since the first Major Projects Report (MPR). This chapter progresses previous Australian National Audit Office (ANAO) analysis over project performance.

Project performance analysis by the ANAO

2.3 The ANAO utilises three key performance indicators to analyse the major dimensions of projects’ progress and performance. These indicators are the:

  • percentage of budget expended (Budget Expended) — which measures the total expenditure as a percentage of the total current budget;
  • percentage of time elapsed (Time Elapsed) — which measures the percentage of time elapsed from original approval to the forecast Final Operational Capability (FOC)85; and
  • percentage of key materiel capabilities delivered (Capability Delivery Progress) — which measures the total capability elements delivered as a percentage of the total capability elements across all Major Projects.

2.4 The ANAO has previously utilised Defence’s prediction of expected final capability, as reported in Section 4.1 of each Project Data Summary Sheet (PDSS). In 2015–16, the ANAO derived an indicator for ‘Capability Delivery Progress’, which aims to show the current capability delivered, in terms of capability elements included within the agreed Materiel Acquisition Agreements (MAAs). These performance indicators are measured in percentage terms, to enable comparisons between projects of differing scope, and to provide a view across the selected projects of progress and performance.

2.5 The following sections of this chapter provide analysis relating to the three principal components of project performance. This includes in-year information, longitudinal analysis and the results of project progress for the year-ended 30 June 2018. The first piece of analysis, in Figure 2 below, sets out each project’s Budget Expended and Time Elapsed.86

Figure 2: Budget Expended and Time Elapsed

Budget Expended and Time Elapsed

Note: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Source: ANAO analysis of the 2017–18 PDSSs.

2.6 Figure 2 shows that for most projects (22 of 26), Budget Expended is broadly in line with, or lagging, Time Elapsed. This relationship is generally expected in an acquisition environment predominantly based on milestone payments. However, due to the varying complexity, stages and acquisition approaches across the portfolio of projects, further analysis of these simple performance measures is required to provide a better understanding of key variances.

2.7 Where Budget Expended is significantly lagging Time Elapsed the project schedule may be at risk, i.e. expenditure lags may indicate delays in milestone achievement. However this is not the case for the four projects where the Budget Expended is over 20 per cent less than the Time Elapsed in 2017–18, as detailed below:

  • Joint Strike Fighter (Budget Expended 17 per cent, Time Elapsed 62 per cent) — a large scope increase ($10.5 billion) for the purchase of additional aircraft was approved in April 2014, with the project now beginning to enter into main production contracts, as aircraft development continues;
  • Battlefield Airlifter (Budget Expended 55 per cent, Time Elapsed 80 per cent) — the project is yet to enter contracts relating to the acquisition of training devices. However, all ten aircraft have been delivered;
  • Battle Comm. Sys. (Land) 2B (Budget Expended 36 per cent, Time Elapsed 58 per cent) — the project is still in design phases for some elements, with the project’s payment schedule structured so that the majority of payment milestones fall towards the end of the project’s life; and
  • LHD Landing Craft (Budget Expended 74 per cent, Time Elapsed 94 per cent) — the variance reflects contingency and unallocated funds remaining as the project approaches closure.

2.8 Where Budget Expended leads Time Elapsed the project budget may be at risk, i.e. expenditure increases may indicate real cost increases. However, for the three projects where Budget Expended leads Time Elapsed by 10 per cent or more, the actual reasons are related either to early procurement of major equipment due to production timing, or schedule delays caused through platform availability, as detailed below:

  • P-8A Poseidon (Budget Expended 66 per cent, Time Elapsed 53 per cent) — the majority of project expenditure is aligned with aircraft production, with seven out of 12 aircraft already delivered and the final aircraft scheduled for delivery in 2019–20, with FOC not scheduled until May 2022;
  • Growler (Budget Expended 67 per cent, Time Elapsed 56 per cent) — expenditure reflects aircraft production costs (which represent a large proportion of project costs) having occurred before a large decrease in annual expenditure over the following years as work continues on the Mobile Threat Training Emitter System. All aircraft have now been delivered to Defence. The variance is also exacerbated by the length of time between Initial Operational Capability (IOC) (July 2018) and FOC (June 2022) with most of the major equipment being delivered by 2018; and
  • Collins R&S (Budget Expended 91 per cent, Time Elapsed 78 per cent) — most of the materiel has been acquired and expenditure undertaken. In addition, originally planned installation dates have been extended based on submarine availability, reducing the proportion of time elapsed. Furthermore, in 2017–18, the project schedule was extended due to additional scope transferred from other projects, but additional budget to fund this scope was not transferred during the financial year, reducing Time Elapsed without reducing Budget Expended.

2.9 In each case, the performance information highlights projects requiring further attention. This is to ensure that surplus funds are returned to the Defence budget for re-allocation in a timely manner, the timing of key deliverables remains in focus, or planning focuses on bringing together all elements in a timely manner, as equipment is delivered.

Cost performance analysis

Budget Expended and Project Maturity

2.10 Figure 3, below, sets out each project’s Budget Expended against Project Maturity87 and shows that Budget Expended lags Project Maturity for the majority of projects (19 of 26). This relationship is expected for two reasons:

  • in an acquisition environment predominantly based on milestone payments, projects will typically develop confidence in delivering their scope through design reviews, testing and demonstration, ahead of formal acceptance of milestone achievement or equipment deliveries (and expenditure of budget); and
  • more generally, Budget Expended will often lag Project Maturity as the result of Defence’s project maturity framework attributing approximately 50 per cent of total Project Maturity at Second Pass Approval (the main investment decision by government)88 prior to any significant expenditure of budget.

2.11 In both cases, the Budget Expended is expected to catch up to Project Maturity over the course of the project’s life, with projects approaching closure expected to show Budget Expended and Project Maturity broadly in line with each other.

2.12 Budget Expended lags Project Maturity with a variance of 20 per cent or more in 13 projects. As expected, the majority of these projects are at relatively early stages and have expended minimal budget while progressing through design and testing phases, or are waiting on significant amounts of equipment to be delivered. The exceptions to this are projects that have delivered the majority of their major equipment, leading to an advanced maturity score, while the budget expended is lagging as items such as training equipment or weapons are yet to be delivered and paid for. Projects fitting this pattern are MH-60R Seahawk (all helicopters delivered while some weapons and training devices are outstanding), Battlefield Airlifter (all aircraft delivered while some training devices are outstanding), and HATS (all helicopters delivered while some training devices are outstanding). Additionally, LHD Landing Craft’s Budget Expended lags Project Maturity as this project has delivered all 12 vessels and other scope without requiring the full budget to be expended.

2.13 Where Budget Expended leads Project Maturity by a significant amount, this may indicate that the project is behind in development or achievement of its scope, or that the required scope is not affordable. There are no instances where Budget Expended leads Project Maturity by 20 per cent or more. The largest variance is for UHF SATCOM, where Budget Expended leads Project Maturity by 17 per cent. The project’s maturity score has been affected by delays in software development, while the majority of budget has been expended and the project is funding further development with contingency.

Figure 3: Budget Expended and Project Maturity

Budget Expended and Project Maturity

Source: ANAO analysis of the 2017–18 PDSSs.

Second Pass Approval and 30 June 2018 approved budget

2.14 Figure 4, below, compares each project’s approved budget at initial Second Pass Approval and its approved budget at 30 June 2018.

2.15 The total budget for the 26 projects at 30 June 2018 was $59.4 billion, a net increase of $23.0 billion, when compared to the approved budget at initial Second Pass Approval of $36.5 billion.89

2.16 Figure 4 indicates all relative budget variations from initial Second Pass Approval. Six projects have variations of $500 million or more. The list below describes the components of these variations:

  • Joint Strike Fighter — increase of $12.8 billion, comprising $10.5 billion for 58 additional aircraft in 2013–14, $1.9 billion for exchange rate variation and $0.4 billion for price indexation;
  • AWD Ships — increase of $1.9 billion, comprising $1.2 billion for a Real Cost Increase90 in July 2015 to complete the project, $1.2 billion for price indexation, offset by a $0.4 billion decrease for exchange rate variation and a $0.1 billion decrease for transfers to facilities projects in 2013–14;
  • P-8A Poseidon — increase of $1.6 billion, comprising $1.3 billion for four additional aircraft in 2015–16 and $0.3 billion for exchange rate variation;
  • MRH90 Helicopters — increase of $2.8 billion, comprising $2.6 billion for 34 additional aircraft in 2005–06 and other minor scope changes, and $0.7 billion for price indexation, offset by a $0.3 billion decrease due to scope transfers for facilities, and a $0.1 billion decrease for exchange rate variation;
  • Growler — increase of $0.8 billion, comprising $0.9 billion for exchange rate variation, $0.2 billion in 2014–15 for the Mobile Threat Training Emitter System and weapons, and $0.1 billion in 2016–17 for Advanced Mobile Threat Training Emitter System scope, offset in 2015–16 by a $0.2 billion decrease for transfers to facilities projects and $0.1 billion for the return to the Defence budget of surplus funds for reallocation, and in 2016–17 by a $0.1 billion decrease for the reduction of project contingency associated with aircraft production being returned to the Defence budget for re-allocation; and
  • Overlander Medium/Heavy — increase of $0.9 billion, comprising $0.7 billion project supplementation associated with easing cost pressures and $0.2 billion exchange rate variation.

Figure 4: Projects’ initial Second Pass Approval and 30 June 2018 approved budget ($m)

Projects’ initial Second Pass Approval and 30 June 2018 approved budget ($m)

Note 1: [] indicates that the budget for the project at 30 June 2018 is less than the original budgeted cost.

Note 2: On 22 May 2015, the Minister for Defence and the Minister for Finance announced there would be further delays to the delivery of the Air Warfare Destroyers and an additional $1.2 billion would be required to complete the project. The budget increase was incorporated into the approved project budget as at 30 June 2016.

Source: ANAO analysis of the 2017–18 PDSSs.

Table 8: Budget variation post initial Second Pass Approval by variation type as at 30 June 2018 and Performance Audits 1

Project

Initial Second Pass Approval Budget $m

Variation

Explanation of Variation

Year/s of Variation

Total Amount of Variation $m

Performance Audits

AIR 6000 Phase 2A/2B New Air Combat Capability 2

2751.6

(Stage 1)

Scope increase/Budgetary Adjustments/Transfer

58 additional aircraft (Stage 2 Second Pass Approval) offset by minor transfers

2013–14

2017–18

10,504.1

Auditor-General Report No.6 of 2012–13: Management of Australia’s Air Combat Capability – F-35A Joint Strike Fighter Acquisition

Auditor-General Report No.14 Joint Strike Fighter — introduction into service and sustainment planning

SEA 4000 Phase 3 Air Warfare Destroyer Build

7207.4

Real Cost Increase/Budget transfers

Real Cost Increase of $1.2b offset by minor transfers for facilities in 2014

2013–14 and 2015–16

1089.6

Auditor-General Report No.22 of 2013–14: Air Warfare Destroyer Program

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

AIR 7000 Phase 2B Maritime Patrol and Response Aircraft System

3577.7

Scope increase

Four additional aircraft

2015–16

1295.4

N/A

AIR 9000 Phase 2/4/6 Multi-Role Helicopter 2

957.2

(Phase 2)

Scope increase/Budget transfers

34 additional aircraft (Phase 4/6 Second Pass Approval), offset by minor transfers

2005–06

2270.7

Auditor-General Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions (paragraph 4.54)

Auditor-General Report No.52 of 2013–14: Multi-Role Helicopter Program

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

AIR 5349 Phase 3 EA-18G Growler Airborne Electronic Attack Capability

2641.4

Scope increase/Real Cost Decrease

Additional training devices offset by return of surplus funds and other minor transfers

2014–15, 2015–16 and 2016–17

(91.6)

N/A

AIR 9000 Phase 8 Future Naval Aviation Combat System Helicopter

3029.6

Budget transfer

Transfer to Defence Support and Reform Group

2013–14

(39.2)

N/A

LAND 121 Phase 3B Medium/Heavy Capability, Field Vehicles, Modules and Trailers 2

2549.2

Project supplementation and Scope increase/Budget transfers

Project supplementation ($684.2m) and additional vehicles, trailers and equipment ($28.0m) at Revised Second Pass Approval

2013–14

712.2

Auditor-General Report No.52 of 2014–15: Australian Defence Force’s Medium and Heavy Vehicle Fleet Replacement (LAND 121 Phase 3B)

JP 2048 Phase 4A/4B Amphibious Ships (LHD)

2958.3

Budget transfer

Transfer from Defence Science and Technology Group

2008–09

9.3

Auditor-General Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions

Auditor-General Report No.57 of 2010–11: Acceptance into Service of Navy Capability

LAND 121 Phase 4 Protected Mobility Vehicle – Light

1945.0

N/A

N/A

N/A

0.0

Auditor-General Report No. 6 of 2018–19: Army’s Protected Mobility Vehicle – Light

AIR 8000 Phase 2 Battlefield Airlift – Caribou Replacement

1156.5

N/A

N/A

N/A

0.0

Auditor-General Report No.3 of 2013–14: AIR 8000 Phase 2 – C-27J Spartan Battlefield Airlift Aircraft

SEA 1654 Phase 3

Maritime Operational Support Capability

1004.6

Budget Transfer

Transfer for training

2015–16

69.1

N/A

AIR 5341 Phase 3 Civil Military Air Traffic Management System (CMATS)

731.4

Real Cost Increase/ Budgetary Adjustment

Real Cost Increase offset by minor transfers

2017–18

240.7

Auditor-General Report No.46 of 2016–17: Conduct of the OneSKY Tender

Auditor-General Report No.1 of 2016–17: Procurement of the International Centre for Complex Project Management to Assist on the OneSKY Australia Program

Performance audit currently underway: OneSKY Contractual Arrangements

AIR 7403 Phase 3 Additional KC-30A Multi-role Tanker Transport

681.9

Scope increase/Budgetary adjustment

Additional capability (Government Transport and Communication) offset by minor transfers

2015–16

183.0

N/A

SEA 1448 Phase 2B ANZAC Anti-Ship Missile Defence

248.8

Budget transfers/Scope increase

Scope increases offset by minor transfers

2005–06 and 2011–12

363.4

Performance audit currently underway: Anzac class frigates - sustainment

 

SEA 1439 Phase 4A Collins Replacement Combat System

455.3

Budget transfers/Budgetary adjustment

Minor transfers

2002–03 and 2004–05

(1.7)

ANAO Report No.57 of 2010–11: Acceptance into Service of Navy Capability

ANAO Report No.23 of 2008–09: Management of the Collins-class Operations Sustainment

JP 2072 Phase 2A

Battlespace Communications Systems

436.4

Real Cost Decrease

Real Cost Decrease

2017–18

(25.6)

 

SEA 1429 Phase 2 Replacement Heavyweight Torpedo

238.1

Scope increase/ Budgetary adjustment/Budget transfers

Implementation of full scope and minor transfers

2002–03 and 2004–05

214.2

ANAO Report No.57 of 2010–11: Acceptance into Service of Navy Capability

 

JP 2008 Phase 5A Indian Ocean Region UHF SATCOM

460.9

Real Cost Decrease

Real Cost Decrease

2013–14

(18.0)

N/A

SEA 1439 Phase 3 Collins Class Submarine Reliability and Sustainability

72.0

Scope increase/ Budget transfers/ Budgetary adjustments

Implementation of full scope, offset by minor transfers

2000–01, 2001–02, 2002–03, 2004–05, 2005–06 and 2006–07

271.3

ANAO Report No.23 of 2008–09: Management of the Collins-class Operations Sustainment

SEA 1448 Phase 2A ANZAC Anti-Ship Missile Defence

449.0

Budget transfers/Budgetary adjustments

Transfer to Phase 2B and Defence Support Group

2004–05, 2005–06 and 2006–07

(159.9)

ANAO Report No.57 of 2010–11: Acceptance into Service of Navy Capability

Performance audit currently underway: Anzac class frigates - sustainment

 

LAND 75 Phase 4 Battle Management System

319.0

Real Cost Increase

Design effort from previous phases

2014–15

8.5

N/A

JP 2048 Phase 3 Amphibious Watercraft Replacement

235.7

Budget transfer

Correction to transfer price

2013–14

(7.7)

ANAO Report No.9 of 2015–16: Test and Evaluation of Major Defence Equipment Acquisitions (paragraph 4.68)

Total

34,107.3

 

 

 

16,887.8

 

       

Note 1: Some projects have multiple Second Pass Approvals, this table reports on variations since the first, i.e. initial, Second Pass Approval.

Note 2: Three projects have had multiple Second Pass Approvals. For the purposes of this table, the ANAO has used the earliest Second Pass Approval.

Note 3: Projects that have had no Real Variations to their budget, and have not appeared in any performance audits, do not appear in this table. They are: Battle Comm. Sys. (Land) 2B, Pacific Patrol Boat Repl, HATS, and Maritime Comms. For a definition of ‘Real Variations’ see page 408 in the 2017–18 MPR Guidelines in Part 4 of this report.

Budget performance

2.17 The following figures and tables illustrate the budget performance for the 26 selected projects by way of:

  • in-year budget variations by project (see Table 9, below); and
  • expenditure forecasting performance against actual expenditure for 2017–18 (see Figure 5).
In-year budget variance analysis

2.18 Table 9, below, sets out the in-year budget variations for each project. Overall, the approved budget for the projects as at 30 June 2018 decreased by $263.5 million, or 0.5 per cent, compared to their approved budget as at 30 June 2017. This was driven by net real increases of $93.3 million, offset by exchange rate variation decreases of $356.6 million.

2.19 Real Variations91 primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability and budgetary adjustments such as administrative savings decisions. In 2017–18, the four projects with more significant Real Variations were:

  • CMATS — variation of $240.7 million reflecting approval for a Real Cost Increase to deliver the approved scope92;
  • MRH90 Helicopters — variation of -$87.4 million reflecting an administrative correction of errors in the project’s budget;
  • Growler — variation of -$27.0 million reflecting the project’s contribution to a rebalancing of the Defence Integrated Investment Program; and
  • Battle Comm. Sys. (Land) 2A — variation of -$25.6 million reflecting funds transferred to project LAND 200 Tranche 2 to offset in-year shortfalls against that project’s capital provision.

2.20 Exchange rate variations result from projects’ exposure to foreign currencies and movements in foreign exchange rates against the Australian dollar.93 Budget adjustments aim to maintain the relative buying power of the project budget. Movements in the US dollar and the Euro are the main influences. Projects with larger movements in foreign exchange in 2017–18 included:

  • Joint Strike Fighter — movement of -$492.4 million, or 3.1 per cent decrease in budget; and
  • MRH90 Helicopters — movement of $124.7 million, or 3.3 per cent increase in budget.

Table 9: In-year (2017–18) budget variations by project

Project

Approved Budget 2016–17 $m

Approved Budget 2017–18 $m

In-year Exchange Variation $m

In-year Real Variation $m

Total Variance $m

Total Variance (per cent)

Joint Strike Fighter 1

16,004.9

15,504.0

(492.4)

(8.4)

(500.9)

(3.1)

AWD Ships

9090.1

9089.3

(0.8)

-

(0.8)

0.0

P-8A Poseidon 1

5262.5

5212.0

(51.4)

1.0

(50.5)

(1.0)

MRH90 Helicopters

3733.8

3771.1

124.7

(87.4)

37.3

1.0

Growler 1

3495.0

3430.4

(37.5)

(27.0)

(64.6)

(1.8)

MH-60R Seahawk 1

3462.5

3430.3

(32.1)

-

(32.2)

(0.9)

Overlander Medium/Heavy

3363.5

3428.9

65.4

-

65.4

1.9

LHD Ships

3091.9

3091.7

(0.2)

-

(0.2)

0.0

Hawkei 1

1951.1

1952.0

0.7

-

0.9

0.0

Battlefield Airlifter

1406.7

1433.3

26.6

-

26.6

1.9

Repl Replenishment Ships

-

1066.8

(6.9)

-

(6.9)

(0.6)

CMATS

730.7

974.2

2.8

240.7

243.5

33.3

Battle Comm. Sys. (Land) 2B

-

920.1

4.4

-

4.4

0.5

Additional MRTT 1

855.5

887.8

32.2

-

32.3

3.8

ANZAC ASMD 2B 1

678.6

678.7

-

-

0.1

0.0

Pacific Patrol Boat Repl

-

501.2

(3.3)

-

(3.3)

(0.7)

HATS

474.2

481.5

7.3

-

7.3

1.5

Collins RCS 1

450.4

450.5

-

-

0.1

0.0

Battle Comm. Sys. (Land) 2A

463.3

438.0

0.3

(25.6)

(25.3)

(5.5)

Maritime Comms

432.1

437.7

5.6

-

5.6

1.3

Hw Torpedo 1

428.0

427.6

(0.3)

-

(0.4)

(0.1)

UHF SATCOM 1

420.5

419.9

(0.5)

-

(0.6)

(0.1)

Collins R&S 1

411.7

411.6

-

-

(0.1)

0.0

ANZAC ASMD 2A 1

386.7

386.8

-

-

0.1

0.0

BMS

369.1

367.9

(1.2)

-

(1.2)

(0.3)

LHD Landing Craft 1

236.8

236.7

-

-

(0.1)

0.0

Total

57,199.6

59,430.0

(356.6)

93.3

(263.5)

(0.5)

       

Note 1: The Total Variance and components for this project do not add due to rounding differences.

Source: ANAO analysis of the 2016–17 and 2017–18 PDSSs.

In-year forecast and actual expenditure

2.21 Accurately forecasting and managing budget expenditure is an important element in the management of a portfolio of projects. Figure 5, below, sets out the expenditure forecasting performance of each project against actual expenditure in 2017–18.94 In total, actual expenditure for the 26 projects at 30 June 2018 was $4585.2 million. This is compared against an initial Portfolio Budget Statements (PBS) forecast expenditure of $5255.2 million, a mid-year Portfolio Additional Estimates Statements (PAES) forecast of $4793.4 million, and a final forecast of $4649.5 million (Final Plan, approved during May 2018). The main factors contributing to the variances were changes to delivery and payment schedules, and foreign exchange fluctuations.

2.22 Figure 5 highlights that notable in-year underspends occurred in the following projects:

  • Joint Strike Fighter (expenditure of $1069.9 million compared to $1148.2 million PBS, $1113.3 million PAES and $1128.1 million Final Plan estimates) — the variance is due to delays in invoicing of payments related to aircraft delivery, and timing and value variations against other contracts;
  • AWD Ships (expenditure of $466.2 million compared to $682.6 million PBS, $520.2 million PAES and $522.8 million Final Plan estimates) — the variance is due to reduced disbursements against the AEGIS Foreign Military Sales case, and various other contract and procurement delays; and
  • Overlander Medium/Heavy (expenditure of $659.7 million compared to $709.9 million PBS, $687.6 million PAES and $697.3 million Final Plan estimates) — the variance is due to a decision by Defence Finance Group to defer payments from 2017–18 to 2018–19 as a result of portfolio cash budget pressures.

2.23 Figure 5 also highlights that a notable in-year overspend occurred in the following project:

  • P-8A Poseidon (expenditure of $705.1 million compared to $852.5 million PBS, $704.3 million PAES and $546.0 million Final Plan estimates) — the variance is due to the acceleration of payments due to earlier aircraft delivery.

Figure 5: In-year (2017–18) projects’ forecast expenditure performance compared to actual expenditure ($m)

In-year (2017–18) projects’ forecast expenditure performance compared to actual expenditure ($m)

Sources: ANAO analysis of the 2017–18 PDSSs and Defence Portfolio Budget Statements.

Schedule performance analysis

2.24 Defence data continues to show that schedule performance is a key issue in delivering and sustaining equipment.95 Project schedule slippage can effectively introduce or exacerbate an existing capability gap, or require an extension to the planned withdrawal date for those platforms being replaced.96

Time Elapsed and Project Maturity

2.25 The proportion of lower risk MOTS acquisitions since 2005 has assisted in meeting schedule timelines across projects.97 Analysis of the available performance information highlights that the selection of MOTS projects assists in reducing risk during project acquisition, where Project Maturity is more advanced at Second Pass Approval than developmental projects. For example, CMATS is a developmental project that has experienced significant schedule slippage; its maturity score at Second Pass Approval was 31 points. In contrast, MH-60R Seahawk is a MOTS project that has not experienced any slippage to date; its maturity score at Second Pass Approval was 37 points.

2.26 Figure 6, below, sets out each project’s Time Elapsed against Project Maturity.98 Time Elapsed lags Project Maturity for 18 of 26 projects. Similar to the analysis of Budget Expended and Project Maturity, at paragraphs 2.10–2.13, this pattern is expected as projects will generally score 50 per cent of their Project Maturity at Second Pass Approval, when Time Elapsed is zero (for the purposes of the ANAO’s analysis in this report). The lag is most pronounced in MOTS and Australianised MOTS acquisitions, including P-8A Poseidon, MH-60R Seahawk, Repl Replenishment Ships, and Pacific Patrol Boat Repl. The exception is Hawkei, where the lag in Time Elapsed against Project Maturity reflects the project’s extensive schedule to FOC, required to deliver over 2000 vehicles and trailers.

2.27 For the 10 projects where Time Elapsed lags Project Maturity by 20 per cent or more, this generally reflects projects at relatively early stages of acquisition processes, including proceeding through design activities, or awaiting significant amounts of their major equipment to be constructed and delivered. There are three significant exceptions to this:

  • Growler, where the EA-18G aircraft have been accepted and transferred to Australia, but project scope was increased in April 2017 to include a phased array radar threat emitter which is used in training;
  • MH-60R Seahawk, where the majority of equipment has been delivered but the project needs to test and integrate a number of ADF Mission System Options and modify Navy vessels to operate with the helicopter; and
  • HATS, where all helicopters have been delivered, but some items of training equipment are outstanding, and time is required to transfer all equipment to operational service.

2.28 For the 7 projects where Time Elapsed leads Project Maturity, there were no instances where this difference was significant (20 per cent or more).

Figure 6: Time Elapsed and Project Maturity

Time Elapsed and Project Maturity

Note 1: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Source: ANAO analysis of the 2017–18 PDSSs.

Schedule slippage and acquisition type by approval date

2.29 Figure 7, below, illustrates the total schedule slippage99 since Second Pass Approval for the 26 selected projects. It also depicts the acquisition type and places projects in order of government approval. Figure 8 illustrates the total schedule slippage for the 18 projects that have exited the review.

2.30 Following implementation of the recommendations of the 2003 Kinnaird review100, in 2005 Defence began focusing on MOTS and Australianised MOTS acquisitions. Figures 7 and 8 show that the inclusion of MOTS acquisitions contributed, prima facie, to a reduction in schedule slippage in the Major Projects portfolio. MOTS projects currently in the MPR report an average of 24 months of slippage per project, while Australianised MOTS projects report an average of 41.2 months and Developmental projects report an average of 46.4 months. Decisions on whether to undertake developmental projects should be considered on a risk basis.101 In this context, the consideration of risk includes not just the project specific attributes related to procurement, but also any compromises to the capabilities that would have been acquired through a developmental acquisition program.

2.31 The 2015 First Principles Review identified technical risk as the major cause of post Second Pass Approval schedule slippage, and also observed that schedule slippage causes cost escalation.102 The challenge of gaining a full understanding of the complexities of developmental aspects of projects at Second Pass Approval is evident by the extent of slippage over time.

2.32 Figures 7 and 8 illustrate that older projects, which achieved Second Pass Approval prior to 2005, have experienced the most slippage. These projects tended to be more developmental (complex) in nature and typically experienced schedule slippage in the past, and have often continued to do so. This demonstrates an ongoing trend of slippage in historically late projects, which is more pronounced in older projects. This trend is also visible, but less prominent, in newer projects.

2.33 While it is not possible to predict the full extent of slippage a project will experience, this analysis has been provided to highlight changes since Kinnaird. Nine post Kinnaird and nine pre Kinnaird projects have exited the MPR. Total slippage of the nine post Kinnaird projects is 7.1 years. Total slippage of the nine pre Kinnaird projects is 51.2 years. Six of the nine post Kinnaird projects were MOTS acquisitions and all of the nine pre Kinnaird acquisitions were Australianised MOTS or Developmental.

Figure 7: Current Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Current Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Note 3: The following projects have had additional scope approved following Second Pass Approval, which has caused slippage: P-8A Poseidon (24 months), Additional MRTT (21 months), and Collins R&S (13 months). The PDSSs indicate that the additional scope for these projects explains 58 months of the slippage reported here.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Figure 8:Exited Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Exited Major Projects — Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: This does not include AIR 5376 Phase 3.2 Hornet Refurb, which exited in 2012, as this project did not introduce a new capability and did not have an FOC date.

Note 3: The slippage shown for Next Gen Satellite related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced which reduced this slippage.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Schedule performance

2.34 The figures and tables that follow illustrate:

  • the original and 30 June 2018 forecasts for achieving FOC;
  • in-year schedule changes to achieving FOC;
  • total schedule slippage across the Major Projects; and
  • total slippage according to a project’s Second Pass Approval date.
Original and 30 June 2018 Final Operational Capability forecasts

2.35 Figure 9, below, presents information on the selected projects’ original and 30 June 2018 forecasts for achieving FOC. The total schedule slippage for the 26 Major Projects to date is 801 months compared to the initial prediction when approved by government. This represents a 32 per cent increase on the approved schedule. Of the 26 projects in the 2017–18 report, 20 have experienced schedule slippage.

2.36 Total schedule slippage across the Major Projects was 801 months in 2017–18. This is 8 months higher than the figure of 793 months reported in the 2016–17 report. The difference is mainly due to significant in-year slippage in MRH90 Helicopters (continuing technical issues requiring remediation prior to FOC), Battle Comm. Sys. (Land) 2A (administrative issues requiring resolution prior to FOC), UHF SATCOM (further delays in software development), ANZAC ASMD 2A and 2B (minor technical issues requiring resolution prior to FOC), and Collins R&S (introduction of additional project scope).103 These projects, combined, added 87 months of the 104 months schedule slippage in 2017–18. Additionally, Pacific Patrol Boat Repl added two months of slippage to the total of 801 months; the slippage occurred in 2016–17 but the project was reported in the MPR for the first time in 2017–18. This slippage was offset by the exit of Bushmaster Vehicles, ARH Tiger Helicopters, Overlander Light and Additional Chinook, which reduced the accumulated slippage by 98 months.

2.37 The reasons for schedule slippage often require a deep understanding of project technical elements and a realistic assessment of the capacity of the private sector to deliver in the expected timeframe. A project office’s ability to gain access to the platform for upgrading can also result in schedule delay (for example, the two Collins submarine projects and Hw Torpedo).104

2.38 A closer examination of the reasons for schedule slippage demonstrates the importance of initial assessments of project complexity. A key factor is whether a project is MOTS, Australianised MOTS or developmental.105 One project, MRH90 Helicopters106, was originally misclassified as MOTS. The project was reclassified by Defence to Australianised MOTS (i.e. more developmental) subsequent to Second Pass Approval. This project has experienced extended schedule slippage.

2.39 Figure 9 further indicates that three projects (Joint Strike Fighter, Overlander Medium/Heavy, and HATS) are currently forecasting an FOC date ahead of that originally approved. However, Joint Strike Fighter and Overlander Medium/Heavy have previously forecast earlier dates than their 30 June 2018 forecasts, and have experienced slippage from those previous forecasts, partially offsetting their schedule recovery. Other projects with schedule recovery offset by slippage are AWD Ships, Growler, Additional MRTT, ANZAC ASMD 2A and 2B, Collins RCS, Hw Torpedo, Collins R&S, and LHD Landing Craft. In total, these projects have contributed 38 months of schedule recovery to the Major Projects; however, ANAO analysis (for example, in Table 2 and Figure 11) excludes this effect to portray the complete amount of slippage experienced by the Major Projects.

Figure 9: Projects’ original and 30 June 2018 FOC forecasts

Projects’ original and 30 June 2018 FOC forecasts

Note 1: [] indicates that the forecast FOC date for the project at 30 June 2018 is earlier than the original FOC date.

Note 2: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Source: ANAO analysis of the 2017–18 PDSSs.

In-year schedule performance

2.40 In 2017–18, there was schedule slippage of 104 months in the forecast achievement of FOC across the 26 Major Projects. In-year project performance, measured by slippage over the last 12 months, may not reflect the project trend. However, Figure 10 below, shows recovery of previously reported slippage for one project:

  • Additional MRTT — the project currently expects to achieve FOC two months ahead of the 2017 forecast schedule, in October 2019.

2.41 In-year schedule slippage occurred for the following ten projects107 (the explanation provided, drawn from the 2017–18 PDSSs, may also include the reasons for prior slippage):

  • P-8A Poseidon — the variance reflects minor rescheduling of the FOC milestone to accommodate the additional four aircraft;
  • MRH90 Helicopters — the project has been affected by continuing technical and supportability issues;
  • Growler — the variance reflects minor rescheduling of the FOC milestone;
  • ANZAC ASMD 2B — delays have been caused by remediation of navigation radar support deficiencies;
  • Battle Comm. Sys. (Land) 2A — administrative processes are still required to declare FOC;
  • Maritime Comms — the project’s schedule is dependent on the Royal Australian Navy’s schedule of ship dockings;
  • UHF SATCOM — the project has experienced further delay in software development;
  • Collins R&S — additional scope has been introduced by the amalgamation of two related Collins projects;
  • ANZAC ASMD 2A — FOC is dependent on implementation of the support equipment contract for the Infrared Search and Track capability; and
  • LHD Landing Craft — final operational test and evaluation trials had not occurred at 30 June 2018.108

Figure 10: In-year (2017–18) schedule changes to achieving FOC

In-year (2017–18) schedule changes to achieving FOC

Note: Defence’s PDSSs indicate that 14 of the 26 Major Projects Report projects did not record changes to their Final Operational Capability dates this year.

Source: ANAO analysis of the 2017–18 PDSSs.

Longitudinal schedule performance

2.42 Figure 11, below, shows the accumulated schedule slippage over time of the Major Projects included in the MPR reports from 2007–08 to 2017–18.109 Table 10 provides the details of the specific projects included in the analysis. The figure shows that 13.6 per cent (9.1 years or 109 months) of the total schedule slippage across the Major Projects covered in the 2017–18 report (66.8 years or 801 months) is made up of the slippage from the one remaining project (Collins RCS) reported in the 2007–08 Major Projects Report.

2.43 Further disaggregation according to a project’s Second Pass Approval date in Table 11 shows that 46 per cent (2016–17: 54 per cent) of the total schedule slippage across the 2017–18 Major Projects is made up of projects approved prior to July 2005.

Figure 11: Longitudinal schedule slippage across years for projects in the 2017–18 MPR (in years)

Longitudinal schedule slippage across years for projects in the 2017–18 MPR (in years)

Note: The total schedule slippage in 2017–18 across the 26 projects is 801 months. BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Table 10: Projects included in Figure 11 analysis by Major Projects Report

Project

2007–08

2008–09

2009–10

2010–11

2011–12

2012–13

2013–14

2014–15

2015–16

2016–17

2017–18

Joint Strike Fighter

 

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

AWD Ships

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

P-8A Poseidon

 

 

 

 

 

 

 

Yes

Yes

Yes

Yes

MRH90 Helicopters

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Growler

 

 

 

 

 

 

Yes

Yes

Yes

Yes

Yes

MH-60R Seahawk

 

 

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Overlander Medium/Heavy

 

 

 

 

 

 

Yes

Yes

Yes

Yes

Yes

LHD Ships

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Hawkei

 

 

 

 

 

 

 

 

 

Yes

Yes

Battlefield Airlifter

 

 

 

 

 

 

Yes

Yes

Yes

Yes

Yes

Repl Replenishment Ships

 

 

 

 

 

 

 

 

 

 

Yes

CMATS

 

 

 

 

 

 

 

 

 

Yes

Yes

Battle Comm. Sys. (Land) 2B

 

 

 

 

 

 

 

 

 

 

Yes

Additional MRTT

 

 

 

 

 

 

 

 

Yes

Yes

Yes

ANZAC ASMD 2B

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Pacific Patrol Boat Repl

 

 

 

 

 

 

 

 

 

 

Yes

HATS

 

 

 

 

 

 

 

 

Yes

Yes

Yes

Collins RCS

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Battle Comm. Sys. (Land) 2A

 

 

 

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Maritime Comms

 

 

 

 

 

 

 

Yes

Yes

Yes

Yes

Hw Torpedo

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

UHF SATCOM

 

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Collins R&S

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

ANZAC ASMD 2A

 

 

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

BMS

 

 

 

 

 

 

 

 

Yes

Yes

Yes

LHD Landing Craft

 

 

 

 

 

 

Yes

Yes

Yes

Yes

Yes

            

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Table 11: Project slippage by project approval

Project

No. of months between Approval and Original FOC date

No. of months between Approval and 30/6/18 FOC date

No. of months slippage between Original FOC and 30/6/18 FOC date

Projects Approved pre July 2005

Collins RCS

88

195

109 1

Hw Torpedo

148

209

63 1

Collins R&S

165

273

112 1

ANZAC ASMD 2A

97

177

82 1

Sub Total – Projects Approved pre July 2005

498

854

366

Percentage of Total – Projects Approved pre July 2005

20%

26%

46%

Projects Approved post July 2005

Joint Strike Fighter

169

167

2 1

AWD Ships

131

163

35 1

P-8A Poseidon

71

99

28

MRH90 Helicopters

119

208

89

Growler

111

111

1 1

MH-60R Seahawk

150

150

0

Overlander Medium/Heavy

125

119

5 1

LHD Ships

113

150

37

Hawkei

94

94

0

Battlefield Airlifter

68

92

24

Repl Replenishment Ships

73

73

0

CMATS

102

130

28

Battle Comm. Sys. (Land) 2B

65

65

0

Additional MRTT

33

52

21 1

ANZAC ASMD 2B

90

155

67 1

Pacific Patrol Boat Repl

89

91

2

HATS

76

73

0 1

Battle Comm. Sys. (Land) 2A

55

85

30

Maritime Comms

125

132

7

UHF SATCOM

111

132

21

BMS 2

N/A

N/A

N/A

LHD Landing Craft

53

86

38 1

Sub Total – Projects Approved post July 2005

2023

2427

435 1

Percentage of Total – Projects Approved post July 2005

80%

74%

54%

Total – All Projects With Slippage

2521

3281

801

    

Note 1: These figures do not add horizontally due to the exclusion of schedule reductions over the life of the project. Refer to footnote 23.

Note 2: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A achieved FMR in December 2017. MAA closure did not occur in October 2018 as forecast.

Source: ANAO analysis of the 2017–18 PDSSs.

Capability performance analysis

2.44 Defence defines capability as the power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.110 An operational effect is achieved by combining the nine Fundamental Inputs to Capability — organisation, command and management, personnel, collective training, major systems, facilities and training areas, supplies, support, and industry111 — and undertaking designated operations.

2.45 In acquiring Defence platforms and systems, a range of documentation (including capability definition, operational concept, function and performance specification, and test concept documents) is developed, which establishes the detailed requirements/performance attributes to be achieved.

2.46 Since the 2009–10 MPR, capability reporting112 has been based on Defence’s prediction of the final capability that would be achieved on the basis of deliverables and/or activities completed. This assessment of capability performance (Expected Capability) is measured against the Materiel Release Milestones (MRMs) and Completion Criteria specified in each project’s Materiel Acquisition Agreement (MAA). As the ANAO has previously noted, this data involves ‘making certain assumptions in forecasting achievements and is therefore subjective in approach …’.113

2.47 For example, for the LHD Landing Craft project, in the 2015–16 MPR Defence predicted and reported that 99 per cent of elements of capability had a ‘high level of confidence of delivery’, with the capability to transport heavy loads still requiring trials prior to declaration of capability achievement; this capability was assessed as ‘under threat, considered manageable’.

2.48 However, these trials have been delayed on multiple occasions since 30 June 2016, as reported to the JCPAA by Defence at public hearings.114 The 2017–18 PDSS continues to report a 1 per cent Amber rating corresponding to the outstanding trials. This indicates that the subjective 2015–16 forecast of the capability achievement may not have been accurate, with the capability proving harder to achieve than expected. To achieve 100 per cent of capability, the heavy load transportation trials must be successfully completed, or heavy load transport must be removed from the scope of the project’s requirements.

2.49 Over time, the JCPAA has sought the use of a more robust measure of capability performance.115

2.50 In October 2017, the JCPAA recommended ‘that the Department of Defence review the procedure for the development of expected capability estimates for future Major Projects Reports. The outcomes of this review should be provided to the Committee within six months of the tabling of this report. Further, the Committee requests that Defence provide a progress report within three months of the tabling of this report.’116

2.51 Defence made a submission to the Committee in March 2018 regarding this recommendation.

Defence will conduct a schedule baseline validation activity for the Major Projects Report projects to drive greater consistency in schedule reporting.

Once this activity is complete, Defence should be in a better position to investigate a more robust approach to measuring Capability estimates. Utilising the validated baseline data could inform:

  • A simple percentage of schedule milestones achieved to measure progress to date. This is a quantitative assessment that relies on the maintenance of a robust project baseline, which is not dissimilar to the approach proposed by ANAO previously;
  • CASG working with Force Design to identify how to measure capability, that considers all elements of Fundamental Inputs to Capability, and that is suitable for unclassified publication; and
  • Defence is working towards a new whole of organisational reporting system (the Enterprise Resource Planning (ERP) System) which is expected to roll-out in Financial Year 2020-21. CASG will endeavour to incorporate the work conducted with Force Design on measuring capability.117

2.52 Defence advised the ANAO in November 2018 that partial progress has been made on the schedule baseline validation activity to support the proposal for a new approach to measuring capability delivery. The ANAO notes that a measurement of schedule milestones will not necessarily reflect a measurement of capability delivered.

2.53 In September 2018, the JCPAA noted that ‘Materiel Capability Delivery Performance charts continue to be ambiguous in displaying actual current capability levels.’118

Modified method of capability reporting

2.54 In light of the above, in 2015–16 the ANAO developed a measure of key materiel capabilities delivered (Capability Delivery Progress). This presents a current assessment of the capability delivered, which differs from Defence’s prediction of final capability. The information used in forming the ANAO’s assessment is primarily based on Section 4.2 of the PDSS, which sets out the capability elements required to achieve Initial Materiel Release and Final Materiel Release, combined with other information in the PDSS reporting the delivery of equipment/achievement of these requirements toward FOC.

2.55 Noting that a system of capability reporting with a robust methodology applicable to materiel acquisition does not exist within Defence, the information presented below is a more meaningful reflection of current project progress than an end-state prediction.

Capability Delivery Progress and Project Maturity

2.56 Figure 12, below, sets out each project’s Capability Delivery Progress against Project Maturity.119 It shows that Capability Delivery Progress lags Project Maturity for the majority of projects (15 of 26). This relationship is expected as projects will typically develop confidence in the ability to deliver their scope and capability through testing and demonstration of capability components (for example, design reviews and acceptance tests) prior to delivery of the majority of equipment.

2.57 Figure 12 also shows that Capability Delivery Progress lags Project Maturity by 20 per cent or more in 10 projects, and for seven of these, Capability Delivery Progress lags by 50 per cent or more.

Figure 12: Project snapshot — Capability Delivery Progress and Project Maturity

Project snapshot — Capability Delivery Progress and Project Maturity

Source: ANAO analysis of the 2017–18 PDSSs.

2.58 As noted in paragraph 2.10, Defence’s project maturity framework attributes approximately 50 per cent of total project maturity at Second Pass Approval.120 These differences further indicate that Defence’s project maturity framework is not appropriately structured to assign project maturity progress throughout the project life cycle, particularly within the acquisition phase, which is predominantly the longest and most expensive component.

2.59 Figure 12 also highlights a continuing issue with the level of specification of capability elements. For the projects that show little or no Capability Delivery Progress, this can be attributed to Defence’s high level description of requirements in the capability elements. This indicates that it would be worthwhile for Defence to undertake additional work to track project progress. In respect of the four projects that show no capability delivery at 30 June 2018, progress is as follows:

  • Repl Replenishment Ships — this project had commenced construction of both ships, with 35 per cent of blocks erected for the first ship;
  • CMATS — this project was in early stages of procurement, and was progressing through early design processes following signing of the primary acquisition contract;
  • Pacific Patrol Boat Repl — this project had commenced construction of four vessels, with the first vessel launched but not yet accepted into service;
  • Maritime Comms — this project was progressing through design reviews prior to commencing ship installations.

2.60 Further, Figure 12 indicates that:

  • 14 projects are still to deliver part of their capability;
  • seven projects, HATS, ANZAC ASMD 2A and 2B, Collins RCS, Battle Comm. Sys. (Land) 2A, Hw Torpedo, and LHD Landing Craft, have delivered essentially all of their capability with only minor items of capability or administrative processes remaining prior to declaration of FMR/FOC; and
  • one project, BMS, has delivered all of the required capability.

!Part 2. Defence Major Projects Report

Part 2. Defence Major Projects Report is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

!Part 3. Assurance by the Auditor-General and the Secretary of Defence

.

Assurance by the Auditor-General and the Secretary of Defence

PRIORITY ASSURANCE REVIEW – SECTION 19A(5) OF THE AUDITOR-GENERAL ACT 1997

Independent Assurance Report

Department of Defence Project Data Summary Sheets

To the President of the Senate
To the Speaker of the House of Representatives

Conclusion

Based on the procedures I have performed and the evidence I have obtained, nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2017–18 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.

The purpose of the Major Projects Report is to report on the performance of selected major Department of Defence (Defence) equipment acquisition projects (Major Projects), since Second Pass Approval, and associated sustainment activities (where applicable), managed by Defence.

I have undertaken a limited assurance review of the PDSSs, reporting on the status of the projects selected by the Joint Committee of Public Accounts and Audit, and the Statement by the Secretary of Defence, for the year-ended 30 June 2018. The following forecast information was excluded from the scope of this engagement:

  1. Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  2. Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  3. forecast dates where included in each PDSS.

The forecast information has not been included in the scope of the engagement, due to the lack of Defence systems from which to provide complete and accurate evidence, in a sufficiently timely manner to facilitate the review. Accordingly, my conclusion does not provide any assurance in relation to this forecast information. However, material inconsistencies identified in relation to the forecast information, are required to be considered in forming my conclusion.

Basis for Conclusion

I have undertaken a limited assurance review in accordance with the ANAO Auditing Standards, which include the relevant Standard on Assurance Engagements ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Auditing and Assurance Standards Board.

I believe that the evidence I have obtained is sufficient and appropriate to provide a basis for my conclusion.

Responsibilities of the Secretary of Defence for the Project Data Summary Sheets

The Secretary of Defence is responsible for the preparation and presentation of the PDSSs for the 26 selected projects, and the Statement by the Secretary of Defence, in accordance with the Guidelines. This responsibility includes the design, implementation and maintenance of internal control that the Secretary determines is necessary to enable the preparation of PDSSs that are free from material misstatement, whether due to fraud or error. The Guidelines provide that the PDSSs and supporting evidence, provided to the ANAO for review, are complete and accurate.

Independence and Quality Control

I have complied with the independence and other relevant ethical requirements relating to assurance engagements, and applied Auditing Standard ASQC 1 Quality Control for Firms that Perform Audits and Reviews of Financial Reports and Other Financial Information, Other Assurance Engagements and Related Services Engagements in undertaking this assurance review.

Responsibilities of the Auditor-General

My responsibility is to express an independent limited assurance conclusion on the PDSSs and Statement by the Secretary of Defence, based on the procedures I have performed and the evidence I have obtained. ASAE 3000 requires that I plan and perform my procedures to obtain limited assurance about whether anything has come to my attention that the PDSSs and the Statement by the Secretary of Defence have not, in all material respects, been prepared in accordance with the Guidelines.

In a limited assurance engagement, the assurance practitioner performs procedures, primarily consisting of: making enquiries of managers and others within the entity, as appropriate; the examination of documentation; and the evaluation of the evidence obtained. The procedures selected depend on my judgement, including identifying areas where the risks of material misstatement are likely to arise. The procedures performed are detailed at paragraph 1.7 of Part 1 of this report.

The procedures performed in a limited assurance engagement vary in nature and timing from, and are less in extent than those performed for, a reasonable assurance engagement. Consequently the level of assurance obtained in a limited assurance engagement is substantially lower than the assurance that would have been obtained had a reasonable assurance engagement been performed. Accordingly I do not express a reasonable assurance opinion on whether the PDSSs and the Statement by the Secretary of Defence are prepared in all material respects in accordance with the Guidelines.

Grant Hehir
Auditor-General
Canberra
12 December 2018

Statement by the Secretary of Defence

The attached Project Data Summary Sheets (PDSS) for the 26 major projects included in this report have been prepared in accordance with the Guidelines developed by Defence in consultation with the Australian National Audit Office (ANAO) and endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).

Project Status as at 30 June 2018

In my opinion, the Project Data Summary Sheets comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2018.

AIR 87 Phase 2 – ARH Tiger Helicopters

The Armed Reconnaissance Helicopter (ARH Tiger Helicopters) Project has achieved Final Operational Capability (with Caveats) and is no longer reporting in the MPR. In accordance with the 2017-18 MPR Guidelines, projects that have been removed from the MPR but still have outstanding caveats are required to report on the status of these caveats in the Statement by the Secretary of Defence until their final status is accepted by the Capability Manager. The information below addresses this requirement.

In April 2016, Chief of Army declared Final Operational Capability with nine caveats. The 2016-17 MPR reported two of these caveats had closed. Of the remaining caveats, five more were closed in February 2018 and the final two were closed in September 2018. The table below provides further detail on the caveats.

Description of Caveat

Status

Electronic Warfare Self Protection System. The system exhibited some deficiencies.

Caveat was closed in July 2017, as reported in the 2016-17 MPR. The deficiencies were rectified by industry at the end of 2016 at no cost to the Commonwealth.

Identification Friend or Foe System. The system was experiencing technical issues.

Caveat was closed in July 2017, as reported in the 2016-17 MPR. These issues have been rectified.

Availability and Rate of Effort. A minimum of six from eight aircraft available in each of Army’s 161 and 162 squadrons was envisaged. Tiger availability has proven to be closer to four from eight aircraft. A mature Rate of Effort of 7147 hours per year was initially expected. A mature Rate of Effort of 5300 hours is now planned. Tiger workforce issues also contributed to the caveat.

Caveat was closed in September 2018. In hindsight, the initial Rate of Effort expectations were unrealistic for both Industry and Defence. Sufficient availability can be generated to meet Defence’s directed level of capability, along with Raise, Train, Sustain requirements. While some challenges remain, the system is being actively managed through sustainment mechanisms, and there is no advantage in keeping the caveat open.

Communications and mission planning. Limitations existed with the voice and data communications systems and the Ground Mission Equipment mission planning suite.

Caveat was closed in September 2018. Project LAND 9000 ARH Capability Assurance Program will fund the remediation of the radio issues. AIR 9000 Ph2/4/6 Multi-Role Helicopter (Taipan) Project will provide the funding source for the remediation of the mission planning issue, as this will be a common system to support both the Tiger and Taipan platforms.

Missiles. AGM-114M Hellfire missiles are no longer being manufactured. Sufficient stocks were available in the short term, until the replacement missile entered service.

Caveat was closed in February 2018. The issue is being actively managed through existing sustainment tracking and reporting mechanisms, and is funded within the existing approved sustainment budget. All integration testing of the replacement missiles – the AGM-114R – has been completed, with authorisation for in-service use granted in August 2018.

Ammunition. Limited stocks were available at Final Operational Capability.

Caveat was closed in February 2018. Additional stocks have been procured and delivered. The issue is being actively managed through existing sustainment tracking and reporting mechanisms.

Spare parts and consumables. This caveat related to supply constraints on breakdown spares and consumables.

Caveat was closed in February 2018. The issue is being actively managed through existing sustainment tracking and reporting mechanisms, and is funded within the approved sustainment budget.

Class IX Fly Away Kits. This caveat related to the spares kits designed to support a troop-level deployment in a field environment for 14 days.

Caveat was closed in February 2018. The original spares to support the Fly Away Kits required by Army have been delivered. Additional kits may need to be procured if more than one squadron was to be deployed, and this will be managed through existing sustainment budget and reporting mechanisms.

Support. This caveat related to the constrained Defence and industry engineering capacity, which had the potential to affect capability.

Caveat was closed in February 2018. Defence and industry continue to closely manage Tiger engineering priorities. The issue is now being actively managed through existing sustainment tracking and reporting mechanisms.

The Tiger aircraft has reached a level of maturity where it is now meeting and maintaining directed levels of capability.

Significant Events Occurring Post 30 June 2018

In stating this opinion that the PDSSs comply in all material respects with the Guidelines, I acknowledge the following material events have occurred post 30 June 2018:

AIR 6000 Phase 2A/2B - Joint Strike Fighter

In third quarter 2018 Government agreed to a Defence proposal to transfer specific scope elements to other F-35A sub-program elements.

The failure of a fuel tube on a United States Marine Corps aircraft in September 2018 led to the first F-35 crash in more than ten years of flying. This resulted in a temporary pause to Australian flight operations and the replacement of the fuel tubes on the two Australian aircraft which utilise the same part.

SEA 4000 Phase 3 - Air Warfare Destroyer Build

The second Air Warfare Destroyer achieved Provisional Acceptance in July 2018 and was commissioned as HMAS Brisbane on 27 October 2018.

AIR 7000 Phase 2B – P-8A Poseidon

The P-8A Poseidon completed its Operational Test and Evaluation (OT&E) during the Rim of the Pacific Exercise (RIMPAC 2018). As part of this Exercise, the first deployment of weapons by an Australian P-8 Poseidon occurred. The Project will now complete the OT&E Report, expected to be finalised by the end of 2018.

The Poseidon Training System was also delivered in the first half of 2018, enabling the commencement of Australian-based training of operators and maintainers from the 2nd of July 2018.

AIR 5349 Phase 3 – EA-18G Growler Airborne Electronic Attack Capability

Initial Operational Capability was expected to be achieved in July 2018. Defence has met the intent for Initial Operational Capability, however, minor deficiencies in some supporting elements prevented the declaration in mid-2018. These included elements of training, facilities and Information and Communication Technology systems. Air Force is currently developing a case for Initial Operational Capability declaration in the near term.

LAND 121 Phase 3B – Overlander Medium/Heavy

Contract Change Proposals were signed for the LAND 121 Phase 3B acquisition contract with Rheinmetall MAN Military Vehicles Australia. These Contract Change Proposals were for the provision of an additional 1,044 vehicles and 872 modules, and Haulmark Trailers Australia for 812 trailers on behalf of the LAND 121 Phase 5B project, and signed on 04 September 2018 and 29 August 2018 respectively.

JP 2048 Phase 4A/4B – LHD Ships

Final Materiel Release has been delayed from December 2018 to October 2019.

LAND 121 Phase 4 - Protected Mobility Vehicle – Light

Two Protected Mobility Vehicle-Light (Hawkei) were successfully deployed to Afghanistan and Iraq, with the eight month deployment concluding on 10 August 2018.

The Reliability Demonstration Test was completed on 19 November 2018, with reliability issues outstanding. The Operator component of the Support System Detailed Design Review was completed in August 2018.

Defence senior management is intensively managing the program to address the outstanding reliability issues. Defence has requested that Thales provides a formal plan to resolve the outstanding reliability issues to inform the decision to commence the Production Reliability Acceptance Test. The achievement of key project milestones is reliant on the resolution of these reliability issues.

SEA 1654 Phase 3 – Replacement Replenishment Ships

The first Auxiliary Oiler Replenishment (AOR) Ship, Supply, was launched in Ferrol Spain on 23 November 2018.

The keel laying for the second AOR Ship, Stalwart, was achieved on 24 November 2018.

JP 2072 Phase 2B – Battle Comm. Sys. (Land) 2B

Preliminary Design Review for the Release 2, which builds on Release 1 delivered for Initial Operating Capability, was conducted and successfully exited in July 2018. Final Materiel Release and Final Operational Capability will be delayed due to the Commonwealth not providing items of Government Furnished Materiel required by the contractor for integration and testing. The Government Furnished Materiel to be provided is the Enhanced Deployable Local Area Network (eDLAN). The eDLAN program is negotiating potential contract changes.

SEA 1448 Phase 2A & 2B - ANZAC Anti-Ship Missile Defence

Final Materiel Release was achieved in November 2018, and Final Operational Capability is scheduled for achievement in March 2019.

SEA 3036 Phase 1 – Pacific Patrol Boat Replacement

The contract for delivery of infrastructure upgrades in Papua New Guinea was signed on 7 September 2018. The contract is for $5 million with PNG Contractor Fletcher Morobe Constructions Limited

The first Pacific Patrol Boat (HMPNGS Rabaul) arrived in Australia on 21 October 2018 for environmentally responsible disposal. The crew is now undergoing conversion training at Austal prior to receiving their new Guardian Class Patrol Boat.

Harbour Acceptance Trials has been completed and the Acceptance Test Reports are being approved by the Project. Chief of Navy declared Initial Materiel Release and Initial Operating Capability with effect 30 November 2018, to coincide with Acceptance and Handover of the first Guardian Class Patrol Boat to Papua New Guinea on 30 November 2018.

JP 9000 Phase 7 - Helicopter Aircrew Training System

The ADF’s new Joint Helicopter School has graduated its first course of Australian Navy and Army pilots, aviation warfare officers and helicopter aircrew on 31 August 2018. The second Pilot, Aviation Warfare Officer and Aircrew courses commenced in October 2018.

JP 2072 Phase 2A – Battlespace Communications System

The Final Operational Capability is forecast for December 2018, and is on track for achievement.

SEA 1442 Phase 4 - Maritime Communications Modernisation

The alignment of the SEA 1442 Phase 4 project to the Anzac Mid-life Capability Assurance Program was reflected in an approved update of the Materiel Acquisition Agreement in September 2018. Initial Operational Capability is planned to occur by the first quarter 2020, and Final Operational Capability by the first quarter 2025.

SEA 1439 Phase 4A – Collins Replacement Combat System and SEA 1429 Phase 2 – Replacement Heavyweight Torpedo

Final Materiel Release for the Collins Replacement Combat System and the Heavyweight Torpedo was scheduled for 31 October 2018, however, this did not occur. All technical work was complete in June 2018. This project is now in its closure phase and Final Materiel Release and Final Operating Capability are expected to occur by mid-2019 and the project to be closed by the end of 2019.

JP 2048 Phase 3 – LHD Landing Craft

The LHD Landing Craft sea trials involving carriage of the M1A1 Main Battle Tank were scheduled for the second quarter 2018, however did not proceed due to an ongoing technical assessment being undertaken by Defence.

The sea trials have been re-scheduled to occur during the second/third quarter 2019, during the annual amphibious exercise period where the necessary assets can be made available. Final Operational Capability declaration will occur after the sea trials later in 2019.

Greg Moriarty
Secretary
Department of Defence
11 December 2018

!Part 4. JCPAA 2017–18 Major Projects Report Guidelines

The JCPAA 2017–18 Major Projects Report Guidelines part is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

Footnotes

1 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), Executive Summary, p. 1.

2 ibid.

3 Defence states that CASG ‘exists to meet the Australian Defence Force’s (ADF) military equipment and supply requirements as identified by Defence and approved by Government’. Department of Defence, About CASG [Internet], Defence, available from http://www.defence.gov.au/casg/AboutCASG/ [accessed 8 October 2018].

4 Department of Defence, Defence Annual Report 17–18, Chapter 3, Annual Performance Statements, Defence, Canberra, 2018, p. 35.

5 ibid., Chapter 11, Financial Statements, p. 175.

6 Department of Defence, Defence Portfolio Budget Statements 2018–19, Defence, Canberra, 2018, p. 5.

7 A performance audit assessing the effectiveness to date of Defence’s planning for the mobilisation of its continuous shipbuilding programs in Australia was tabled during this report period (Auditor-General Report No.39 2017–18 Naval Construction Programs—Mobilisation).

8 M Turnbull (Prime Minister), C Pyne (Minister for Defence Industry), M Payne (Minister for Defence), M Cormann (Minister for Finance), ‘The Hunter Class – defending Australia and securing our shipbuilding sovereignty’, media release, Parliament House, Canberra, 29 June 2018.

9 The 2017–18 Major Projects Report Guidelines were endorsed by the JCPAA in September 2017 and are included in Part 4 of this report.

10 Department of Defence, Defence Annual Report 17–18, Chapter 3, Annual Performance Statements, Defence, Canberra, 2018, p. 35.

11 For example, Defence project risk management records can be managed in spreadsheets, where the risk to the completeness and accuracy of records is too high to be included within the scope of the review.

12 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016-17), (2018), Recommendation 2, p. vii.

13 The MPR Guidelines provide that data of a classified nature is to be prepared in such a way as to allow for unclassified publication.

14 IMR and FMR are milestones that Defence utilises to mark the completion and release of acquisition project supplies required to support the achievement of IOC and FOC respectively. They are defined in the relevant MAA (Materiel Acquisition Agreement). See Department of Defence, Defence Material Standard Procedure (Project Management) DMSP (PROJ) 11-0-008, ‘Initial Material Release And Final Material Release Across The Project Lifecycle’, 2013, p. 2.

15 Declaration of initial operating capability (IOC) is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group that the fundamental inputs to capability have been delivered. Declaration of final operating capability (FOC) is made by the Capability Manager, supported by the results of operational test and evaluation and confirmation by the Delivery Group that the fundamental inputs to capability have been delivered as agreed and also marks the conclusion of the acquisition phase. See Department of Defence, ‘Interim Capability Life Cycle Manual’, 2017, p. 41.

16 The project maturity framework — outlined in the Department of Defence’s Defence Materiel Standard Procedure (Project Management), DMSP (PROJ) 11-0-007, ‘Project Maturity Scores at Life Cycle Gates’, 2010 — is a methodology used to quantify the maturity of projects as they progress through the acquisition life cycle.

17 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. See paragraph 1.55 for further detail.

18 Refer to footnote 16.

19 Defence has advised that: ‘Real Cost Increases [are] attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget: excluding ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; and ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors.’

20 In addition to the Real Cost Increase received by the CMATS project, Air Force received $2.2 million of additional funds for CMATS, making a total Real Cost Increase of $242.9 million paid by Government to Defence. At the same time Government also returned to the project $6.8 million of funds that had been temporarily harvested.

21 The JCPAA requested in May 2018 that the ANAO report back to the Committee on how Defence Major Projects cost variations and the costs of retaining project staff over time might be reported in future MPRs. See paragraphs 1.45 to 1.49 for the outcomes of this consideration.

22 See Defence’s analysis in Part 2 of this report.

23 As noted in Note 5 of Table 2, slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project. In November 2017, Defence raised with the ANAO, for the purposes of calculating total schedule slippage, the feasibility of identifying what the proportion of slippage represented by the expanded scope of projects is (for example with respect to the P-8A Poseidon and Additional MRTT projects). See Note 3 of Figure 7 of this report which shows that the slippage attributable to increases in project scope is 58 months.

24 See Table 6 in Part 2 of this report for Defence classifications.

25 Off-The-Shelf: Systems, hardware or software that already exists or is confirmed in service for an equivalent purpose and requires no, or minimal, change. Sometimes expressed as commercial off-the-shelf or military off-the-shelf. Department of Defence, ‘Interim Defence Test and Evaluation Manual’, 2016, Annex 1A, Definitions, p. iii.

26 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 442: Inquiry into the 2012-13 Defence Materiel Organisation Major Projects Report, (2014), Recommendation 5, p. 31.

27 Refer to footnote 11.

28 The Auditor-General was unable to provide an unqualified Independent Assurance Report for 2016–17 as a number of matters were identified, in the course of the ANAO’s review, that resulted in the qualification of progress and performance as reported in the ARH Tiger Helicopters PDSS. See Auditor-General Report No.26 of 2017–18, 2016–17 Major Projects Report, paragraphs 20–25 and pp. 111–113.

29 Department of Defence, ‘Independent Assurance Reviews for Projects and Sustainment Products’, 2016, pp. 3 and 9.

30 Although referred to by Defence as ‘assurance’ reviews, these administrative reviews are not carried out within frameworks issued by the Australian Auditing and Assurance Standards Board.

31 Defence advised in November 2017 that ‘Gate Review’ is now a description for a separate process that leads to Gate submission (to the Investment Committee) including the CASG Independent Assurance Review and the Capability Manager Gate Review.

32 Independent Assurance Reviews were conducted for: Joint Strike Fighter, AWD Ships, MRH90 Helicopters, Growler, MH-60R Seahawk, Overlander Medium/Heavy, LHD Ships, Hawkei, Battlefield Airlifter, Repl Replenishment Ships, Battle Comm. Sys. (Land) 2B, CMATS, Pacific Patrol Boat Repl, HATS, Battle Comm. Sys. (Land) 2A, UHF SATCOM, and LHD Landing Craft. Fourteen projects had reviews scheduled for late 2018.

33 Department of Defence, Defence Annual Report 17–18, Chapter 8, Asset Management, Purchasing and Capital Investment, Defence, Canberra, 2018, p. 123.

34 A performance audit assessing whether Defence’s Project of Concern regime is effective in managing the recovery of underperforming projects is expected to table in Quarter 3 of 2018–19.

35 Issues in the project were discussed in Auditor-General Report No.52, 2013–14, Multi-Role Helicopter Program.

36 C Pyne (Minister for Defence Industry), M Payne (Minister for Defence), ‘Projects of Concern Update’, media release, Parliament House, Canberra, 18 August 2017.

37 Defence has advised that the project was placed on the list because ‘negotiations were taking longer than expected, and the costs were increasing significantly, and the schedule was not being achieved.’ Commonwealth, Estimates, Senate Foreign Affairs, Defence, and Trade Legislation Committee, 25 October 2017, VADM R Griggs, Acting Chief of the Defence Force, Department of Defence.

38 See the CMATS PDSS in Part 3 of this report.

39 C Pyne (Minister for Defence Industry), M Payne (Minister for Defence), ‘OneSKY ready for take off’, media release, Parliament House, 26 February 2018.

40 A performance audit assessing whether the contract for the acquisition of the CMATS has demonstrably delivered value for money is in progress.

41 Department of Defence, June 2018, Quarterly Performance Report, Defence, Canberra, 2018, p. 5.

42 Auditor-General Report No.2 2017–18, Defence’s Management of Materiel Sustainment, paragraph 16, p. 9, found that the contents of Quarterly Performance Reports are neither complete nor reliable, and may not include additional information available to Defence that is critical to the reader’s ability to understand the status of significant military platforms. Since this audit, Defence has advised that it has reduced the jargon within QPRs and included considerations of Maturity Scores and Independent Assurance Review outcomes.

43 These are CASG acquisition projects that have variances significant enough — in the areas of schedule, cost, and/or capability performance — to warrant attention from senior management. Department of Defence, June 2018, Quarterly Performance Report, Defence, Canberra, 2018, p. 15.

44 UHF SATCOM had 12 months of in-year slippage.

45 The Project Directive defines the Project, in terms of fundamental inputs to capability, together with the resources necessary to deliver the project and is developed in accordance with the exact parameters agreed by government. Department of Defence, ‘Interim Capability Life Cycle Manual’, Annex A, 2017, p. 93. The mechanism for providing the directive is via the Capability Life Cycle (CLC) management tool, which records the Government decision in relation to a project. The accountabilities and responsibilities of specific roles within the CLC are defined in the Interim Capability Life Cycle Manual. Where necessary, the Joint Force Authority may provide a specific documented directive.

46 The Interim Capability Life Cycle Manual does not describe MAAs and instead refers to Product Delivery Agreements (PDAs) (see paragraph 1.23). Projects in this MPR have an approved MAA.

47 Joint Strike Fighter (Stage 2), P-8A Poseidon, Growler, MH-60R Seahawk, Overlander Medium/Heavy, Battlefield Airlifter, Repl Replenishment Ships, Battle Comm Sys (Land) 2B, Additional MRTT, CMATS, Pacific Patrol Boat Repl, HATS, Battle Comm. Sys. (Land) 2A, Maritime Comms, BMS and LHD Landing Craft. As at 30 June 2018, the JPD for the Hawkei project was still in draft.

48 A PDA is an agreement between the Sponsor and Lead Delivery Group which specifies the scope, resourcing, priorities and performance and preparedness requirements for support of a capability system throughout its life, to support performance measurement. Department of Defence, ‘Interim Capability Life Cycle Manual’, Annex A, Definitions, 2017, p. 92.

49 Business system weaknesses, such as project offices having inconsistent record keeping and methods of tracking project progress were highlighted by the Committee in Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, (2014), paragraph 3.116, p. 39. With regard to Defence’s risk management systems see Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), paragraph 2.10, p. 5.

50 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. See paragraph 1.55 for further detail.

51 Department of Defence, (PM) 003, ‘CASG Project Controls Manual’, Acronyms, Abbreviations and Definitions, 2017, p. 8.

52 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 436: Review of the 2011–12 Defence Materiel Organisation Major Projects Report, (2013), paragraph 3.4, p. 14.

53 At the same time government also returned $6.8 million of funds that had been temporarily harvested.

54 Auditor-General Report No.26 of 2017–18, 2016–17 Major Projects Report, p. 265.

55 Department of Defence, DMM (PROJ) 11-0-002, ‘DMO Project Risk Management Manual 2013’, Chapter 9 – Management of Contingency Budgets in DMO Acquisition Projects, 2013, p. 108.

56 The manual requires that the Project Contingency Budget Log is kept up to date for the proper overall management of risk and that it is submitted for internal review at Additional and Budget estimates.

57 Slippage in this context refers to ‘ensuring that planned project expenditure does not exceed available cash… [as these] underspends would be forfeited in each year and gradually reduce the total approved budget available for a project.’ As noted in Department of Defence, ‘Contingency Funding Management’, Internal Audit, 2018, p. 6.

58 Department of Defence, ‘Contingency Funding Management’, Internal Audit, 2018, pp. 3 and 5.

59 See the ANZAC ASMD 2B PDSS in Part 3 of this report.

60 A performance audit on sustainment of the ANZAC class frigates is in progress and is expected to table in February 2019.

61 The Collins R&S project does not have a formal contingency allocation.

62 Auditor-General Report No.26 2017–18, 2016–17 Major Projects Report, p. 41.

63 The reporting of cost variations was also raised at the JCPAA’s public hearing into the 2016–17 MPR on 23 March 2018 and at additional estimates on 27 February 2018.

64 Where a project has multiple Second Pass Approvals, the budget at Second Pass Approval reported in the Header of the PDSS refers to the total budget as at the latest Second Pass Approval. The body of the PDSS also references the budget at initial Second Pass Approval.

65 This can be found in the following Sections of each PDSS: Header and Section 2.1 Project Budget (out-turned) and Expenditure History.

66 Defence has advised that CASG is implementing a ‘Capacity Management’ system to examine workforce demand and supply data, which is expected to reach maturity in 2020.

67 See paragraph 1.3 in Part 1 of this report for more information.

68 See Part 2 of this report.

69 Predict! is a risk management tool used by Defence to manage risks and issues.

70 The Defence Project Risk Management Manual version 2.4, Business Rule 2 requires the project manager to validate the currency of the RMP on transition from one stage of the Materiel Life Cycle to the next stage and, for any stage that is longer than six months, every six months within that stage.

71 Defence risk management guidance for acquisition projects is the Project Risk Management Manual, version 2.4, 2013; and guidance for sustainment products is the DMM (LOG) 04-0-003, Defence Materiel Manual (Logistics Management), which provide different consequence and likelihood descriptors.

72 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error.

73 The 13 projects are: MRH90 Helicopters, Growler, Battlefield Airlifter, Repl Replenishment Ships, Additional MRTT, CMATS, ANZAC ASMD 2B, HATS, Battle Comm. Sys. (Land) 2A, UHF SATCOM, Collins R&S, ANZAC ASMD 2A and BMS.

74 The seven projects are: AWD Ships, P-8A Poseidon, Overlander Medium/Heavy, Hawkei, Battle Comm. Sys. (Land) 2B, Pacific Patrol Boat Repl and Maritime Comms.

75 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), List of Recommendations, p. vii.

76 Department of Defence, DMM (PROJ) 1-0-001, ‘DMO Project Management Manual 2012’, Glossary, 2012, p. 75. This manual has since been superseded by PM 002 ‘CASG Project Management Manual’ which does not refer to project maturity.

77 See Appendix 4 in Part 2 of this report and footnote 16 for further detail.

78 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014-15), (2016), Recommendation 3, p. 50. The JCPAA sought an update from Defence in the course of public hearings on 31 March 2017.

79 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), Recommendation 2, p. vii.

80 Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 23 March 2018, Mr G Divall, Group Business Manager, Capability Acquisition and Sustainment Group, Department of Defence, p. 11.

81 In May 2016, Defence described caveats to the ANAO as ‘alerts’ to operational decision makers about risks to be taken into account when making decisions about the use of the ARH Tiger Helicopter in particular operational circumstances. Auditor-General Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, p. 25.

82 The following projects which have exited the MPR, had also achieved FOC with caveats: Wedgetail (achieved FOC with caveats in 2015), Overlander Light (achieved FOC with caveats in 2016), ARH Tiger Helicopters (achieved FOC with caveats in 2016 — see Part 3 of this report).

83 This requirement was agreed to by Defence and included in the 2017–18 Major Projects Report Guidelines endorsed by the JCPAA in September 2017 which are included in Part 4 of this report.

84 See Statement by the Secretary of Defence p. 135.

85 Refer to footnote 14 for the definition of IMR and FMR milestones, and footnote 15 for the definition of IOC and FOC milestones.

86 A project’s budgeted cost and schedule data is at 30 June 2018, and may differ from originally approved budgets and schedules.

87 The JCPAA has previously recommended that Defence work with the ANAO to review and revise its policy regarding Project Maturity Scores. Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014–15), (2016), pp. 49–50, and Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), pp. 9–10. In Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016-17), (2018), the JCPAA recommended that Defence advise the Committee of progress in updating Project Maturity Scores.

88 The JCPAA has observed that “Defence remains behind the Committee’s expectations on working to update Project Maturity Scores – the Committee recommended reform in this area several years ago, and changes remain slow and uncertain.” Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016-17), (2018), p. 4.

89 Budget variances since initial Second Pass Approval may result from: increasing the scope of a project via revised Second Pass Approvals, programmatic decisions, Real Cost Increases/Decreases, transfers to/from other projects and budgetary adjustments. A summary of budget variations is at Table 3 and a more detailed analysis of this variance is included in Table 8.

90 See Note 2 of Figure 4, below, for further information.

91 Real Variations include ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors; ‘Real Cost Increases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget; and ‘Real Cost Decreases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been handed back to the Defence portfolio.

92 During 2017–18, $6.8 million was temporarily harvested from and returned to the project budget with no net impact on the budget approved at Second Pass.

93 Australian Government arrangements for foreign exchange variation involve ‘no win/no loss’ supplementation. As a matter of policy, unless specifically approved, individual entities are not permitted to ‘hedge’ against foreign exchange risk.

94 It should be noted that the PDSSs report expenditure on a cash basis, consistent with the budget figures, for the first time in 2017–18.

95 See Table 2 in Part 1 of this report.

96 Extensions to planned withdrawal dates may involve additional costs relating to the maintenance and servicing of equipment.

97 See paragraphs 2.29 to 2.33 and Figures 7 and 8 for more information.

98 Refer to footnote 16 for more detail.

99 Refer to footnote 23.

100 M Kinnaird, Defence Procurement Review 2003, Department of Defence, Canberra, 2003.

101 Of the five projects added to the MPR in the last two years, three have been developmental (Hawkei, CMATS, and Battle Comm. Sys. (Land) 2B). Of these projects, CMATS has experienced 28 months of schedule slippage, while Hawkei and Battle Comm. Sys. (Land) 2B are yet to experience slippage.

102 D Peever, First Principles Review: Creating One Defence, Department of Defence, Canberra, 2015, p. 34 and p. 92.

103 Refer to footnote 23.

104 See the Collins RCS, Hw Torpedo and Collins R&S PDSSs in Part 3 of this report.

105 Auditor-General Report No.6 2013–14, Capability Development Reform, paragraphs 9.1 to 9.4, pp. 198–199.

106 Further information on MRH90 Helicopters can be found in Auditor-General Reports No.48 2008–09, Planning and Approval of Defence Major Capital Equipment Projects, pp. 84, 90 and 133; No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, pp. 86–87 and pp. 130–133; and No.52 2013–14, Multi-Role Helicopter Program.

107 In the Statement by the Secretary of Defence in Part 3 of this report, the Secretary also makes reference to additional information on achieved milestone dates for AWD Ships, P-8A Poseidon, Repl Replenishment Ships, ANZAC ASMD 2A and 2B, and Pacific Patrol Boat Repl.

108 See paragraphs 2.47 to 2.48 below, and the Statement by the Secretary of Defence and the LHD Landing Craft PDSS in Part 3 of this report for more information on this project. The Secretary notes that ‘the sea trials have been re-scheduled to occur during the second/third quarter 2019, during the annual amphibious exercise period where the necessary assets can be made available. Final Operational Capability declaration will occur after the sea trials later in 2019.’

109 Tables 4 and 5 report on the slippage for each project that has been in the MPR since 2007–08.

110 Department of Defence, ‘Interim Capability Life Cycle Manual’, 2017, p. 85.

111 Department of Defence, ‘Interim Capability Life Cycle Manual’, 2017, p. 13.

112 As per the 2017–18 MPR Guidelines, a project is defined as the acquisition or upgrade of Specialist Military Equipment, which normally excludes facilities and other Fundamental Inputs to Capability. The 2017–18 MPR Guidelines also note that the MPR may report on associated sustainment activities (where applicable).

113 Auditor-General Report No.17 2010–11, 2009–10 Major Projects Report, p. 35.

114 Source 1: Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 17 March 2016, Mr. P Croser, Director General, Specialist Ships Acquisition, Capability Acquisition and Sustainment Group, Department of Defence, p. 5.

Source 2: Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 31 March 2017, Mr K Gillis, Deputy Secretary, Capability Acquisition and Sustainment Group, Department of Defence, p. 14.

Source 3: Commonwealth, Public Hearing, Joint Committee of Public Accounts and Audit, 23 March 2018, CDRE S Hughes, Deputy General, Littoral, Department of Defence, p. 13.

115 Source 1: Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 442: Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, (2014), pp. 37–39.

Source 2: Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 458: Defence Major Projects Report (2014-15), (2016), pp. 48–49.

116 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 468: Defence Major Projects Report (2015–16), (2017), Recommendation 1, p. vii.

117 Department of Defence, written submission to the Joint Committee of Public Accounts and Audit, Inquiry into the 2016-17 Defence Major Projects Report, p. 1.

118 Joint Committee of Public Accounts and Audit, Parliament of the Commonwealth of Australia, Report 473: Defence Major Projects Report (2016–17), (2018), p. 2.

119 Refer to footnote 89 for more detail.

120 Refer to footnote 90 for more detail.