The objective of this report was to provide the Auditor‐General’s independent assurance over the status of the selected Major Projects, as reflected in the Statement by the Secretary of Defence, and the Project Data Summary Sheets (PDSSs) prepared by Defence, in accordance with the Guidelines endorsed by the Joint Committee of Public Accounts and Audit.

Due to the complexity of material and the multiple sources of information for the 2015-16 Major Projects Report, we are unable to represent the entire document in HTML. You can download the full report in PDF or view selected sections in HTML below. PDF files for individual Project Data Summary Sheets (PDSS) are also available for download.

!Part 1. ANAO Review and Analysis

Summary and Review Conclusion

The Major Projects Report

1. Major Defence equipment acquisition projects (Major Projects) continue to be the subject of parliamentary and public interest. This is due to their high cost and contribution to national security, and the challenges involved in completing them within the specified budget and schedule, and to the required capability.

2. The Australian National Audit Office (ANAO) has reviewed 26 of Defence’s Major Projects in this ninth report (2014–15: 25). The objective of the report is ‘…to improve the accountability and transparency of Defence acquisitions for the benefit of Parliament and other stakeholders.’1

3. The Capability Acquisition and Sustainment Group (CASG) within the Department of Defence (Defence), manages the process of bringing new capabilities into service.2 In 2015–16 CASG provided support to the Australian Defence Force (ADF) through the acquisition and sustainment of required military equipment and supplies3, and expended some $6.8 billion on major and minor capital acquisition projects.4

4. The February 2016 Defence White Paper established the Government’s priorities for future capability investment for the next 20 years and provided for additional spending of over $29 billion across the next decade. More recently, the Minister for Defence Industry announced an investment of $195 billion over the next decade, for building Defence capability.5

Major Projects selected for review

5. Major Projects are selected for review, based on the criteria included in the 2015–16 Major Projects Report (MPR) Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).6 They represent a selection of the most significant Major Projects managed by Defence.

6. The total approved budget for the Major Projects included in this report is approximately $62.7 billion, covering nearly 58 per cent of the budget within the Approved Major Capital Investment Program of $107.4 billion.7 The selected projects and their approved budgets are listed in Table 1, below.

Table 1: 2015–16 MPR projects and approved budgets at 30 June 2016 1, 2

Project Number (Defence Capability Plan)

Project Name (on Defence advice)

Defence Abbreviation (on Defence advice)

Approved Budget $m

AIR 6000 Phase 2A/2B

New Air Combat Capability

Joint Strike Fighter

16 738.4

SEA 4000 Phase 3

Air Warfare Destroyer Build

AWD Ships

9 120.8

AIR 7000 Phase 2B

Maritime Patrol and Response Aircraft System

P-8A Poseidon

5 519.9

AIR 9000 Phase 2/4/6

Multi-Role Helicopter

MRH90 Helicopters

3 773.9

AIR 5349 Phase 3

EA-18G Growler Airborne Electronic Attack Capability

Growler

3 556.5

AIR 9000 Phase 8

Future Naval Aviation Combat System Helicopter

MH-60R Seahawk

3 520.4

LAND 121 Phase 3B

Medium Heavy Capability, Field Vehicles, Modules and Trailers

Overlander Medium/Heavy

3 465.6

JP 2048 Phase 4A/4B

Amphibious Ships (LHD)

LHD Ships

3 092.9

AIR 87 Phase 2

Armed Reconnaissance Helicopter

ARH Tiger Helicopters

2 033.0

AIR 5402

Air to Air Refuelling Capability

Air to Air Refuel

1 821.7

AIR 8000 Phase 2

Battlefield Airlift – Caribou Replacement

Battlefield Airlifter

1 434.5

LAND 116 Phase 3

Bushmaster Protected Mobility Vehicle

Bushmaster Vehicles

1 250.7

LAND 121 Phase 3A

Field Vehicles and Trailers

Overlander Light

1 017.7

AIR 7403 Phase 3

Additional KC-30A Multi-role Tanker Transport

Additional MRTT3

911.4

SEA 1448 Phase 2B

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2B

678.6

AIR 9000 Phase 5C

Additional Medium Lift Helicopters

Additional Chinook

642.4

JP 9000 Phase 7

Helicopter Aircrew Training System

HATS3

487.6

JP 2072 Phase 2A

Battlespace Communications System

Battle Comm. Sys. (Land)

464.6

SEA 1442 Phase 4

Maritime Communications Modernisation

Maritime Comms

456.0

SEA 1439 Phase 4A

Collins Replacement Combat System

Collins RCS

450.6

SEA 1429 Phase 2

Replacement Heavyweight Torpedo

Hw Torpedo

429.7

JP 2008 Phase 5A

Indian Ocean Region UHF SATCOM

UHF SATCOM

421.4

SEA 1439 Phase 3

Collins Class Submarine Reliability and Sustainability

Collins R&S

411.7

SEA 1448 Phase 2A

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2A

386.8

LAND 75 Phase 4

Battle Management System

BMS3

372.8

JP 2048 Phase 3

Amphibious Watercraft Replacement

LHD Landing Craft

237.9

Total: 26

 

 

62 697.5

       

Note 1: Once a project is selected for review, it remains within the portfolio of projects under review until the JCPAA endorses its removal, normally once it has met the capability requirements of Defence.

Note 2: Wedgetail and Battle Comm. Sys. were removed from the MPR program in 2015–16.

Note 3: Additional MRTT, HATS and BMS are included in the MPR program for the first time in 2015–16.

Source: See the Project Data Summary Sheets in Part 3 of this report.

Report objective and scope

7. The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects. The status of the selected Major Projects is reported in the Statement by the Secretary of Defence and the Project Data Summary Sheets (PDSSs) prepared by Defence. Assurance from the ANAO’s review is conveyed in the Independent Assurance Report by the Auditor-General.

8. The following forecast information is excluded from the scope of the ANAO’s review:

  • Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  • Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  • forecast dates where included in each PDSS.

Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.

9. The exclusions to the scope of the review noted above, are due to a lack of Defence systems from which to provide complete and accurate evidence8, in a sufficiently timely manner to facilitate the review. This has been an area of focus of the JCPAA over a number of years9, and it is intended that all components of the PDSSs will eventually be included within the scope of the ANAO’s review.

10. Separate to the formal review, the ANAO has undertaken an analysis of key elements of the PDSSs—including cost, schedule, progress towards delivery of required capability, project maturity, and risks and issues. Longitudinal analysis across these key elements of projects, over time, has also been undertaken.

11. Defence provides further insights and context in its commentary and analysis—although this is not included within the scope of the ANAO’s review.

Review methodology

12. The ANAO has reviewed the PDSSs as a priority assurance review under section 19A(5) of the Auditor-General Act 1997. The criteria to conduct the review are provided by the Guidelines, and include whether Defence has procedures in place designed to ensure that project information and data was recorded in a complete and accurate manner, for all 26 projects.

13. The review included an assessment of the systems and controls, including the governance and oversight in place, to ensure appropriate project management. The ANAO also sought representations and confirmations from senior management and industry in relation to the status of the Major Projects in this report.

Report structure

14. The report is organised into four parts:

  • Part 1 comprises the ANAO’s review and analysis (pp. 1–56);
  • Part 2 comprises Defence’s Commentary, Analysis and Appendices (not included within the scope of the Independent Assurance Report by the Auditor-General) (pp. 57–126);
  • Part 3 incorporates the Independent Assurance Report by the Auditor-General, the Statement by the Secretary of Defence, and the PDSSs prepared by Defence as part of the assurance review process (pp. 127–454); and
  • Part 4 reproduces the 2015–16 MPR Guidelines endorsed by the JCPAA, which provide the criteria for the compilation of the PDSSs by Defence and the ANAO’s review (pp. 455–484).

Figure 1, below, depicts the four parts of this report.

Figure 1: 2015–16 Report structure

2015–16 Report structure

Note: To assist in conducting inter-report analysis, the presentation of data in the PDSSs remains largely consistent and comparable with the 2014–15 MPR.

Project Data Summary Sheets

15. The PDSSs include unclassified information on project performance, prepared by Defence. As projects appear in the MPR for multiple years, changes to the PDSS from the previous year are depicted in bold orange text.

16. Each PDSS comprises:

  • Project Header: including name; capability and acquisition type; approval dates; total approved and in-year budgets; stage; complexity; and an image;
  • Section 1—Project Summary: including description; current status, financial assurance and contingency statement; context, including background, uniqueness, major risks and issues; and other current sub-projects;
  • Section 2—Financial Performance: including budgets and expenditure; variances; and major contracts in place (in addition to quantities delivered as at 30 June 2016);
  • Section 3—Schedule Performance: providing information on design development; test and evaluation; and forecasts and achievements against key project milestones, including Initial Materiel Release (IMR), Final Materiel Release (FMR)10, Initial Operational Capability (IOC) and Final Operational Capability (FOC)11;
  • Section 4—Materiel Capability Delivery Performance: provides a summary of Defence’s assessment of its expected delivery of key capabilities, the extent to which milestones were achieved, and a description of the constitution of each key milestone;
  • Section 5—Major Risks and Issues: outlines the major risks and issues of the project and remedial actions undertaken for each;
  • Section 6—Project Maturity: provides a summary of the project’s maturity, as defined by Defence, and a comparison against the benchmark score;
  • Section 7—Lessons Learned: outlines the key lessons that have been learned at the project level (further information on lessons learned by Defence are included in Defence’s Appendix 5); and
  • Section 8—Project Line Management: details current project management responsibilities within Defence.

Overall outcomes

Statement by the Secretary of Defence

17. The Statement by the Secretary of Defence was signed on 15 February 2017. The Secretary’s statement provides his opinion that the PDSSs for the 26 selected projects ‘… comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2016.’

18. The Secretary also ‘acknowledge[s] the difference of view between Defence and the ANAO in relation to the AIR 87 Phase 2 – Armed Reconnaissance Helicopter (Tiger) and the JP2048 Phase 3 LHD Landing Craft (LLC)’. Further detail is provided in paragraphs 20 to 24 below (see Conclusion by the Auditor-General).

19. In addition, the Statement by the Secretary of Defence details significant events occurring post 30 June 2016, which materially impact the projects included in the report, and which should be read in conjunction with the individual PDSSs. These include AWD Ships, P-8A Poseidon, LHD Ships, ARH Tiger Helicopters, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Additional MRTT, Battle Comm. Sys. (Land), Maritime Comms, Collins RCS, Collins R&S, BMS, and LHD Landing Craft.

Conclusion by the Auditor-General

20. The Auditor-General was unable to provide an unqualified Independent Assurance Report as a number of matters were identified, in the course of the ANAO’s review, that resulted in the qualification of progress and performance as reported in two Project Data Summary Sheets (PDSSs).

21. The Guidelines define a project as the acquisition or upgrade of Specialist Military Equipment. The Guidelines provide that the scope of Defence reporting includes the performance of selected major equipment acquisitions and associated sustainment activities, where applicable.

22. The ARH Tiger Helicopters PDSS has been prepared on the basis of the Defence acquisition project12, which is narrower than the scope established in the Guidelines.

  • The Project Financial Assurance Statement in Section 1.2 of the PDSS reports that sufficient funding is available to complete the acquisition project. The statement does not address the significant caveats, capability deficiencies and obsolescence issues identified in the declaration of FOC, in April 2016.13 Additional funding for these elements would need to be provided separately to the acquisition project, the amount of which is unable to be quantified.
  • The project maturity score in Section 6.1 of the PDSS reports a total of 67 out of a maximum of 70 (95.7 per cent) at the end of the acquisition project. Noting the significant caveats, capability deficiencies and obsolescence issues at FOC, this score does not accurately or completely represent the project’s maturity as at 30 June 2016.

23. In addition, the following material inconsistencies have been identified in the forecast information:

  • Section 4.1 in the ARH Tiger Helicopters PDSS reports that materiel capability delivery performance is at 99.8 per cent, indicating that there is a high degree of confidence that materiel capability performance will be met. Expert analysis commissioned by Defence indicates that the program will remain incapable of fully meeting expectations relating to reliability, availability, maintainability and rate of effort.14
  • Section 4.1 in the LHD Landing Craft PDSS reports that materiel capability delivery performance is at 99 per cent, indicating that there is a high degree of confidence that materiel capability performance will be met. Evidence to support the estimated 99 per cent was not available during the review.

24. With the exception of the matters above, the Auditor-General concluded that ‘…nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2015–16 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.’

25. Additionally, in 2015–16, a number of administrative issues were observed in the course of the ANAO’s review, as summarised below:

  • non-compliance with corporate guidance resulting in inconsistent approaches taken to contingency allocation (Section 1 of the PDSS). See further explanation in paragraphs 1.36 to 1.40;
  • a lack of oversight, non-compliance with corporate guidance and the use of spreadsheets15 in the management of risks and issues (Section 5 of the PDSS). See further explanation in paragraphs 1.42 to 1.45;
  • outdated policy guidance for the project maturity framework16 (Section 6 of the PDSS). See further explanation in paragraphs 1.46 to 1.52; and
  • differences in the methodology applied in the cost per flying hour calculation reported in the Project Financial Assurance Statement in the ARH Tiger Helicopters PDSS (Section 1 of the PDSS) compared with previous evidence provided to the ANAO by Defence. See further explanation in paragraphs 1.53 to 1.56.

ANAO’s analysis of project performance

26. As discussed, the ANAO has undertaken an analysis of key elements of the Defence PDSSs—including cost, schedule, progress towards delivery of required capability, project maturity, risks and issues, and in particular, longitudinal analysis across these key elements of projects over time. Table 2, below, provides: summary data on Defence’s progress toward delivering the capabilities for the Major Projects covered in this report; and compares current data against that reported in previous editions of the MPR. This section also contains a summary analysis of the three principal components of project performance: cost, schedule and capability.

Table 2: Summary longitudinal analysis

 

2013–14 MPR

2014–15 MPR

2015–16 MPR

Number of Projects

30

25

26

Total Approved Budget

$59.4 billion

$60.5 billion

$62.7 billion

Total Expenditure Against Total Approved Budget

$28.9 billion (48.6 per cent)

$29.0 billion (48.0 per cent)

$29.4 billion (46.9 per cent)

Total In-year Expenditure Against In-year Budget

$3.3 billion (98.7 per cent)

$4.8 billion (96.8 per cent)

$3.9 billion (91.2 per cent)

Total Budget Variation since Second Pass Approval

$16.8 billion (28.3 per cent)

$18.5 billion (30.6 per cent)

$22.8 billion (36.3 per cent)

In-year Approved Budget Variation

$12.8 billion (21.5 per cent)

$2.9 billion (4.9 per cent)

$4.9 billion (7.8 per cent)

Total Schedule Slippage 1, 2

1 115 months (36 per cent)

768 months (28 per cent)

708 months (26 per cent)

Average Schedule Slippage per Project

38 months

31 months

28 months

In-year Schedule Slippage 3

205 months (7 per cent)

41 months (2 per cent)

42 months (1 per cent)

Total Project Maturity 4

1 726 / 2 100 (82 per cent)

1 401 / 1 750 (80 per cent)

1 479 / 1 820 (81 per cent)

Total Reported Risks and Issues 5, 6

129

129

123

Expected Capability (Defence Reporting)

High level of confidence of delivery (Green)

96 per cent

97 per cent

99 per cent

Under threat, considered manageable (Amber)

4 per cent

3 per cent

1 per cent

Unlikely to be met (Red)

0 per cent

0 per cent

0 per cent 7

       

Refer to paragraphs 26 to 42 in Part 1 of this report.

Note 1: The data for the 26 Major Projects in the 2015–16 MPR compares the data from projects in the 2014–15 MPR and 2013–14 MPR.

Note 2: Slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project.

Note 3: Based on the 26 repeat projects from the 2012–13 MPR, 23 repeat projects from the 2013–14 MPR and 23 repeat projects from the 2014–15 MPR respectively.

Note 4: The figures represent the total of the reported maturity scores for all 26 projects, divided by the total benchmark maturity score.

Note 5: The figures represent the combined number of open high and extreme risks and issues reported in the PDSSs across all projects. Risks and issues may be aggregated at a strategic level.

Note 6: The grey section of the table is excluded from the scope of the ANAO’s priority assurance review, due to a lack of systems from which to obtain complete and accurate evidence in a sufficiently timely manner to facilitate the review.

Note 7: Defence has advised that Joint Strike Fighter will not deliver one element of capability at FOC (which equates to approximately one per cent). However, across all 26 Major Projects this percentage rounds to zero per cent.

Cost

27. Cost management is an ongoing process in Defence’s administration of the Major Projects. While all projects reported that they could continue to operate within the total approved budget of $62.7 billion, AWD Ships received a Real Cost Increase17 of $1.2 billion in July 2015. In addition, ARH Tiger Helicopters was provided a heavily caveated Final Operational Capability (FOC) and has not yet delivered all of the capabilities required. Delivery of outstanding requirements is being transferred from acquisition to sustainment management within CASG, without a detailed costing of remaining acquisition elements being provided in the PDSS.

28. The approved budget for Major Projects included in this MPR has increased by $22.8 billion (36.3 per cent) since Second Pass Approval, as detailed in Table 3, below. However, as the MPR predominantly focusses on the approved capital budget, the ongoing costs of Project Offices (acquisition), training, replacement capability, etc., are not reported here.

Table 3: Budget variation post Second Pass Approval by variation type 1, 2

Project

Variation

Explanation

Year

Amount $b

MRH90 Helicopters

Scope increase/budget transfers

34 additional aircraft

2005–06

2.4

Bushmaster Vehicles

Scope increases

715 additional vehicles

2007–08, 2011–12 and 2012–13

0.8

Joint Strike Fighter

Scope increase

58 additional aircraft

2013–14

10.5

Overlander Medium/Heavy

Scope increase/budget transfers

Real Cost Increase

2013–14

0.7

AWD Ships

Real Cost Increase/budget transfers

Real Cost Increase of $1.2b offset by $0.1b transfer for facilities in 2014

2013–14 and 2015–16

1.1

P-8A Poseidon

Scope increase

Four additional aircraft

2015–16

1.3

Other

Scope increase/budget transfers (net)

Other scope changes and transfers

Various

(2.5)

 

Sub-total

14.3

 

Price Indexation – materials and labour (net) (to July 2010) 3

4.1

 

Exchange Variation – foreign exchange (net) (to 30 June 2016)

4.4

 

Total

22.8

         

Note 1: Variations greater than $500 million are included in this table. For the breakdown of in-year variation, refer to Table 9 of this report.

Note 2: The Government indicated in the [Defence] 2016 Integrated Investment Plan that Defence will commence spending some $500–$750 million remediating and upgrading ARH Tiger Helicopters in 2017.18

Note 3: Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.

Source: ANAO analysis of the 2015–16 PDSSs.

Schedule

29. Delivering Major Projects on schedule remains an ongoing challenge for Defence19; affecting when the capability is made available for operational release and deployment by the Australian Defence Force, as well as the cost of delivery.20

30. The total schedule slippage for the 26 selected Major Projects, as at 30 June 2016, is 708 months (2014–15: 768 months) when compared to the initial schedule. This represents a 26 per cent (2014–15: 28 per cent) increase since approval. Table 4 below includes details of in-year and total schedule slippage by project. While the table shows a one per cent in-year slippage for 2015–16, the removal of completed projects (Wedgetail and Battle Comm. Sys.) has removed 102 months of slippage. The reduction in total schedule slippage in 2015–16 therefore predominantly reflects projects with accumulated schedule slippage exiting the review.

Table 4: Schedule slippage from original planned Final Operational Capability

Project

In-year (months)

Total (months)

Joint Strike Fighter

0

2

AWD Ships

0

34

P-8A Poseidon

0

0

MRH90 Helicopters

0

60

Growler

0

0

MH-60R Seahawk

0

0

Overlander Medium/Heavy

3

5

LHD Ships

11

11

ARH Tiger Helicopters 1

3

82

Air to Air Refuel

2

64

Battlefield Airlifter 3

15

24

Bushmaster Vehicles

0

0

Overlander Light

0

9

Additional MRTT

0

0

ANZAC ASMD 2B

0

57

Additional Chinook

0

0

HATS

0

0

Battle Comm. Sys. (Land) 3

4

8

Maritime Comms

0

0

Collins RCS

0

109

Hw Torpedo

0

63

UHF SATCOM 2, 3

0

2

Collins R&S

0

99

ANZAC ASMD 2A

0

72

BMS 4

N/A

N/A

LHD Landing Craft

4

9

Total (months)

42

708

Total (per cent)

1

26

     

Note 1: FOC for ARH Tiger Helicopters was declared with caveats. That is, not all capabilities required by government were delivered by the project.

Note 2: UHF SATCOM has not reported any delays to FOC, but reports delays of 49 months to Final Materiel Release. See the UHF SATCOM PDSS in Part 3 of this report.

Note 3: These projects have been identified by Defence as continuing to underperform (see paragraph 1.15 in Part 1).

Note 4: BMS does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the 2015–16 PDSSs.

31. Platform availability has contributed to the slippage experienced within some projects. For example, the submarine programs have been impacted by changes to docking schedules, following government commissioned reviews. Significant delays have also been experienced by those projects with the most developmental content: AWD Ships, MRH90 Helicopters, ARH Tiger Helicopters, Air to Air Refuel and ANZAC ASMD 2B.

32. Table 5, below, provides details of total schedule slippage by project, for projects that have exited the MPR. Compared to the 708 months total schedule slippage for the current 26 Major Projects, the 13 projects which have exited the MPR have reported accumulated schedule slippage of 537 months, as at their respective exit dates. Again, schedule slippage was more pronounced in projects with the most developmental content.

Table 5: Schedule slippage for projects which have exited the MPR

Project

Total (months)

Wedgetail (Developmental)

78

Super Hornet (MOTS)

0

Hornet Upgrade (Australianised MOTS)

39

C-17 Heavy Airlift (MOTS)

0

FFG Upgrade (Developmental)

132

Next Gen Satellite 1 (MOTS)

0

HF Modernisation (Developmental)

147

Armidales (Australianised MOTS)

45

SM-2 Missile (Australianised MOTS)

26

155mm Howitzer (MOTS)

7

Stand Off Weapon (Australianised MOTS)

37

Battle Comm. Sys. (Australianised MOTS)

24

C-RAM (MOTS)

2

Total

 537

   

Note 1: Next Gen Satellite shows slippage in Figure 8, which related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced and slippage was reduced.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

33. Additional ANAO analysis (refer to Figure 7, on page 4) has compared project slippage against the Defence classification of projects as Military Off-The-Shelf (MOTS), Australianised MOTS or developmental.21 These classifications are a general indicator of the difficulty associated with the procurement process. This analysis highlights, prima facie, that the more developmental in nature a project is, the more likely it will result in project slippage, as well as demonstrating one of the advantages of selecting MOTS acquisitions.22

34. Figure 8 (on page 4) provides analysis of projects either completed, or removed from the MPR review, and shows that a focus on MOTS acquisitions has assisted in reducing schedule slippage. Figure 8 was requested by the JCPAA in May 2014.23

35. Longitudinal analysis indicates that while the reasons for schedule slippage vary, it primarily reflects the underestimation of both the scope and complexity of work, particularly for Australianised MOTS and developmental projects (see page 100 in Part 2).

Capability

36. The third principal component of project performance examined in this report is progress towards the delivery of capability required by government. While the assessment of expected capability delivery by Defence is outside the scope of the Auditor-General’s formal review conclusion, it is included in the analysis to provide an overall perspective of the three principal components of project performance.

37. The Defence PDSSs report that 25 projects in this year’s report will deliver all of their key capability requirements. Defence’s assessment indicates that some elements of the capability required may be ‘under threat’, but that the risk is ‘manageable’. The three project offices experiencing challenges with expected capability delivery (2014–15: one) are ARH Tiger Helicopters, Collins R&S and LHD Landing Craft. One project office (Joint Strike Fighter) is unable to deliver all of the required capability by FOC.

38. Defence’s presentation of capability delivery performance in the PDSSs is a forecast and therefore has an element of uncertainty. The ANAO has developed an additional measure of the status of current capability delivery progress to assist the Parliament—Capability Delivery Progress—which is a tally of the capability delivered as at 30 June 2016, as reported by Defence. Tables 6 and 7 below provide two worked examples of the ANAO’s methodology, utilising the performance information provided in the relevant PDSS.

Table 6: Capability Delivery Progress assessment – AWD Ships

Capability elements as per Section 4.2 of the PDSS

No. of elements approved

No. of elements delivered at 30 June 2016

Comments

All three Hobart Class Ship Systems with up to Category 5 (sea acceptance) trials, testing and certification completed.

3

0

No ships have been delivered to date.

Initial sustainment arrangements in place to support IOC.

1

0

Initial sustainment arrangements are not yet in place.

Training of the Hobart Class Systems for the commissioning crew to support IOC.

1

0

Training has not been completed

All sustainment arrangements in place to provide materiel support to the Hobart Class.

1

0

Final sustainment arrangements are not yet in place.

Total (number)

6

0

 

Total (per cent)

100

0

 

       

Source: PDSSs in published Major Projects Reports and ANAO analysis.

Table 7: Capability Delivery Progress assessment – Bushmaster Vehicles

Capability elements as per Section 4.2 of the PDSS

No. of elements approved

No. of elements delivered at 30 June 2016

Comments

Commencement of delivery of full rate production for Production Period 1 (PP1) vehicles.

1

1

All PP1 vehicles have been completed.

Completion of vehicle deliveries for all five production periods as detailed in Section 1.1. FMR is scheduled for September 2016.

1 015

1 007

Only eight vehicles were outstanding at 30 June 2016.

Total (number)

1 016

1 008

 

Total (per cent)

100

99

 

       

Source: PDSSs in published Major Projects Reports and ANAO analysis.

39. Table 8 below, summarises expected capability delivery as at 30 June 2016 – as reported by Defence and using the ANAO capability delivery progress measure.

Table 8: Capability delivery

Capability delivery

Note 1: Defence has advised that Joint Strike Fighter will not deliver one element of capability at FOC, of a total of 79 elements required for the project (which equates to approximately one per cent). However, across all 26 Major Projects this percentage rounds to zero.

Note 2: In addition, ARH Tiger Helicopters had six elements not delivered at FOC, of a total of 29 elements required for the project. However, as there is a total of 26 141 elements across all 26 Major Projects, these percentages round to zero.

Note 3: Excluding the five projects with the largest number of elements for delivery (i.e. Overlander Medium/Heavy, Bushmaster Vehicles, Overlander Light, Battle Comm. Sys. (Land), and BMS), results in an increase to the proportion of capability ‘not yet delivered’ to 54 per cent (from 33 per cent) and ‘not delivered at FOC’ to two per cent (from zero per cent). These five projects disproportionately weight the calculation of Capability Delivery Progress due to a large number of physical elements for delivery. These five projects represent 25 715 deliverables out of a total of 26 141 deliverables for all 26 Major Projects.

Source: PDSSs in published Major Projects Reports and ANAO analysis.

40. The ARH Tiger Helicopters platform was provided a heavily caveated FOC and Defence faces significant risks and issues in relation to delivering the remaining capabilities.24 It is also impacted by significant technological obsolescence, related to delays in delivery, which impact future use. The impact of these issues has not translated into Defence’s assessment of future capability delivery performance, although it could reasonably be assumed to have a long term effect on capability. Refer to paragraphs 17 to 24 for further detail.

41. Similarly, the results of trials for the LHD Landing Craft project25 highlight that there remain significant issues to be addressed prior to project conclusion. These issues are not disclosed in the Defence PDSS as significantly impacting on the delivery of expected capability. This is reflective of the optimism of Defence’s capability delivery assessments and reporting. Refer to paragraph 23 for further detail.

42. In addition to reporting on expected capability delivery, Defence has continued the practice of including declassified information on settlement actions for projects. Prior settlements for projects within this report related to MRH90 Helicopters, LHD Ships, ARH Tiger Helicopters, Air to Air Refuel and Maritime Comms. In 2015–16, two project offices also reported receiving goods and services as a result of liquidated damages. These are LHD Ships26 and Air to Air Refuel.27

1. The Major Projects Review

1.1 This chapter provides an overview of the review’s scope and approach, as implemented by the Australian National Audit Office (ANAO), for the review of the 26 Project Data Summary Sheets (PDSSs) prepared by the Department of Defence (Defence). This chapter also provides the results of the Major Projects Report (MPR) review.

Review scope and approach

1.2 In 2012 the Joint Committee of Public Accounts and Audit (JCPAA) identified the review of the PDSSs as a priority assurance review, under section 19A(5) of the Auditor-General Act 1997. This provided the ANAO with full access to the information gathering powers under the Act. The ANAO’s review of the individual project PDSSs, which are reproduced in Part 3 of this report, was conducted in accordance with the Australian Standard on Assurance Engagements (ASAE) 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Australian Auditing and Assurance Standards Board.

1.3 The following forecast information is excluded from the scope of the ANAO’s review: capability delivery, risks and issues, and forecast dates. These exclusions are due to the lack of Defence systems from which to provide complete and accurate evidence28, in a sufficiently timely manner to complete the review. Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.

1.4 The ANAO’s work is appropriate for the purpose of providing an Independent Assurance Report in accordance with ASAE 3000. However, the review of individual PDSSs is not as extensive as individual performance and financial statements audits conducted by the ANAO, in terms of the nature and scope of issues covered, and the extent to which evidence is required by the ANAO. Consequently, the level of assurance provided by this review, in relation to the 26 major Defence equipment acquisition projects (Major Projects), is less than that provided by the ANAO’s program of audits.29

1.5 Separately, the ANAO undertakes analysis of key elements of the PDSSs and examines systemic issues and provides longitudinal analysis for the 26 projects reviewed.

1.6 The review was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $2.0 million.

Review methodology

1.7 The ANAO’s review of the information presented in the individual PDSSs included examination and assessment of:

  • the governance and oversight in place to ensure appropriate project management;
  • an assessment of the systems and controls that support project financial management, risk management, and project status reporting, within Defence;
  • an examination of each PDSS and the documents and information relevant to them;
  • a review of relevant processes and procedures used by Defence in the preparation of the PDSSs;
  • interviews with persons responsible for the preparation of the PDSSs and management of the projects;
  • analysis of project information, for example, cost and schedule variances;
  • taking account of industry contractor comments provided on draft PDSS information;
  • assessing the assurance by Defence managers attesting to the accuracy and completeness of the PDSSs;
  • examination of the representations by the Chief Finance Officer supporting the project financial assurance and contingency statements, and the independent third-party assessment of the project financial assurance statements (commissioned by Defence);
  • examination of confirmations, provided by the Capability Managers, relating to each project’s progress toward Initial Materiel Release (IMR) and Final Materiel Release (FMR), and Initial Operational Capability (IOC) and Final Operational Capability (FOC); and
  • examination of the ‘Statement by the Secretary of Defence’, including significant events occurring post 30 June, and management representations by the Secretary of Defence.

1.8 The ANAO’s review of PDSSs also focused on project management and reporting arrangements contributing to the overall governance of Major Projects. These included:

  • developments in acquisition governance (Chapter 1 in Part 1, below);
  • the financial framework, particularly as it applies to the project financial assurance and contingency statements, and managing project budgets in the out-turned budget environment (Section 2 of the PDSSs);
  • schedule management and test and evaluation processes (Section 3 of the PDSSs);
  • capability assessments, including Defence statements of the likelihood of delivering key capabilities (Section 4 of the PDSSs);
  • the ongoing review of the implementation of the Enterprise Risk Management Framework, and major risk and issue data (Section 5 of the PDSSs); and
  • the project maturity framework and reporting and the systems in place to support the provision of this data (Section 6 of the PDSSs).

1.9 This review informed the ANAO’s understanding of the systems and processes supporting the PDSSs for the 2015–16 review period, and highlighted issues in those systems and processes that warrant attention.

Developments in acquisition governance

1.10 Consistent with previous years, key developments in acquisition governance processes are covered in the ANAO’s review in order to inform the planning process. As might be anticipated, while some initiatives are mature, others require further progress prior to achieving their intended impact.

Gate Review Boards

1.11 First introduced in 2008, the Gate Review (acquisition) process30 was designed to provide the Defence Senior Executive with assurance that all identified risks for a project are manageable, and that costs and schedule are likely to be under control prior to a project passing through the various stages of its life cycle. Eight years since inception, the Gate Review process continues to evolve, including the introduction of Gate Reviews for sustainment in 2013–14. The process is also set to introduce a contestability function, to focus on project business cases prior to government approval.

1.12 In July 2016, Defence has advised that Gate Reviews would henceforth be referred to as Independent Assurance Reviews31, and that corporate policies and procedures were being updated for the revised processes under development for the modified Capability Life Cycle. Twenty of the Gate Reviews conducted during 2015–16 related to 19 of the projects included in this report32, and formed key corroborative evidence for the ANAO’s review.33

Projects of Concern

1.13 First established in 2008, the Projects of Concern (PoC) process was implemented to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. The process has continued to play an important, although limited role, across the portfolio of MPR projects. The last PoC summit was in December 2015, involving the Minister and the contractors involved in the projects concerned. As at 30 June 2016, two MPR projects were continuing projects of concern:

  • AWD Ships, a project of concern since June 2014, due to increasing commercial, schedule and cost risks, including difficulties and delays in shipbuilding; and
  • MRH90 Helicopters, a project of concern since November 2011, due to technical issues preventing the achievement of milestones on schedule.
Quarterly Performance Report

1.14 The Quarterly Performance Report (QPR) introduced in 2014, aims to provide senior stakeholders within government and Defence with a clear and timely understanding of emerging risks and issues in the delivery of capability to the Australian Defence Force end-users.34 Defence has advised that the report is provided to the Minister for Defence and the Minister for Defence Industry on a quarterly basis, with future reports expected to cover the broader remit of the Capability Acquisition and Sustainment Group (CASG) deliverables, post First Principles Review implementation.

1.15 In 2015–16, further to the two MPR projects of concern noted above, the April to June 2016 QPR also identified three MPR projects for continuing underperformance35:

  • Battlefield Airlifter, due to contractor manufacturing changes arising from the US Air Force leaving the C-27J program (a risk known at Second Pass Approval), which has impacted the availability of aircraft for Royal Australian Air Force training and testing;
  • Battle Comm. Sys. (Land), due to the delays in achieving design acceptance, which has impacted on the forecast dates for FMR and FOC; and
  • UHF SATCOM, due to issues with the modification of Commercial Off-The-Shelf software (an element of the project now considered developmental).

1.16 The ongoing issues highlighted above for Battlefield Airlifter, Battle Comm. Sys. (Land) and UHF SATCOM align with the results of the ANAO’s review. Delays to progress have impacted the delivery schedule of the three projects (see Table 4, on page 4).

Joint Project Directives and Materiel Acquisition Agreements

1.17 The longstanding issue for Defence in maintaining complete and accurate records of government approvals for Major Projects, led to the introduction of Joint Project Directives (JPDs) (from March 2010).36 The implementation of JPDs is intended ‘to provide an appropriately declassified means of promulgating Cabinet and Ministerial approvals’.37 JPDs are used to inform internal documentation such as Materiel Acquisition Agreements (MAAs)38 between CASG and the Service Chiefs.39

1.18 However, the initiative started slowly, with Defence taking over two years to begin to produce the first JPDs.40 Further, JPDs are regularly finalised after the MAAs they are intended to inform and, as a result, care is required to ensure that JPDs properly reflect the relevant government decision, and that MAAs are appropriately aligned with the relevant JPD.41

1.19 Each of the 14 MPR projects approved from 1 March 2010, have completed a JPD.42 However, the ANAO requires access to original approval documents to validate the requirements of projects. At this time, validation by internal Defence documentation is not always possible.

1.20 The ANAO will continue to take JPDs into account in its review program in future years. However, the extent to which they can be relied upon will be dependent on the completeness and accuracy of JPDs, in relation to recording the detail of government approvals.

1.21 In July 2016 Defence advised that as part of a current review process, Program Delivery Agreements (PDAs) are being developed to replace the existing MAAs and Materiel Sustainment Agreements (MSAs). PDAs will be a higher level document (reviewed annually) that combine the MAA and MSA for each program.

Business systems rationalisation

1.22 Defence’s business systems rationalisation is aimed at consolidating processes and systems in order to provide a more manageable system environment.43 CASG has advised that it is making progress in this area. The Monthly Reporting System, which provides much of the data for the PDSSs, is to be replaced by the Project Status Review for acquisition, and the Sustainment Performance Management System for sustainment. Defence’s advice is that these are reporting tools and that the intention is to rely on interfaces with existing systems, such as Open Plan Professional (OPP – the scheduling tool), rather than create another ‘system’. As these applications will not be fully implemented until 2017–18, the ANAO will review the progress of their implementation during the next reporting period.

Results of the review

1.23 The following sections outline the results of the ANAO’s review, which inform the overall conclusion in the Independent Assurance Report by the Auditor-General for 2015–16.

Financial framework

1.24 The project financial assurance statement was introduced in the 2011–12 Major Projects Report and the contingency statements were introduced for the first time in the 2013–14 report. Together, they are aimed at providing greater transparency over projects’ financial status, following the move to out-turned budgeting in 2010. The contingency statement also reports on the use of contingency funding to mitigate project risks.

1.25 For the first time in 2014–15, the ANAO included the project financial assurance statements within the scope of the Independent Assurance Report by the Auditor-General. This reflected the increased level of confidence sought by stakeholders in the accuracy and completeness of the related disclosures.

1.26 Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.44 This change in supplementation policy has meant that price indexation can be an ongoing risk for projects, and is required to be managed individually, by each project office.

1.27 In effect, projects which slip past original delivery dates must now access contingency funding where pre-calculated indexation is insufficient. Previously, the separation and reporting of yearly indexation funding from other budget components allowed for greater transparency and fewer risks for project offices to manage.45

1.28 A project’s total approved budget comprises:

  • the programmed budget, which covers the project’s approved activities, for both contractual and limited departmental aspects of each project. The programmed budget includes amounts specified for the financial years in which they are to be spent as well as amounts which are as yet unallocated to specific years; and
  • the contingency budget, which is established to provide adequate budget to cover the inherent cost, schedule and technical risks involved in managing complex acquisitions.46

1.29 In 2015–16, the ANAO reviewed the financial framework as it applied to managing project budgets and expenditure, including contingency, in the out-turned budget environment, as well as the project financial assurance and contingency statements.

Project financial assurance statement

1.30 The project financial assurance statement was added to the PDSSs to provide readers with an articulation of a project’s financial position in relation to delivering project capability and to provide transparency in regard to the extent to which there is ‘sufficient remaining budget for the project to be completed’.47

1.31 In 2015–16 financial risks and their outcomes are as follows:

  • the AWD Ships project office48 was provided an additional $1.2 billion Real Cost Increase in July 2015;
  • the ARH Tiger Helicopters PDSS does not recognise any shortfall in project funding. The Government’s Defence Integrated Investment Plan released on 25 February 2016, acknowledges that ‘the Tiger has had a troubled history – essential upgrades are programmed to maintain the capability’s effectiveness’, and that starting in 2017 Defence plans to spend some $500-$750 million remediating and upgrading the Tiger49;
  • the Battlefield Airlifter project office disclosed a cost risk for contracts yet to be executed; and
  • the ANZAC ASMD 2B project office continued to recognise that available funding may be insufficient as contracted indices escalation may be greater than the approved project budget.

1.32 Defence has continued to subject a sample of project financial assurance statements to an independent third-party agreed-upon procedures engagement. The third party agreed-upon procedures engagement supports Defence in assessing the project financial assurance statements.

1.33 Projects selected for the 2015–16 third-party engagement, in support of the financial assurance statement assurance process, included:

  • additional procedures—ARH Tiger Helicopters and Battlefield Airlifter; and
  • standard procedures—Overlander Light, Collins RCS and LHD Landing Craft.

1.34 Defence advised that the third-party engagement ‘found no adverse factual findings that would indicate any issues with the [project financial assurance statements] PFAS for the five selected projects’. It also identified an exception against one project (ARH Tiger Helicopters), relating to the inability to ‘agree the budget values in the latest Project Plan/OPP [Open Plan Professional]50 to the budget values included in the most recent Additional Estimates process’.

1.35 In conclusion, for the 2015–16 Major Projects Report, the Chief Finance Officer’s representation letter to the Secretary on the project financial assurance statements was unqualified, noting the one adverse finding mentioned above. The project financial assurance statement is restricted to the current financial contractual obligations of Defence for these projects, including the result of settlement actions and the receipt of any liquidated damages; and current known risks and estimated future expenditure as at 30 June 2016.

Contingency statements and contingency management

1.36 The purpose of the project contingency budget is ‘to provide adequate budget to cover the inherent risk of the in-scope work of the project’.51 Defence policy requires project offices to maintain a contingency budget log to identify and track components of the contingency budget.52

1.37 PDSSs are required to include a statement regarding the application of contingency funds during the year, if applicable, as well as disclosing the risks mitigated by the application of those contingency funds. Defence’s Project Risk Management Manual (PRMM version 2.4, page 110) requires that contingency be applied for identified risk mitigation activities which have been assessed as being cost effective and representing value for money.

1.38 The seven project offices which had contingency funds applied in 2015–16 were AWD Ships (indexation funding shortfall), MRH90 Helicopters (crash rescue helicopter service, replacement Mission Management System, and establishment of support contracts), Overlander Light (Command Post Mobile module installation kit), ANZAC ASMD 2A and 2B (gain share and radar test set), Additional Chinook (Australian-based maintenance training system and crashworthy seating solution), and BMS (instructional support, vehicle movement for installation activities, and offer definition and improvement activities).

1.39 The ANAO’s examination of the contingency statements as at 30 June 2016 also highlighted that:

  • the clarity of the relationship between contingency application and identified risks has improved. Of the 25 project offices that have a formal contingency allocation53, only four projects (Growler, Bushmaster Vehicles, Additional Chinook and BMS) did not explicitly align their contingency log with their risk log, by including risk identification numbers as required by PRMM version 2.4;
  • the method for applying contingency varied, with 23 project offices using the ‘expected costs’ of the risk treatment (as required by PRMM version 2.4), and the remaining two (HATS and Maritime Comms) using a proportionate allocation of the likelihood of the risk eventuating (the method outlined in PRMM version 2.2); and
  • there were 12 project offices that did not meet all the requirements of PRMM version 2.4 in terms of keeping a record of review of contingency logs, however, the ANAO observed that the information required could be located in other documents.

1.40 Non-compliance with PRMM version 2.4 has resulted in inconsistent approaches taken to the management of contingency.

Enterprise Risk Management Framework

1.41 While major risks and issues data in the PDSSs remains out of scope of the Independent Assurance Report by the Auditor-General, the ANAO will continue to work with Defence to bring risks and issues into the scope of future MPR reviews. However, the following is provided for an overall perspective of how risks and issues are managed within the selected Major Projects.

1.42 In 2015–16, the ANAO again examined project offices’ risk and issue logs, which are created and maintained utilising spreadsheets and/or Predict! software.54 Overall, the issues with risk management that the ANAO observed related to:

  • variable compliance with corporate guidance, for example, three out of 26 Major Projects did not update their Risk Management Plan in line with PRMM version 2.4;
  • the visibility of risks and issues when a project is transitioning to sustainment;
  • the frequency with which risk and issue logs are reviewed to ensure risks and issues are appropriately managed in a timely manner, and accurately reported to senior management;
  • risk management logs and supporting documentation of variable quality, particularly where spreadsheets are being used55; and
  • lack of quality control resulting in inconsistent approaches in the recording of issues within Predict!.

1.43 Defence’s use of spreadsheets as a primary form of record for risk management, is a high risk approach. Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools, as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by who, and what change was made. Overall, of the 26 project offices, 15 utilise spreadsheets56 as their primary risk management tool and 11 utilise Predict!.

1.44 While some project offices will experience greater challenges with risks and issues administration—project complexity, scale and timing—it is important that Defence ensure that risk management systems and processes are used appropriately. This is particularly important for higher cost developmental projects.

1.45 In this context, the Joint Strike Fighter project is an example where the project has developed a hierarchical view of risks (due to the number), which are summarised in the project’s PDSS at a strategic level. As supported by the project’s Risk Management Plan (finalised July 2016), the project’s primary risk management tool is Predict! with strategic risks managed separately in spreadsheets.

Project maturity framework

1.46 Project Maturity Scores have been a feature of the Major Projects Report since its inception in 2007–08. The DMO Project Management Manual 2012, defined a maturity score as:

The quantification, in a simple and communicable manner, of the relative maturity of capital investment projects as they progress through the capability development and acquisition life cycle.57

1.47 Maturity scores are a composite indicator, cumulatively constructed through the assessment and summation of seven different attributes. The attributes are: Schedule, Cost, Requirement, Technical Understanding, Technical Difficulty, Commercial, and Operations and Support, which are assessed on a scale of one to 10.58 Comparing the maturity score against its expected life cycle gate benchmark provides internal and external stakeholders with a useful indication of a project’s progress.

1.48 The ANAO has previously raised inconsistency in the application of Project Maturity Scores as an issue. Instances of this have reduced in recent reviews and the ANAO considers that with the guidance available, assigning a maturity score is a repeatable process.

1.49 The policy guidance underpinning the attribution of maturity scores would benefit from a review for internal consistency and relationship to Defence’s contemporary business. For example, allocating approximately 50 per cent of the maturity score at Second Pass Approval, regardless of acquisition type, is often inconsistent with the proportion of project budget expended, and the remaining work required in order to deliver the project.

1.50 Further, the existing project maturity score model does not always effectively reflect a project’s progress during the often protracted build phase, particularly for developmental projects. During this phase it can be expected that maximum expenditure will occur, and that many risks will be realised, some of which will only emerge as test and evaluation activities are pursued through to acceptance into operational service. For example, the ARH Tiger Helicopters project has significant capability deficiencies and obsolescence issues at FOC, however the maturity score does not accurately represent the project’s maturity as at 30 June 2016. Refer to paragraphs 17 to 24 for further detail.

1.51 The policy guidance underpinning maturity scores was due for review in September 2012.59 Last year, the ANAO was advised that while work had occurred to review the guidance, the release of the First Principles Review meant that the guidance would require further consideration. This is due to occur once the redesign of the Capability Life Cycle, in line with the direction of the First Principles Review, has been completed.

1.52 The JCPAA has recently re-affirmed its interest in the maturity score framework, recommending ‘that the Department of Defence work with the Australian National Audit Office to review and revise Defence’s policy regarding Project Maturity Scores in time for the new approach to be implemented in the next Major Projects Report.’60 An updated policy has not yet been provided to the ANAO for input.

Cost per flying hour

1.53 Cost per flying hour is a metric widely used by military services for use in contract management and performance comparisons across aviation platforms. In the case of the ARH Tiger Helicopters information presented in the PDSS, there is no consistent calculation methodology to support the calculations provided.

1.54 The ARH Tiger Helicopters PDSS reports a cost per flying hour of $39 825 for 2013–14. Calculations previously provided by Defence for the purposes of an ANAO performance audit61 indicated a cost per flying hour of $43 026 for 2013–14. This information was provided by project office finance staff. Defence advised that the difference between the two figures ($9.7 million) equates to $3 201 per flying hour (when divided by the achieved rate of effort) and is predominantly comprised of Full Flight Mission Simulator upgrades ($6.7 million) and Electronic Warfare Mission Support System Technical Services ($1.3 million).

1.55 The ANAO also noted Defence advice to Airbus in May 2014 of a cost per flying hour of $41 000 for 2013–14.62 In a Sustainment Gate review held in March 2015 on the ARH Weapon System, ‘the Board recognised the CEO’s cost per flying hour calculation [based on dividing the annual sustainment budget by the actual flying hours] was a simple figure to emphasis[e] to Airbus the magnitude of the Commonwealth’s concern and to demand a new and much lower figure be achieved by 2017/18.’

1.56 Defence should define an appropriate calculation methodology that can be applied consistently to better support comparisons over time and across Defence’s aviation platforms.

2. Analysis of Projects’ Performance

2.1 Performance information is important in the management and delivery of major Defence equipment acquisition projects (Major Projects). It informs decisions about the allocation of resources, supports advice to government, and enables stakeholders to assess project progress.

2.2 Project performance has been the subject of many of the reviews of the Department of Defence (Defence), and a consistent area of focus of the Joint Committee of Public Accounts and Audit (JCPAA) since the first Major Projects Report (MPR). This chapter progresses previous Australian National Audit Office (ANAO) analysis over project performance.

Project performance analysis by the ANAO

2.3 The ANAO utilises three key performance indicators to analyse the major dimensions of projects’ progress and performance. These indicators are the:

  • percentage of budget expended (Budget Expended)—which measures the total expenditure as a percentage of the total current budget;
  • percentage of time elapsed (Time Elapsed)—which measures the percentage of time elapsed from original approval to the forecast Final Operational Capability (FOC)63; and
  • percentage of key materiel capabilities delivered64 (Capability Delivery Progress)—which measures the total capability elements delivered as a percentage of the total capability elements across all Major Projects.

2.4 The ANAO has previously utilised Defence’s prediction of expected final capability, as reported in Section 4.1 of each Project Data Summary Sheet (PDSS). For the first time this year, the ANAO has derived an indicator for ‘Capability Delivery Progress’, which aims to show the current capability delivered, in terms of capability elements included within the agreed Materiel Acquisition Agreements (MAAs). These performance indicators are measured in percentage terms, to enable comparisons between projects of differing scope, and to provide a view across the selected projects of progress and performance.

2.5 The following sections of this chapter provide analysis relating to the three principal components of project performance. This includes in-year information, longitudinal analysis and the results of project progress for the year-ended 30 June 2016. The first piece of analysis, in Figure 2 below, sets out each project’s Budget Expended and Time Elapsed.65

Figure 2: Budget Expended and Time Elapsed

Budget Expended and Time Elapsed

Note: BMS does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the 2015–16 PDSSs.

2.6 Figure 2 shows that for most projects (21 of 26), Budget Expended is broadly in line with, or lagging, Time Elapsed.66 This relationship is generally expected in an acquisition environment predominantly based on milestone payments. However, due to the varying complexity, stages and acquisition approaches across the portfolio of projects, further analysis of these simple performance measures is required to provide an overall picture of key variances.

2.7 Where Budget Expended is significantly lagging Time Elapsed the project schedule may be at risk, i.e. expenditure lags may indicate delays in milestone achievement. However, for the five projects where the Budget Expended is approximately 20 per cent less than the Time Elapsed, in only two cases (Overlander Medium/Heavy and LHD Landing Craft) this is the direct result of schedule delays. The remaining cases reflect either ongoing development or reduced costs, as detailed below:

  • Joint Strike Fighter (Budget Expended six per cent, Time Elapsed 47 per cent)—a large scope increase ($10.5 billion) for the purchase of additional aircraft was approved in April 2014, with the project yet to enter into main production contracts, as aircraft development continues;
  • Overlander Medium/Heavy (Budget Expended nine per cent, Time Elapsed 29 per cent, based on the current (LAND 121 Phase 3B) project approval)67—initial delays in contractor engagement have resulted in the project remaining in an early stage of delivery, reflected by the project continuing to undertake design reviews;
  • Additional Chinook (Budget Expended 66 per cent, Time Elapsed 92 per cent)—the variance reflects cost savings achieved through the integration of a number of previously post production modifications (including some of the Australian Defence Force (ADF) unique modifications) on the production line and progressive price reductions in the Foreign Military Sales (FMS) case;
  • Battle Comm. Sys. (Land) (Budget Expended 70 per cent, time elapsed 87 per cent)— the variance reflects the remaining budget as the project approaches completion, with Defence advising that the majority of these funds are to be re-allocated to other projects; and
  • LHD Landing Craft (Budget Expended 74 per cent, Time Elapsed 100 per cent)—the variance reflects the remaining budget as the project approaches completion. However, while the PDSS continued to report June 2016 as the FOC date, it also anticipates a delay to FOC due to an outstanding trial at 30 June 2016.

2.8 Where Budget Expended leads Time Elapsed the project budget may be at risk, i.e. expenditure increases may indicate real cost increases. However, for the four projects where Budget Expended leads Time Elapsed by more than 10 per cent, the actual reasons are related either to early procurement of major equipment, due to production timing, or schedule delays caused through platform availability, as detailed below:

  • Growler (Budget Expended 55 per cent, Time Elapsed 34 per cent)—most of the expenditure on equipment is in line with aircraft production in advance of training. All aircraft have now been manufactured, but are yet to be accepted by Defence;
  • MH-60R Seahawk (Budget Expended 54 per cent, Time Elapsed 40 per cent)—23 of 24 aircraft have been accepted as at June 2016. The variance is caused by the time between final aircraft delivery and FOC, with this period being used to test and integrate a number of ADF mission system options and modify Navy vessels to operate with the MH-60R Seahawk; and
  • Collins RCS (Budget Expended 97 per cent, Time Elapsed 84 per cent) and Collins R&S (Budget Expended 87 per cent, Time Elapsed 73 per cent) projects—most of the materiel has been acquired and expenditure undertaken. In addition, originally planned installation dates have been extended based on submarine availability, reducing the proportion of total Time Elapsed.

2.9 In each case, the performance information highlights projects requiring further attention. This is to ensure that surplus funds are returned to the Defence budget for re-allocation in a timely manner, the timing of key deliverables remains in focus, or planning focuses on bringing together all elements in a timely manner, as equipment is delivered.

Cost performance analysis

Sustainment reporting in the Major Projects Report

2.10 Historically, the majority of projects within the MPR have not been required to disclose significant detail in relation to sustainment activity to meet the requirements of the MPR Guidelines. However, the practice of providing caveated achievement of IOC or FOC provides for advancement through the process of acceptance into operational service, notwithstanding known shortcomings.

2.11 The practice of issuing caveated milestones will require Defence to exercise appropriate judgement for the capability disclosures within the MPR, in order to provide project PDSSs that provide an accurate depiction of performance to readers of the PDSS. Additionally, the ANAO will need to monitor and report on projects ‘in sustainment’, as projects complete tasks defined, and funded, for delivery in acquisition.

  • For example, the ARH Tiger Helicopters acquisition received caveated FOC and requires additional funding to address outstanding issues. The ANAO’s recent performance audit68 identified that the funding required to remediate the ARH Tiger Helicopters was beyond the scope of the already approved $2 033.0 million for the project. Expert analysis commissioned by Defence indicates that the issues arising from the developmental nature of the ARH Tiger Helicopter platform and sub-optimal sustainment arrangements will endure.69

2.12 The JCPAA has endorsed retention of the ARH Tiger Helicopters acquisition in the MPR for review in 2016–17.

Budget Expended and Project Maturity

2.13 Figure 3, below, sets out each project’s Budget Expended against Project Maturity70 and shows that Budget Expended lags Project Maturity for the majority of projects (20 of 26). This relationship is expected for two reasons:

  • in an acquisition environment predominantly based on milestone payments, projects will typically develop confidence in delivering their scope through testing and demonstration, ahead of formal acceptance of milestone achievement (and expenditure of budget); and
  • where there is a larger proportion of Military Off-The-Shelf (MOTS) projects. MOTS products are generally in-service with other military forces, and will generally have benefited from significant development and testing, prior to selection by Defence.

2.14 Budget Expended lags Project Maturity with a variance of 20 per cent or more in 10 projects. As expected, these projects are classified as either MOTS or Australianised MOTS, except Joint Strike Fighter, which is expected to be classified as MOTS by the time of aircraft delivery. There are no instances where Budget Expended leads Project Maturity by 20 per cent or more.

2.15 The variances are, in part, the result of Defence’s project maturity framework attributing approximately 50 per cent of total Project Maturity at Second Pass Approval (the main investment decision by government).71 This reduces the value of project maturity assessments during the early stages of acquisition.

2.16 Defence’s focus on typically lower risk MOTS acquisitions in recent years, has assisted in meeting schedule timelines across projects.72 Analysis of the available performance information highlights that the selection of MOTS projects assists in reducing risk during project acquisition, where Project Maturity is more advanced at Second Pass Approval than developmental projects.

Figure 3: Budget Expended and Project Maturity

Budget Expended and Project Maturity

Note: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Source: ANAO analysis of the 2015–16 PDSSs.

Second Pass Approval and 30 June 2016 approved budget

2.17 Figure 4, below, compares each project’s approved budget at Second Pass Approval and its approved budget at 30 June 2016.

2.18 The total budget for the 26 projects at 30 June 2016 was $62.7 billion, a net increase of $22.8 billion, when compared to the approved budget at Second Pass Approval of $39.9 billion (detailed analysis of this variance is included in Table 3).

2.19 Figure 4 indicates relative budget variations from Second Pass Approval of $500 million or more for six projects. The list below describes the components of these variations:

  • Joint Strike Fighter—increase of $14.0 billion, comprising $10.5 billion for 58 additional aircraft in 2013–14, $3.1 billion for exchange rate variation and $0.4 billion for price indexation;
  • AWD Ships—increase of $1.9 billion, comprising $1.2 billion for a Real Cost Increase73 in July 2015 to complete the project, $1.2 billion for price indexation, offset by a $0.3 billion decrease for exchange rate variation and a $0.1 billion decrease for transfers to facilities projects in 2013–1474;
  • P-8A Poseidon—increase of $1.9 billion, comprising $1.3 billion for four additional aircraft in 2015–16 and $0.6 billion for exchange rate variation;
  • MRH90 Helicopters—increase of $2.8 billion, comprising $2.4 billion for 34 additional aircraft in 2005–06 and other minor scope changes, $0.7 billion for price indexation, offset by a $0.2 billion decrease for exchange rate variation75;
  • Growler—increase of $888.4 million, comprising $955.7 million for exchange rate variation, $200.6 million in 2014–15 for the Mobile Threat Training Emitter System and weapons, offset in 2015–16 by a $267.9 million decrease for transfers to facilities projects and the return to the Defence budget of surplus funds for re-allocation; and
  • Bushmaster Vehicles—increase of $955.7 million, comprising $832.2 million for 715 additional vehicles in 2007–08 (437 vehicles), 2011–12 (70 vehicles) and 2012–13 (208 vehicles) and other minor scope changes, $124.6 million for price indexation, offset by a $1.1 million decrease for exchange rate variation.

Figure 4: Projects’ Second Pass Approval and 30 June 2016 approved budget ($m)

Projects' Second Pass Approval and 30 June 2016 approved budget ($m)

Note 1:  [vertical white-filled rectangle] indicates that the budget for the project at 30 June 2016 is less than the original budgeted cost. However, for Overlander Light this reflects a transfer of $2.2 billion to Overlander Medium/Heavy on separation of the original project into two phases in December 2011.

Note 2: The Second Pass Approval amount for the Overlander Medium/Heavy project includes a Real Cost Increase of $0.7 billion, which was provided as part of the revised Second Pass Approval in July 2013.

Note 3: On 22 May 2015, the Minister for Defence and the Minister for Finance announced there would be further delays to the delivery of the Air Warfare Destroyers and an additional $1.2 billion would be required to complete the project. The budget increase was incorporated into the approved project budget as at 30 June 2016.

Source: ANAO analysis of the 2015–16 PDSSs.

Budget performance

2.20 The following figures and tables illustrate the budget performance for the 26 selected projects by way of:

  • in-year budget variations by project (see Table 9); and
  • expenditure forecasting performance against actual expenditure for 2015–16 (see Figure 5).
In-year budget variance analysis

2.21 Table 9, below, sets out the in-year budget variations for each project. Overall, the approved budget for the projects as at 30 June 2016 increased by $4 923.1 million, or 8.8 per cent, compared to their approved budget as at 30 June 2015. This was driven by net real increases of $2 409.9 million, and exchange rate variation increases of $2 512.8 million.

2.22 Real Variations76 primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability and budgetary adjustments such as administrative savings decisions. In 2015–16, there were four projects with Real Variations:

  • AWD Ships—variation of $1 199.5 million reflecting the Real Cost Increase to complete the project77;
  • P-8A Poseidon—variation of $1 295.4 million reflecting the approval of an additional four aircraft;
  • Growler—variation of -$267.9 million reflecting the transfer of funding for facilities and the return to the Defence budget of surplus funds for re-allocation; and
  • Additional MRTT—variation of $182.9 million reflecting interim approval to incorporate an enhanced communications capability for long-range government transport.

2.23 Exchange rate variations can result from projects’ exposure to foreign currencies and movements in foreign exchange rates against the Australian dollar.78 Budget adjustments aim to maintain the relative buying power of the project budget. Movements in the US dollar and the Euro are the main influences. Projects with larger movements in foreign exchange in 2015–16 included:

  • Joint Strike Fighter—$1 557.2 million, or 10.3 per cent increase in budget;
  • P-8A Poseidon—$246.7 million, or 6.2 per cent increase in budget;
  • Growler—$293.0 million, or 8.3 per cent increase in budget; and
  • MH-60R Seahawk—$111.9 million, or 3.3 per cent increase in budget.

Table 9: In-year (2015–16) budget variations by project

Project

Approved Budget 2014–15 $m

Approved Budget 2015–16 $m

In-year Exchange Variation $m

In-year Real Variation $m

Total Variance $m

Total Variance (per cent)

Joint Strike Fighter 1

15 181.1

16 738.4

1 557.2

-

1 557.3

10.3

AWD Ships

7 891.1

9 120.8

30.2

1 199.5

1 229.7

15.6

P-8A Poseidon

3 977.8

5 519.9

246.7

1 295.4

1 542.1

38.8

MRH90 Helicopters

3 747.5

3 773.9

26.4

-

26.4

0.7

Growler

3 531.4

3 556.5

293.0

(267.9)

25.1

0.7

MH-60R Seahawk

3 408.5

3 520.4

111.9

-

111.9

3.3

Overlander Medium/Heavy

3 387.6

3 465.6

78.0

-

78.0

2.3

LHD Ships

3 091.0

3 092.9

1.9

-

1.9

0.1

ARH Tiger Helicopters

2 032.7

2 033.0

0.3

-

0.3

-

Air to Air Refuel

1 822.3

1 821.7

(0.6)

-

(0.6)

-

Battlefield Airlifter

1 369.2

1 434.5

65.3

-

65.3

4.8

Bushmaster Vehicles

1 250.5

1 250.7

0.2

-

0.2

-

Overlander Light

1 015.7

1 017.7

2.0

-

2.0

0.2

Additional MRTT

-

911.4

46.5

182.9

229.4

25.2

ANZAC ASMD 2B

678.6

678.6

-

-

-

-

Additional Chinook

633.8

642.4

8.6

-

8.6

1.4

HATS

-

487.6

13.7

-

13.7

2.8

Battle Comm. Sys. (Land)

461.9

464.6

2.7

-

2.7

0.6

Maritime Comms

442.1

456.0

13.9

-

13.9

3.1

Collins RCS

450.4

450.6

0.2

-

0.2

-

Hw Torpedo 1

427.9

429.7

1.7

-

1.8

0.4

UHF SATCOM 1

420.4

421.4

0.8

-

1.0

0.2

Collins R&S

411.7

411.7

-

-

-

-

ANZAC ASMD 2A

386.8

386.8

-

-

-

-

BMS

-

372.8

10.5

-

10.5

2.8

LHD Landing Craft

236.2

237.9

1.7

-

1.7

0.7

Total

56 256.2

62 697.5

2 512.8

2 409.9

4 923.1

8.8

             

Note 1: The Total Variance and components for this project do not add due to rounding differences.

Source: ANAO analysis of the 2014–15 and 2015–16 PDSSs.

In-year forecast and actual expenditure

2.24 Accurately forecasting and managing budget expenditure is an important element in the management of a portfolio of projects. Figure 5, below, sets out the expenditure forecasting performance of each project against actual expenditure in 2015–16. In total, actual expenditure for the 26 projects at 30 June 2016 was $3.9 billion, against an initial (PBS) forecast expenditure of $4.9 billion, a mid-year (PAES) forecast of $4.2 billion, and a final forecast of $4.3 billion (Final Plan, approved during May 2016). The main factors contributing to the variances were changes to delivery and payment schedules, and foreign exchange fluctuations.79

2.25 Figure 5 highlights that notable in-year underspends occurred in the following projects:

  • Joint Strike Fighter (expenditure of $248.9 million compared to $463.0 million PBS, $384.6 million PAES and $246.0 million Final Plan estimates)—the variance was primarily due to delays in United States Government contracting processes and the unpredictability of F-35 Joint Program Office invoicing;
  • AWD Ships (expenditure of $683.3 million compared to $745.5 million PBS, $715.6 million PAES and $696.1 million Final Plan estimates)—the variance was primarily due to delays, resulting in the reprogramming of Alliance and Program Management Office deliverables as part of the AWD Reform Program;
  • Growler (expenditure of $320.0 million compared to $889.8 million PBS, $419.5 million PAES and $555.0 million Final Plan estimates)—the variance was due to payments for Aircraft production and Airborne Electronic Attack Kits being brought forward to 2014–15 from 2015–16. In addition, there were lower March and no June 2016 FMS payments due to lower billing forecasts from the United States Government;
  • MH-60R Seahawk (expenditure of $282.2 million compared to $546.2 million PBS, $343.5 million PAES and $293.5 million Final Plan estimates)—the variance was due to payments for some MH-60R Seahawk helicopters being brought forward to 2014–15 from 2015–16. In addition, there was a reduction in the termination liability (deposit) required against the FMS case (funds released for use), and a reduction in expected expenditure based on updated United States Government billing forecasts;
  • LHD Ships (expenditure of $59.1 million compared to $145.7 million PBS, $100.2 million PAES and $65.2 million Final Plan estimates)—the variance was primarily due to the delay in production and testing milestones after ship acceptance; and
  • Battlefield Airlifter (expenditure of $108.7 million compared to $230.4 million PBS, $174.4 million PAES and $141.5 million Final Plan estimates)—the variance was due to delays associated with aircraft production, ordering of spare parts and contracting for training devices, including a flight simulator.

2.26 There was only one notable in-year overspend, relating to the P-8A Poseidon project (expenditure of $925.8 million compared to $717.3 million PBS, $766.3 million PAES and $931.5 million Final Plan estimates). The variance was due to the acceleration of FMS payments for the aircraft and the associated foreign exchange impacts due to earlier aircraft acquisition.

Figure 5: In-year (2015–16) projects’ forecast expenditure performance compared to actual expenditure ($m)

In-year (2015–16) projects' forecast expenditure performance compared to actual expenditure ($m)

Sources: ANAO analysis of the 2015–16 PDSSs and Defence Portfolio Budget Statements.

Schedule performance analysis

2.27 Defence data continues to show that schedule performance is a key issue in delivering and sustaining equipment.80 Project schedule slippage can effectively introduce or exacerbate an existing capability gap, or require an extension to the planned withdrawal date for those platforms being replaced.81 Additionally, projects which experience schedule slippage may result in platforms being delivered with significant obsolescence issues, for example ARH Tiger Helicopters.82

Time Elapsed and Project Maturity

2.28 Figure 6, below, sets out each project’s Time Elapsed against Project Maturity.83 Time Elapsed lags Project Maturity for 16 of 26 projects. The 16 projects are classified as either MOTS or Australianised MOTS, except Joint Strike Fighter, which is expected to be classified as MOTS by the time of aircraft delivery.

2.29 For the seven projects where Time Elapsed lags Project Maturity by 20 per cent or more, this reflects projects at an early stage of their acquisition and awaiting significant amounts of their major equipment to be constructed and delivered. The exception to this is MH-60R Seahawk, where the majority of equipment has been delivered but the project needs to test and integrate a number of ADF Mission System Options and modify Navy vessels to operate with the MH-60R Seahawk.

2.30 For the nine projects where Time Elapsed leads Project Maturity, there were no instances where this difference was significant (20 per cent or more).

Figure 6: Time Elapsed and Project Maturity

Time Elapsed and Project Maturity

Note 1: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Note 2: BMS does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the 2015–16 PDSSs.

Schedule slippage and acquisition type by approval date

2.31 Figure 7, below, illustrates the total schedule slippage since Second Pass Approval for the 26 selected projects. It also depicts the acquisition type and places projects in order of government approval. Figure 8 illustrates the total schedule slippage for the 13 projects that have exited the review.

2.32 Figures 7 and 8 show that the continued focus on MOTS and Australianised MOTS acquisitions is, prima facie, contributing to a reduction in schedule slippage in the Major Projects portfolio. However, it is not always possible to acquire the necessary capability in this manner, and decisions on whether to undertake developmental projects should be considered on a risk basis.

2.33 The 2008 Audit of the Defence Budget (Pappas Review) identified technical risk as the largest source of post Second Pass Approval schedule slippage for ‘post Kinnaird’ projects84, and also observed that schedule slippage causes cost escalation.85 The challenge of gaining a full understanding of the complexities of developmental aspects of projects at Second Pass Approval, is evident by the extent of slippage over time.

2.34 Figures 7 and 8 illustrate that older projects, which achieved Second Pass Approval prior to 2005, have experienced the most slippage. These projects tended to be more developmental (complex) in nature and typically experienced schedule slippage in the past, and have often continued to do so. This demonstrates an ongoing trend of slippage in historically late projects, which is more pronounced in older projects. This trend is also visible, but less prominent, in newer projects.

2.35 While it is not possible to predict the full extent of slippage a project will experience, this analysis has been provided to highlight changes since Kinnaird. Seven post Kinnaird and six pre Kinnaird projects have exited the MPR. Total slippage of the seven post Kinnaird projects is 5.7 years. Total slippage of the six pre Kinnaird projects is 38.9 years. Five of the seven post Kinnaird projects were MOTS acquisitions and all of the six pre Kinnaird acquisitions were Australianised MOTS or Developmental.

Figure 7: Current Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Current Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s analysis is Production Period Five.

Note 3: BMS is not included in this analysis as it does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Figure 8: Exited Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Exited Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: This does not include AIR 5376 Phase 3.2 Hornet Refurb, which exited in 2012, as this project did not introduce a new capability and did not have an FOC date.

Note 3: The slippage shown for Next Gen Satellite related to the final capability milestones at the time. By the time it reached FOC, a new final capability milestone had been introduced which reduced this slippage.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Schedule performance

2.36 The figures and tables that follow illustrate:

  • the original and 30 June 2016 forecasts for achieving FOC;
  • in-year schedule changes to achieving FOC;
  • total schedule slippage across the Major Projects; and
  • total slippage according to a project’s Second Pass Approval date.
Original and 30 June 2016 Final Operational Capability forecasts

2.37 Figure 9, below, presents information on the selected projects’ original and 30 June 2016 forecasts for achieving FOC. The total schedule slippage for the 26 Major Projects to date is 708 months compared to the initial prediction when approved by government. This represents a 26 per cent increase on the approved schedule.86 Of the 26 projects in the 2015–16 report, 16 have experienced schedule slippage.

2.38 Total schedule slippage across the Major Projects was 708 months in 2015–16. This is 60 months lower than the figure of 768 months reported in the 2014–15 report. The difference is due to projects with accumulated schedule slippage exiting the review at the end of 2014–15 (total of 102 months from Wedgetail and Battle Comm. Sys.), partially offset by in-year schedule slippage (42 months).

2.39 The reasons for schedule slippage often require a deep understanding of project technical elements and a realistic assessment of the capacity of the private sector to deliver in the expected timeframe. A project office’s ability to gain access to the platform for upgrading can also result in schedule delay (for example, the two Collins submarine projects and Hw Torpedo).87

2.40 A closer examination of the reasons for schedule slippage demonstrates the importance of initial assessments of project complexity. A key factor is whether a project is MOTS, Australianised MOTS or developmental.88 Two projects, MRH90 Helicopters89 and ARH Tiger Helicopters90, were originally misclassified as MOTS. The projects were reclassified by Defence to Australianised MOTS (i.e. more developmental) subsequent to Second Pass Approval. Both projects have experienced extended schedule slippage.

Figure 9: Projects’ original and 30 June 2016 FOC forecasts

Projects' original and 30 June 2016 FOC forecasts

Note 1: [vertical white-filled rectangle] indicates that the forecast FOC date for the project at 30 June 2016 is earlier than the original FOC date.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s Major Projects Report analysis is Production Period Five.

Note 3: BMS does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the 2015–16 PDSSs.

In-year schedule performance

2.41 In 2015–16, there was schedule slippage of 42 months in the forecast achievement of FOC across the 26 Major Projects. In-year project performance, measured by slippage over the last 12 months, may not reflect the project trend. However, Figure 10 below, shows recovery of previously reported slippage for one project, and that two projects anticipate delivery ahead of the original schedule:

  • Joint Strike Fighter—currently expects to achieve FOC in October 2023, two months ahead of the original schedule based on current assessments of aircraft delivery;
  • AWD Ships—currently expects to achieve FOC in December 2020, 31 months later than originally planned, but three months ahead of the 2014–15 forecast schedule. This is due to the delivery of the final two ships being brought forward under the AWD Reform program; and
  • HATS—currently expects to achieve FOC in September 2020, three months ahead of the original schedule.

2.42 In-year schedule slippage occurred for the following seven projects91 (the explanation provided, drawn from the 2015–16 PDSSs, may also include the reasons for prior slippage):

  • Overlander Medium/Heavy—there has been a delay of three months, however the project expects to achieve FOC in June 2023, six months ahead of the reapproved schedule92;
  • LHD Ships—the delay of 11 months is related to the availability of both ships to demonstrate their ability to conduct various operational scenarios;
  • ARH Tiger Helicopters—the slippage reflects the delayed sign off of the caveated FOC, which occurred in April 2016 rather than January 201693;
  • Air to Air Refuel—there was a delay to the approval of FOC due to the finalisation of activities from the Airworthiness Board meeting in March 2016;
  • Battlefield Airlifter—delays resulting from aircraft production setbacks, the acquisition of mature training system devices and delays to approvals for construction of facilities. In addition, government agreed to delay FOC to December 2019 and redefine FOC to exclude the flight simulator;
  • Battle Comm. Sys. (Land)—the delay of four months is due to the need to clarify the forecast dates for Final Materiel Release and FOC after Full Design Acceptance is achieved; and
  • LHD Landing Craft—further delays resulting from the need to reschedule test and evaluation trials to the second half of 2016.

Figure 10: In-year (2015–16) schedule changes to achieving FOC

In-year (2015–16) schedule changes to achieving FOC

Note: Defence’s PDSSs indicate that 15 of the 26 Major Projects Report projects did not record changes to their Final Operational Capability dates this year.

Source: ANAO analysis of the 2015–16 PDSSs.

Longitudinal schedule performance

2.43 Figure 11, below, shows the accumulated schedule slippage over time of the Major Projects included in the MPR reports from 2007–08 to 2015–16.94 Table 10 provides the details of the specific projects included in the analysis. The figure shows that 27.0 per cent (15.9 years or 191 months) of the total schedule slippage across the Major Projects covered in the 2015–16 report (59.0 years or 708 months) is made up of the slippage from the three remaining projects reported in the 2007–08 report.

2.44 Further disaggregation according to a project’s Second Pass Approval date in Table 11, on page 4, shows that 69 per cent (2014–15: 73 per cent) of the total schedule slippage across the 2015–16 Major Projects is made up of projects approved prior to July 2005.

Figure 11: Longitudinal schedule slippage across years for projects in the 2015–16 MPR (in years)

Longitudinal schedule slippage across years for projects in the 2015–16 MPR (in years)

Note 1: The total schedule slippage in 2015–16 across the 26 projects is 708 months. Additional MRTT and HATS, which are new to this year’s Major Projects Report, have not experienced schedule slippage against FOC according to Defence’s PDSSs. BMS, also new to this year’s Major Projects Report, does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s analysis is Production Period Five.

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Table 10: Projects included in Figure 11 analysis by Major Projects Report

Project

2007–08

2008–09

2009–10

2010–11

2011–12

2012–13

2013–14

2014–15

2015–16

Joint Strike Fighter

 

 

 

AWD Ships

 

P-8A Poseidon

 

 

 

 

 

 

 

MRH90 Helicopters

 

Growler

 

 

 

 

 

 

MH-60R Seahawk

 

 

 

 

Overlander Medium/Heavy

 

 

 

 

 

 

LHD Ships

 

ARH Tiger Helicopters

Air to Air Refuel

 

Battlefield Airlifter

 

 

 

 

 

 

Bushmaster Vehicles

Overlander Light

 

 

Additional MRTT

 

 

 

 

 

 

 

 

ANZAC ASMD 2B

 

 

Additional Chinook

 

 

 

HATS

 

 

 

 

 

 

 

 

Battle Comm. Sys. (Land)

 

 

 

 

 

Maritime Comms

 

 

 

 

 

 

 

Collins RCS

Hw Torpedo

 

 

UHF SATCOM

 

 

 

Collins R&S

 

 

ANZAC ASMD 2A

 

 

BMS

 

 

 

 

 

 

 

 

LHD Landing Craft

 

 

 

 

 

 

                   

Source: ANAO analysis of the PDSSs in published Major Projects Reports.

Table 11: Project slippage by project approval

Project

No. of months between Approval and Original FOC date

No. of months between Approval and 30/6/16 FOC date

No. of months slippage between Original FOC and 30/6/16 FOC date

Projects Approved pre July 2005

ARH Tiger Helicopters

123

205

82

Air to Air Refuel

94

158

64

Bushmaster Vehicles

217

217

0

Collins RCS

88

197

109

Hw Torpedo

148

211

63

Collins R&S

165

260

99 1

ANZAC ASMD 2A

97

167

72 1

Sub Total – Projects Approved pre July 2005

932

1 415

489 1

Percentage of Total – Projects Approved pre July 2005

35%

42%

69%

Projects Approved post July 2005

Joint Strike Fighter

169

167

1

AWD Ships

131

162

34 1

P-8A Poseidon

71

71

0

MRH90 Helicopters

119

179

60

Growler

111

111

0

MH-60R Seahawk

150

150

0

Overlander Medium/Heavy

125

119

1

LHD Ships

113

124

11

Battlefield Airlifter

68

92

24

Overlander Light

54

58

1

Additional MRTT

33

33

0

ANZAC ASMD 2B

90

145

57 1

Additional Chinook

83

83

0

HATS

76

73

1

Battle Comm. Sys. (Land)

55

63

8

Maritime Comms

125

125

0

UHF SATCOM

111

111

0

BMS 2

N/A

N/A

N/A

LHD Landing Craft

53

57

1

Sub Total – Projects Approved post July 2005

1 737

1 923

219 1

Percentage of Total – Projects Approved post July 2005

65%

58%

31%

Total – All Projects With Slippage

2 669

3 338

708 1

       

Note 1: These figures do not add horizontally due to the exclusion of schedule reductions over the life of the project.

Note 2: BMS does not have IOC or FOC milestones. These are linked to Work Packages B-D which are yet to receive government approval. Work Package A is currently a work in progress.

Source: ANAO analysis of the 2015–16 PDSSs.

Capability performance analysis

2.45 Defence defines capability as the power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.95 An operational effect is achieved by combining the eight Fundamental Inputs to Capability: personnel; organisation; collective training; major systems; supplies; facilities and training areas; support; and command and management96, and undertaking designated operations.

2.46 In acquiring Defence platforms and systems, a range of documentation (including capability definition, operational concept, function and performance specification, and test concept documents) is developed, which establishes the detailed requirements/performance attributes to be achieved.

2.47 Since the 2009–10 MPR, capability reporting has been based on Defence’s prediction of the final capability that would be achieved on the basis of deliverables and/or activities completed. This assessment of capability performance (Expected Capability) was measured against the Materiel Release Milestones (MRMs) and Completion Criteria specified in each project’s Materiel Acquisition Agreement (MAA). As the ANAO has previously noted, this data involved ‘…making certain assumptions in forecasting achievements and is therefore subjective in approach…’.97

2.48 For example, for the LHD Landing Craft project, Defence has predicted and reported that 99 per cent of elements of capability have a ‘high level of confidence of delivery’. Defence consequently reflected a predominantly ‘green’ pie chart graphic in the PDSS. However, as reported to the JCPAA on 17 March 2016 during public hearings, trials to test ‘the ability to transport a M1A1 Main Battle Tank’ are required prior to the achievement of Final Operational Capability.98

  • Subsequent trials conducted in May 2016 were unsuccessful. Carrying the M1A1 on the LHD Landing Craft requires the operation of the craft in an overload state. In consideration of the unsuccessful trials, the PDSS depicts that one per cent of capability for the LHD Landing Craft is ‘under threat, considered manageable (Amber)’, based on a ‘professional assessment of what the residual risk to the delivery of capability is’. Empirical evidence to support the estimated 99 per cent was not available during the review.

2.49 Over time, the JCPAA has sought the use of a more robust measure of capability performance. For example, in JCPAA Report 442, the Committee recommended:

Recommendation 7:

To improve the robustness of capability performance information, that the Australian National Audit Office and Defence Materiel Organisation consult as necessary and propose amendments to Section 5.1 and 1.2 in the 2014-15 MPR Guidelines, to:

  • Apply a more objective method to assessing capability performance; and
  • Distinguish capability achieved from capability yet to be achieved, capability unlikely to be achieved, and capability exceeded.

ANAO and DMO should provide a specific proposal to the Committee preferably by the end of August 2014 in line with submission of the 2014-15 MPR Guidelines.99

2.50 Defence has not developed this measure, reporting that the difficulties relate to the varied nature of projects being managed, the inherent subjectivity of the content, and the lack of a system that tracks at a sufficient level of detail the progress of inputs to capability.100

2.51 Noting with concern the ANAO’s assessment that delivery of capability estimates were in some cases overly optimistic, the Committee recommended that Defence further review the procedure for development of expected capability estimates.101

2.52 Following this recommendation, Defence has not developed an alternative method of capability assessment which would enable an improvement in capability reporting in the MPR.

Modified method of capability reporting

2.53 In light of the above, the ANAO has developed a measure of key materiel capabilities delivered (Capability Delivery Progress). This presents a current assessment of the capability delivered which differs from Defence’s prediction of final capability. The information used in forming the ANAO’s assessment is primarily based on Section 4.2 of the PDSS, which sets out the capability elements required to achieve Initial Materiel Release and Final Materiel Release, combined with other information in the PDSS reporting the delivery of equipment/achievement of these requirements toward FOC.

2.54 Noting that a system of capability reporting with a robust methodology applicable to materiel acquisition does not exist within Defence, the information presented below is a more meaningful reflection of project progress than an end-state prediction.

Capability Delivery Progress and Project Maturity

2.55 Figure 12, at page 4, sets out each project’s Capability Delivery Progress against Project Maturity.102 It shows that Capability Delivery Progress lags Project Maturity for the majority of projects (19 of 26). This relationship is expected as projects will typically develop confidence in the ability to deliver their scope and capability through testing and demonstration of capability components (for example, design reviews and acceptance tests) prior to delivery of the majority of equipment.

2.56 Figure 12 also shows that Capability Delivery Progress lags Project Maturity by 20 per cent or more in 12 projects, and for nine of these, Capability Delivery Progress lags by 50 per cent or more.

Figure 12: Project snapshot—Capability Delivery Progress and Project Maturity

Project snapshot—Capability Delivery Progress and Project Maturity

Note: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Source: ANAO analysis of the 2015–16 PDSSs.

2.57 As noted in paragraph 2.15, Defence’s project maturity framework attributes approximately 50 per cent of total project maturity at Second Pass Approval.103 These differences further indicate that Defence’s project maturity framework is not appropriately structured to assign project maturity progress throughout the project life cycle, particularly within the acquisition phase, which is predominantly the longest and most expensive component.

2.58 Figure 12 also highlights a continuing issue with the level of specification of capability elements. For the projects that show little or no Capability Delivery Progress, this can be attributed to Defence’s high level description of requirements in the capability elements. This indicates that it would be worthwhile for Defence to undertake additional work to track project progress. In respect to the six projects that show no capability delivery at 30 June 2016, progress is as follows:

  • AWD Ships—the first ship has been launched in preparation for sea trials and the remaining two ships are in advanced stages of construction;
  • P-8A Poseidon—design reviews are complete and aircraft production is underway;
  • Growler—most design reviews have been completed and all aircraft have been produced, however all aircraft reside in the United States and are pending acceptance from the United States Government;
  • Additional MRTT—all design reviews have been completed and aircraft acquired, however, conversion activities and subsequent delivery are yet to be completed;
  • HATS—this project is at an early stage and continues to work through design reviews and deficiencies discovered during testing of the flight simulator; however three of the 15 training helicopters have been completed and are pending acceptance from the contractor; and
  • Maritime Comms—this project is progressing through design reviews prior to commencing ship installations.

2.59 Further, Figure 12 indicates that:

  • six projects are still to deliver any of their capability (refer to paragraph 2.58 for detail);
  • 15 projects are still to deliver part of their capability;
  • one project, Air to Air Refuel, has delivered all of its capability and has achieved FOC;
  • two projects, Overlander Light and Battle Comm. Sys. (Land), have delivered essentially all of their capability with only minor items remaining before achievement of FOC;
  • one project, Joint Strike Fighter, will not deliver one element at FOC (as advised by Defence); and
  • one project, ARH Tiger Helicopters, has not delivered all of its intended capability at FOC.

!Part 2. Defence Major Projects Report

Part 2. Defence Major Projects Report is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

!Part 3. Assurance by the Auditor-General and the Secretary of Defence

Assurance by the Auditor-General and the Secretary of Defence

PRIORITY ASSURANCE REVIEW – SECTION 19A(5) OF THE AUDITOR-GENERAL ACT 1997

Independent Assurance Report

Department of Defence Project Data Summary Sheets

To the President of the Senate
To the Speaker of the House of Representatives

Qualified Conclusion

Based on the procedures I have performed and the evidence I have obtained, except for the effects of the matters described in the Bases for Qualified Conclusion paragraphs, nothing has come to my attention that causes me to believe that the information in the 26 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2015–16 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.

The purpose of the Major Projects Report is to report on the performance of selected major Defence equipment acquisition projects (Major Projects), since Second Pass Approval, and associated sustainment activities (where applicable), managed by Defence.

I have undertaken a limited assurance engagement of the PDSSs, reporting on the status of the projects selected by the Joint Committee of Public Accounts and Audit, and the Statement by the Secretary of Defence, for the year-ended 30 June 2016. The following forecast information was excluded from the scope of this engagement:

  1. Section 1.2 Current Status—Materiel Capability Delivery Performance and Section 4.1 Measures of Materiel Capability Delivery Performance;
  2. Section 1.3 Project Context—Major Risks and Issues and Section 5 – Major Risks and Issues; and
  3. forecast dates where included in each PDSS.

The forecast information has not been included in the scope of the engagement, due to the lack of Defence systems from which to provide complete and accurate evidence, in a sufficiently timely manner to facilitate the review. Accordingly, my conclusion does not provide any assurance in relation to this forecast information. However, material inconsistencies identified in relation to the forecast information, are required to be considered in forming my conclusion.

Bases for Qualified Conclusion

The Guidelines define a project as the acquisition or upgrade of Specialist Military Equipment. The Guidelines provide that the scope of Defence reporting includes the performance of selected major equipment acquisitions and associated sustainment activities, where applicable.

The ARH Tiger Helicopters PDSS has not been prepared on the basis of the Guidelines in the following respects:

  1. The Project Financial Assurance Statement in Section 1.2 of the PDSS reports that sufficient funding is available to complete the acquisition project. The statement does not address the significant caveats, capability deficiencies and obsolescence issues identified in the declaration of Final Operational Capability, in April 2016. Additional funding for these elements would need to be provided separately to the acquisition project, the amount of which is unable to be quantified; and
  2. The project maturity score in Section 6.1 of the PDSS reports a total of 67 out of a maximum of 70 (95.7 per cent) at the end of the acquisition project. Noting the significant caveats, capability deficiencies and obsolescence issues at Final Operational Capability, this score does not accurately or completely represent the project’s maturity as at 30 June 2016.

In addition, the following material inconsistencies have been identified in the forecast information:

  1. Section 4.1 in the ARH Tiger Helicopters PDSS reports that materiel capability delivery performance is at 99.8 per cent, indicating that there is a high degree of confidence that materiel capability performance will be met. Expert analysis commissioned by Defence indicates that the program will remain incapable of fully meeting expectations relating to reliability, availability, maintainability and rate of effort; and
  2. Section 4.1 in the LHD Landing Craft PDSS reports that materiel capability delivery performance is at 99 per cent, indicating that there is a high degree of confidence that materiel capability performance will be met. Evidence to support the estimated 99 per cent was not available during the review.

Secretary’s Responsibility for the Project Data Summary Sheets

The Secretary of Defence is responsible for the preparation and presentation of the PDSSs for the 26 selected projects, and the Statement by the Secretary of Defence, in accordance with the Guidelines. This responsibility includes the design, implementation and maintenance of internal control relevant to the preparation of PDSSs that are free from material misstatement, whether due to fraud or error. The Guidelines provide that the PDSSs and supporting evidence, provided to the ANAO for review, are complete and accurate.

Auditor’s Responsibility

My responsibility is to express an independent limited assurance conclusion on the PDSSs and Statement by the Secretary of Defence, based on the procedures I have performed and the evidence I have obtained. I conducted the engagement in accordance with Standard on Assurance Engagements ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Auditing and Assurance Standards Board. ASAE 3000 requires that I comply with relevant ethical requirements and that I plan and perform my procedures to obtain limited assurance about whether the PDSSs and the Statement by the Secretary of Defence are prepared in all material respects in accordance with the Guidelines.

In a limited assurance engagement, the assurance practitioner performs procedures, primarily consisting of: making enquiries of managers and others within the entity, as appropriate; the examination of documentation; and the evaluation of the evidence obtained. The procedures selected depend on my judgement, including identifying areas where the risks of material misstatement are likely to arise.

The procedures performed in a limited assurance engagement vary in nature and timing from, and are less in extent than those performed for, a reasonable assurance engagement. Consequently the level of assurance obtained in a limited assurance engagement is lower than the assurance that would have been obtained had a reasonable assurance engagement been performed. Accordingly I do not express a reasonable assurance conclusion on whether the PDSSs and the Statement by the Secretary of Defence are prepared in all material respects in accordance with the Guidelines.

I believe that the evidence I have obtained is sufficient and appropriate to provide a basis for my qualified conclusion.

In accordance with Auditing Standard ASQC 1 Quality Control for Firms that Perform Audits and Reviews of Financial Reports and Other Financial Information, Other Assurance Engagements and Related Services Engagements the Australian National Audit Office maintains a comprehensive system of quality control including documented policies and procedures regarding compliance with ethical requirements, professional standards and applicable legal and regulatory requirements.

Independence

I have complied with the relevant ethical requirements relating to assurance engagements, which include independence and other requirements founded on fundamental principles of integrity, objectivity, professional competence and due care, confidentiality and professional behaviour.

Australian National Audit Office

Grant Hehir
Auditor-General
Canberra
16 February 2017

Statement by the Secretary of Defence

On 1 July 2015, the Defence Materiel Organisation transitioned to the newly established Capability Acquisition and Sustainment Group (CASG) within the Department of Defence as recommended by the Defence First Principles Review. CASG exists to manage the acquisition and sustainment of Defence materiel equipment to meet Government and Defence requirements.

The attached PDSSs for the 26 major projects included in this report have been prepared in accordance with the Guidelines developed by CASG in consultation with the Australian National Audit Office (ANAO) and endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).

In making this statement, I acknowledge the difference of view between Defence and the ANAO in relation to the AIR 87 Phase 2 - Armed Reconnaissance Helicopter (Tiger) and the JP2048 Phase 3 LHD Landing Craft (LLC).

Specifically, the ANAO has highlighted the following issues in relation to these PDSSs:

  • The ANAO is of the view that the AIR87 Phase 2 Tiger Project Financial Assurance Statement in Section 1.2 of the PDSS does not address caveats to the declaration of Final Operational Capability (FOC) in April 2016. The Defence PDSS reports that sufficient funding is available to complete to acquisition project. All materiel‐related matters are being addressed at no additional cost. Obsolescence issues will be funded either under the extant sustainment budget, or through the Tiger Capability Assurance Program.
  • The ANAO is of the view that the AIR87 Phase 2 Tiger project maturity score in Section 6.1 of the PDSS does not accurately represent the project’s maturity as at 30 June 2016. The Defence PDSS reports a total score of 67 out of a maximum of 70 at the end of the acquisition project. The MPR Guidelines detail that maturity scores should be developed in accordance with a referenced Defence Materiel Standard Procedure, and should correspond with the maturity score recorded in the June 2016 Monthly Reporting System Majors Master Data Report. The PDSS reflects this outcome. In relation to materiel delivery, the Guidelines detail that achievement of these milestones and completion criteria will be measured against the Project’s Materiel Acquisition Agreement (MAA) and/or Joint Project Directive. For the AIR87 Phase 2 Tiger, 99.8 per cent of materiel detailed in the MAA was delivered by the project. The remainder represents the budget allocated for a Deployable Aircraft Maintenance Rig that was not delivered by the project.
  • Notwithstanding that Section 4.1 is outside the scope of the formal ANAO review, the ANAO is of the view that the JP2048 Phase 3 LLC PDSS Section 4.1 ‐ Measures of Materiel Capability Performance pie graph is misstated. The Guidelines require that this graph represent a percentage breakdown of the Materiel Release Milestones and Completion Criteria, focusing on materiel capability delivery. Interface trials conducted during the contractor’s acceptance test activities successfully demonstrated the specification requirements for carriage of loads up to 65 tonnes, and the designer’s operational envelope has been certified by Lloyds Register. This represents completion of the materiel elements required under the MAA, and Final Materiel Release was achieved in December 2016. A nominal amber component has been included in the Measures of Materiel Capability Performance pie graph to acknowledge that the Navy LLC naval operational testing of the landing craft has not yet been completed. The trials relating to the carriage of Abrams Main Battle Tank are ongoing and are expected to be completed in 2017, leading to the declaration of FOC.

I also note that the preparation of the 2015‐16 AIR87 Phase 2 and JP2048 Phase 3 PDSSs is consistent with the approach undertaken for all previous Major Projects Reports.

I am confident that the PDSS for these projects are an accurate reflection of the acquisition of this capability as at 30 June 2016, and are compliant with the 2015-16 Major Projects Report Guidelines.

Project Status as at 30 June 2016

In my opinion, the Project Data Summary Sheets comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2016. In stating this opinion, and in agreement with the ANAO, I acknowledge that the following sections of each PDSS are not covered in the scope of the Auditor-General’s assessment:

  • Section 1.2 Materiel Capability Delivery Performance, Section 1.3 Project Context–Major Risks and Issues, Section 4.1 Measures of Materiel Capability Delivery Performance, Section 5 Major Risks and Issues; and
  • Future dates that are ‘forecasts’ and capability assessments regarding a project’s expected achievement of delivery schedules and capability where included in Sections 1 and 3 of each PDSS Project Data Summary Sheets.

Significant Events Occurring Post 30 June 2016

In stating this opinion, I acknowledge the following material events have occurred post 30 June 2016:

SEA 4000 Phase 3 Air Warfare Destroyer

The second of three destroyers (NUSHIP Brisbane) was formally named and launched in Australian waters for the first time in December 2016. A review of the Initial Operational Capability (IOC) date for NUSHIP Hobart, based on a better understanding of the certification process for the Aegis Combat System, has seen IOC move to Quarter 4 2018 vice June 2018.

AIR 7000 Phase 2 Poseidon P8-A

The project delivered the first aircraft and received the Certificate of Airworthiness in October 2016. The project is on schedule to meet the planned Final Materiel Release and Final Operational Capability dates for the original scope. With the approval of four additional aircraft to be delivered under this project, the schedule has been approved to extend Final Materiel Release and Final Operational Capability to early 2022 to accommodate the increased scope.

JP 2048 Phase 4AB Amphibious Ships (LHD)

The project has advised that Final Acceptance and Final Materiel Release have been delayed, and are now expected to be achieved in the second half of 2017 due to defect/deficiency close-out activities and milestones not yet completed by the contractor.

AIR 87 Phase 2 ARH Tiger Helicopters

The PDSS noted that as at 30 June 2016, the Electronic Warfare Self Protection System exhibits some deficiencies which will be rectified by industry by the end of 2016. As at end‐December 2016, all operational aircraft have completed upgrades to the electronic warfare systems to rectify these deficiencies.

AIR 8000 Phase 2 Battlefield Airlift- Caribou Replacement

The project achieved a C-27J Initial Operational Capability with caveats in December 2016. These caveats relate to some mission limitations to the Air Logistic Support and Airborne Operation capabilities and notes some Operational Test and Evaluation is yet to be completed for the Search and Rescue capability. Caveats intend to be lifted as a soon as possible; contingent upon further evidence from test and evaluation activities being available.

AIR 7403 Additional KC-30A Multi-role Tanker Transportation

On 3 August 16, Defence signed a contract with Airbus Defence and Space that will deliver a Government Transport and Communications capability.

LAND 116 Phase 3 Bushmaster Protected Mobility Vehicle

The project declared Final Materiel Release in October 2016 and achieved Final Operational Capability in January 2017.

LAND 121 Phase 3A Overlander Light Field Vehicles and Trailers

The project achieved Final Operational Capability with caveats in October 2016 as scheduled. These caveats will be fully resolved, and relate to final approval of external lift by CH-47 helicopters, and completion of training for G-Wagon Command Post Modules and G-Wagon Winches. The Air Transportation task is in progress with testing already complete, and a formal plan is being released to ensure completion of training requirements.

JP 2072 Phase 2A- Battlespace Communications System

In consultation with the Capability Manager, the project has advised that Final Materiel Release and Final Operational Capability will be delayed to March 2017 and June 2017 respectively.

SEA 1439 Phase 4A Collins Replacement Combat System

The project was awarded Operational Release for the Combat System 05 (CS05) baseline in July 2016.

SEA 1439 Phase 3 Collins Class Submarine Reliability and Sustainability

The release of Beyond Benchmark in October 2016, the follow-up review to the 2012 Coles Review, found a remarkable improvement in the capability to successfully manage the sustainment of the Collins Class submarines.

SEA 1442 Phase 4 Maritime Communications

Current re-planning of Anzac Mid-life Capability Assurance Program (AMCAP) will impact ship availability and will delay the SEA1442 Phase 4 scheduled Initial Operational Capability by six to twelve months as it is dependent on the completion of the first ship under the AMCAP process.

LAND 75 Phase 4 Battle Management System

The project has advised that Materiel Release Milestones two and three have been delayed, and are now expected to be achieved in January 2017 due to the availability of serviceable vehicles to undergo installation at the contractor’s premises. Final Materiel Release is forecast to be achieved as scheduled in June 2017.

JP 2048 Phase 3 Amphibious Watercraft Replacement

Head Navy Capability accepted the declaration of Final Materiel Release in December 2016. Final Operational Capability for the LHD Landing Craft capability is expected to be achieved in mid-2017. These platforms are part of the Amphibious Capability for which Navy is managing key test and evaluation activities to achieve Final Operational Capability.

Dennis Richardson
Secretary
Department of Defence
15 February 2017

!Part 4. JCPAA 2015–16 Major Projects Report Guidelines

The JCPAA 2015–16 Major Projects Report Guidelines part is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.

Footnotes

1 Joint Committee of Public Accounts and Audit, Report 458, Review of the 2014–15 Defence Materiel Organisation Major Projects Report, May 2016, Foreword, p. vii.

2 Defence describes CASG’s role as ‘purchases and maintains military equipment and supplies in the quantities and to the service levels that are required by Defence and approved by the Government’. Department of Defence, Defence Annual Report 2015–16, Volume One, Performance, governance and accountability, p. 14.

3 ibid.

4 ibid., Volume Two, Audited financial statements, p. 33.

5 The Minister for Defence Industry, the Hon. Christopher Pyne MP, National Defense Industrial Association Conference Speech, 7 October 2016.

6 The 2015–16 Major Projects Report Guidelines were endorsed by the JCPAA in October 2015 and are included in Part 4 of this report.

7 Based on information provided to the ANAO by the Directorate of Capital Investment Program, Department of Defence.

8 For example, Defence project risk management records can be managed in spreadsheets, where the risk to the completeness and accuracy of records is too high to be included within the scope of the review.

9 JCPAA Report 458, Review of the 2014–15 Defence Materiel Organisation Major Projects Report, May 2016, Recommendation 2, p. xii.

10 IMR and FMR are milestones that [Defence utilises to] mark the completion and release of acquisition project supplies required to support the achievement of Initial Operational Release and FOC respectively. They are defined in the MAA [Materiel Acquisition Agreement]. Department of Defence, Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, pp. B2–B3.

11 IOC and FOC are the points when the first or final subset of a capability system that can be operationally employed is realised. They are capability states endorsed at project approval at Second Pass, and reported as having been reached by the Capability Manager. ibid.

12 An acquisition project can be closed at Defence’s discretion.

13 The caveats, capability deficiencies and obsolescence issues were discussed in ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, pp. 25–33 and pp. 50–53.

14 Department of Defence, Houston Review into Army Aviation, April 2016.

15 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. See paragraph 1.43 for further detail.

16 The project maturity framework outlined in the Department of Defence, Defence Materiel Standard Procedure (Project Management), DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010, is a methodology used to quantify the maturity of projects as they progress through the acquisition life cycle.

17 Real Cost Increases [are] attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget: excluding ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; and ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors.

18 ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, p. 53.

19 See Defence’s analysis on page 90 in Part 2 of this report.

20 M Thomson, Australian Strategic Policy Institute, Indexation, inflation and the cost of defence projects, 25 June 2015, available from <http://www.aspistrategist.org.au/indexation-inflation-and-the-cost-of-defence-projects/> [accessed 9 September 2016].

21 See Table 2 in Part 2 of this report for Defence classifications.

22 Off-The-Shelf (OTS): Hardware or software that already exists, is in service with one or more other customers for an equivalent purpose and requires no, or minimal, change. Sometimes expressed as COTS (Commercial Off-The-Shelf) or MOTS (Military Off-The-Shelf). Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 8.

23 JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 5, p. 31.

24 Defence has subsequently advised that the caveat relating to the Electronic Warfare Self-Protection System has been remediated. FOC over the caveat remains pending from Chief of Army.

25 Relating to the ability of the LHD Landing Craft to carry loads as heavy as the Abrams tanks and other similar vehicles, as discussed at the JCPAA Hearing held 17 March 2016.

26 See the LHD Ships PDSS (Section 1.2 Current Status—Schedule Performance) in Part 3 of this report.

27 See the Air to Air Refuel PDSS (Section 1.2 Current Status—Cost Performance (In-year)) in Part 3 of this report.

28 Refer to footnote 8.

29 See ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, for further detailed information on this project.

30 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 13, pp. 15–16, found that while Defence had improved the effectiveness of the program, there remained opportunities for further improvement and rigour.

31 These reviews are not carried out within frameworks issued by the Australian Auditing and Assurance Standards Board.

32 Gate Reviews were conducted for: Joint Strike Fighter, AWD Ships, Poseidon, MRH90 Helicopters, Growler, MH-60R Seahawk, Overlander Medium/Heavy, LHD Ships, Air to Air Refuel, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Additional MRTT, HATS, Maritime Comms, UHF SATCOM, Collins R&S, BMS and LHD Landing Craft. Twelve projects had reviews scheduled for late 2016. LHD Landing Craft was also subject to a Sustainment review. Two projects (ANZAC ASMD 2B and ANZAC ASMD 2A) were not expected to undergo reviews in 2016.

33 The ANAO notes that Defence has not disclosed the six month delay to IMR for the AWD Ships project as recommended in the Gate Review held in December 2015, or the six to nine month delay to FMR for the MRH90 Helicopters project as assessed in the Gate Review held in December 2015.

34 Department of Defence, Quarterly Performance Report, April to June 2016, p. 6.

35 Defence has advised that underperforming projects are projects that do not necessarily require remediation, however, would benefit from closer attention from senior Defence management, to manage their unique challenges or opportunities more effectively.

36 Joint Project Directive: A project-specific directive issued by the Secretary of the Department of Defence and the Chief of [the] Defence Force to the nominated Capability Manager or Project Realisation Manager and other involved action addresses, detailing the basis of project approval and assigning overall responsibility, authority and accountability for realisation of the capability system to an in-service state. Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 6.

37 Senate Standing Committee on Foreign Affairs, Defence and Trade, Additional Estimates Hearing, 23 February 2011, Questions Taken On Notice, answer to Question W21.

38 Materiel Acquisition Agreement: An agreement that states in concise terms what services and products the Capability Acquisitions and Sustainment Group (as supplier) will deliver, for how much and when, in support of unapproved and approved Major and Minor Capital Equipment projects. Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 7.

39 For further information on Joint Project Directives see ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 11.1 to 11.54, pp. 219–232.

40 ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraph 11.53, p. 232.

41 ibid., paragraph 11.54, p. 232.

42 Joint Strike Fighter (Stage 2), P-8A Poseidon, Growler, MH-60R Seahawk, Overlander Medium/Heavy, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Additional MRTT, HATS, Battle Comm. Sys. (Land), Maritime Comms, BMS and LHD Landing Craft.

43 Business system weaknesses, such as project offices having inconsistent record keeping and methods of tracking project progress were highlighted by the Committee in JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, paragraph 3.116, p. 39.

44 Out-turning a project budget takes into account the planned increases in overall Defence spending due to inflationary pressures. JCPAA Report 429, Review of the 2010–11 Defence Materiel Organisation Major Projects Report, May 2012, Appendix C, p. 46.

45 M Thomson, Australian Strategic Policy Institute, Indexation, inflation and the cost of defence projects, 25 June 2015, available from <http://www.aspistrategist.org.au/indexation-inflation-and-the-cost-of-defence-projects/> [accessed 9 September 2016].

46 Department of Defence, Defence Materiel Manual Project, DMM (PROJ) 1-0-001, DMO Project Management Manual, April 2012, Chapter 7 – Cost Management, paragraph 7.1.5, p. 39.

47 JCPAA Report 436, Review of the 2011–12 Defence Materiel Organisation Major Projects Report, May 2013, paragraph 3.4, p. 14.

48 The AWD Ships project was also the subject of an ANAO performance audit. See ANAO Report No.22 2013–14, Air Warfare Destroyer Program, March 2014, for further detailed information on this project.

49 ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, p. 53.

50 Defence scheduling software.

51 Department of Defence, DMM (PROJ) 11-0-002, DMO Project Risk Management Manual 2013, July 2013, Chapter 9 – Management of Contingency Budgets in DMO Acquisition Projects, p. 108.

52 The manual requires that the Project Contingency Budget Log is kept up to date for the proper overall management of risk and that it is submitted for internal review at Additional and Budget estimates.

53 The Collins R&S project does not have a formal contingency allocation.

54 Predict! is a risk management tool used by Defence to manage risks and issues.

55 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error.

56 The 15 projects are: MRH90 Helicopters, Growler, MH-60R Seahawk, ARH Tiger Helicopters, Air to Air Refuel, Battlefield Airlifter, Additional MRTT, ANZAC ASMD 2B, Additional Chinook, HATS, Battle Comm. Sys. (Land), UHF SATCOM, Collins R&S, ANZAC ASMD 2A and BMS.

57 Department of Defence, DMM (PROJ) 1-0-001, DMO Project Management Manual 2012, April 2012, Glossary, p. 75.

58 See Appendix 3 in Part 2 of this report and footnote 16 for further detail.

59 Department of Defence, DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010, p. 9, with a stated 24 month review period.

60 JCPAA Report 458, Defence Major Projects Report (2014-15), May 2016, Recommendation 3, p. 50.

61 ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, p. 46.

62 This advice was provided in correspondence to Airbus Helicopters from the CEO DMO, 29 May 2014.

63 Refer to footnote 10 for the definition of IMR and FMR milestones, and footnote 11 for the definition of IOC and FOC milestones.

64 See paragraphs 2.47 to 2.54 for further explanation.

65 A project’s budgeted cost and schedule data is at 30 June 2016, and may differ from originally approved budgets and schedules.

66 A project’s budget expended is cash based. In cases where pre-payments have been made, but have not been expensed/amortised, cash paid by a project will be greater than the accrued expenditure.

67 Originally, the Medium/Heavy capability was to be acquired through the LAND 121 Phase 3 project, which was initially reported in the 2009–10 MPR. The LAND 121 Phase 3 project substantially sought to acquire what was later split into the two LAND 121 3A and LAND 121 3B projects. The original FOC date for the LAND 121 Phase 3 project was December 2019. The LAND 121 3B project now anticipates FOC occurring in June 2023, 3 years and 6 months later. Reporting in the MPR is based on the latter LAND 121 3B government project approval parameters, and does not include the delay that is contained in the LAND 121 Phase 3 to LAND 121 Phase 3B transition.

68 The caveats, capability deficiencies and obsolescence issues were discussed in ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, pp. 25–33 and pp. 50–53. See also paragraph 1.31 in Part 1 of this report.

69 Department of Defence, Houston Review into Army Aviation, April 2016.

70 The JCPAA has recommended that a capacity to publish Project Maturity Scores be maintained [by Defence] until they are no longer required by the JCPAA. JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 8, p. 39.

71 The JCPAA has recommended that Defence work with the ANAO to review and revise their policy regarding Project Maturity Scores. JCPAA Report 458, Defence Major Projects Report (2014–15), May 2016, pp. 49–50.

72 See paragraphs 2.31 to 2.35 and Figure 8 for more information. However, acceptable MOTS solutions may not always be available.

73 See Note 3 of Figure 4, below, for further information.

74 The components for this project do not add up to $1.9 billion due to rounding differences.

75 The components for this project do not add up to $2.8 billion due to rounding differences.

76 Real Variations include ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors; ‘Real Cost Increases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget; and ‘Real Cost Decreases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been handed back to the Defence portfolio.

77 See Note 3 of Figure 4, above, for further information.

78 Australian Government arrangements for foreign exchange variation involve ‘no win/no loss’ supplementation. As a matter of policy, unless specifically approved, individual entities are not permitted to ‘hedge’ against foreign exchange risk.

79 Foreign exchange movement in the US dollar was from 77 to 74 cents. Movement in the Euro was from 69 to 67 Euro cents.

80 See Table 2 in Part 1 of this report.

81 Extensions to planned withdrawal dates may involve additional costs relating to the maintenance and servicing of equipment.

82 See ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, for further detailed information on this project.

83 Refer to footnote 70 for more detail.

84 M Kinnaird, Defence Procurement Review 2003, August 2003.

85 G Pappas, Department of Defence, 2008 Audit of the Defence Budget, April 2009, p. 76.

86 In instances where a Major Project has multiple segments/capabilities with separate Final Operational Capability (FOC) dates, the ANAO has used the project’s current lead/main capability FOC for calculating schedule performance. Defence’s approach is to use the final FOC date for a project listed in the 2015–16 PDSSs. These approaches, both valid, led to a small difference in the calculated percentage by which the Major Projects’ total schedule has slipped for the 2015–16 MPR (ANAO—26 per cent; Defence—24.1 per cent).

87 See the Collins RCS, Hw Torpedo and Collins R&S PDSSs in Part 3 of this report.

88 ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 9.1 to 9.4, pp. 198–199.

89 Further information on MRH90 Helicopters can be found in ANAO Reports No.48 2008–09, Planning and Approval of Defence Major Capital Equipment Projects, June 2009, pp. 84, 90 and 133; No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, pp. 86–87 and pp. 130–133; and No.52 2013–14, Multi-Role Helicopter Program, June 2014.

90 Refer to footnote 82 for more detail.

91 In the Statement by the Secretary of Defence in Part 3 of this report, the Secretary also makes reference to additional information on achieved milestone dates for Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Collins RCS and LHD Landing Craft.

92 Refer to footnote 67 for more detail.

93 Refer to footnote 82 for more detail.

94 Tables 4 and 5, on pages 12 and 13 respectively, report on the slippage for each project that has been in the MPR since 2007–08.

95 Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, p. B1.

96 Source 1: Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Chapter 2, pp. 1–2. Source 2: Department of Defence, DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, paragraph 1, p. 1.

97 ANAO Report No.17 2010–11, 2009–10 Major Projects Report, November 2010, p. 35.

98Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 17 March 2016, p. 5.

99 JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, pp. 37–39.

100Source 1: Ms S McKinnie, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 27 February 2015, p. 10. Source 2: Mr H Dunstall, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 27 February 2015, p. 10. Source 3: The CEO DMO Mr Warren King advised the JCPAA that ‘Landing on a method to have an easily auditable statement of what the capability is that we have delivered is really a complex issue and still there is, I think, work to be done’. Commonwealth of Australia, JCPAA, Defence major projects report 2012–13, 20 March 2014, pp. 1–3.

101 JCPAA Report 458, Defence Major Projects Report (2014-15), May 2016, pp. 48–49.

102 Refer to footnote 70 for more detail.

103 Refer to footnote 71 for more detail.