The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects, as reflected in the Statement by the Secretary of Defence, and the Project Data Summary Sheets (PDSSs) prepared by Defence, in accordance with the Guidelines endorsed by the Joint Committee of Public Accounts and Audit.

!Part 1. ANAO Review and Analysis

Foreword

Defence’s Major Projects have continued to be the subject of considerable parliamentary and public interest throughout 2014–15, and are expected to remain so into the foreseeable future. With acquisition expenditure expected to nearly double over the period to 2018–19, and government investment in Defence acquisitions covering major land, sea and air platforms, there is also strong justification to maintain a high level of transparency and accountability over this activity.

Following the delisting of the Defence Materiel Organisation (DMO) from 1 July 2015, and consistent with the recommendations of the Government’s First Principles Review: Creating One Defence, the functions of the DMO were merged back into Defence under the new Capability Acquisition and Sustainment Group. The ongoing reporting of the status of Major Projects under the new Capability Acquisition and Sustainment Group is the subject of this, the eighth Major Projects Report.

The former DMO oversaw the introduction of a Military-Off-The-Shelf focused acquisition strategy for Major Projects, following the Defence Procurement Review 2003 (Kinnaird Review). This has resulted in an improvement in schedule performance over time, with current analysis showing that 73 per cent of the total schedule slippage across the Major Projects relates to projects approved prior to DMO’s demerger from Defence in 2005. It is important that Defence continue to pursue these improvements in project delivery.

For the first time, the 2014–15 Major Projects Report’s scope includes the project financial assurance statement, which provides readers with an articulation of each project’s financial position in relation to delivering project capability. The independent conclusion now provides assurance over these statements within each Project Data Summary Sheet.

The ongoing support of the Joint Committee of Public Accounts and Audit (JCPAA) remains important to the development of the Major Projects Report, providing guidance and insight through their considerations. Each year the JCPAA endorses the Guidelines for the review, and provides direction and recommendations to assist the development of future Major Projects Reports.

As previously, this year’s review continued the strong working relationship between the ANAO and Defence departmental staff. The three Defence service chiefs and the Chief Information Officer, and industry, also provided valuable input to the review.

Summary

Introduction

1. Due to the high cost and impact on the economy, contribution to national security and the challenges involved in completing them within budget, on time and to the required level of capability, major Defence equipment acquisition projects (Major Projects) have continued to be the subject of considerable parliamentary and public interest throughout 2014–15. In this respect, the proposed Defence White Paper is expected to set out the Government’s priorities for future capability investment over the next 20 years. This is expected to include investment of over $89 billion in the acquisition of new submarines, frigates, offshore patrol vessels and other specialist naval vessels, following the Government’s commitment to a ‘continuous build’ of surface warships in Australia.1 Additionally, the land vehicle fleet will be replaced, new and improved air capabilities will be delivered, and investments will be made in supporting infrastructure, personnel and information technology systems.2

2. The proposed Defence White Paper is to be released during the implementation of the recommendations from the Government’s First Principles Review: Creating One Defence (First Principles Review). Notably, as an outcome of this process, the former Defence Materiel Organisation (DMO) was delisted from 1 July 2015, and its functions merged back into the Department of Defence (Defence), under the new Capability Acquisition and Sustainment Group.3 Previously, the DMO had provided support to the Australian Defence Force’s (ADF) operations through the acquisition and sustainment of ADF capabilities4, expending some $5.3 billion on major and minor capital acquisition projects in 2014–15.5

3. While acquisition alone does not generate new capability for the ADF, the DMO performed a significant role in Defence acquisition. As such, the DMO was the focus of the Australian National Audit Office’s (ANAO’s) review work on Major Projects, including for the majority of this, the 2014–15 Major Projects Report (2014–15 MPR).

4. Following prescription as a Financial Management and Accountability Act 1997 agency in 2005, the DMO oversaw the introduction of a Military-Off-The-Shelf focused acquisition strategy, the analysis of which is depicted in Figures 8 and 9. The DMO also introduced a performance reporting regime in 2004, the Maturity Score Framework (discussed further at paragraphs 1.48 to 1.58), the data from which has been included within the analysis in the MPRs, to assist users in assessing the progress of projects over time. The DMO was responsible for acquisition risk management and financial management frameworks; assisted in the development of Materiel Acquisition (and Sustainment) Agreements, which progressed the level of governance in Defence acquisition; and introduced programs of professionalisation for project managers throughout Defence and Defence Industry.

5. The newly formed group will now manage the process of bringing new capabilities into service, including the Fundamental Inputs to Capability6, for example, the provision of personnel, training and command. The ANAO will continue to review Defence acquisition in the 2015–16 MPR, as the group assumes its acquisition responsibilities, and while progress on the implementation of the First Principles Review recommendations is ongoing.7

The 2014–15 Major Projects Report

6. This eighth report covers 25 of Defence’s Major Projects (2013–14: 30; 2012–13 and 2011–12: 29), and builds on the earlier work to improve the transparency of, and accountability for, the status of Major Projects. The Major Projects review is supported by the commitment of the Joint Committee of Public Accounts and Audit (JCPAA), ‘…to maximise transparency and accountability in the Defence acquisition process for Major Projects that have been managed by DMO and will continue to be managed by the Department of Defence in future.’8

7. The benefits of this report have been noted by a variety of stakeholders, including Ministers, Parliamentary Committee members, industry and the media.

8. The ANAO’s review of Major Projects is completed in conjunction with the regular program of performance and financial statement audits conducted within the Defence portfolio. While by its nature, the report is not as in depth as a performance audit, it provides an opportunity to analyse data across a consistent range of projects over time, and complements the ANAO’s other Defence auditing and assurance functions.

2014–15 Major Projects selected for review

9. Projects are selected based on criteria included in the 2014–15 Major Projects Report Guidelines (the Guidelines), as endorsed by the JCPAA9, and provide a selection of the most significant Major Projects managed by Defence. The total approved budget for the Major Projects included in the report is approximately $60.5 billion, covering nearly 63 per cent of the budget within the Approved Major Capital Investment Program of $96.1 billion.10 The projects and their approved budgets are listed in Table 1, below.

Table 1: 2014–15 MPR projects and approved budgets at 30 June 2015

Project Number (Defence Capability Plan)

Project Name (on Defence advice)

Defence Abbreviation (on Defence advice)

Approved Budget $m

AIR 6000 Phase 2A/2B

New Air Combat Capability

Joint Strike Fighter

15 181.1

SEA 4000 Phase 3

Air Warfare Destroyer Build

AWD Ships

7 891.1

AIR 7000 Phase 2B

Maritime Patrol and Response Aircraft System

P-8A Poseidon1

3 977.8

AIR 5077 Phase 3

Airborne Early Warning and Control Aircraft

Wedgetail

3 893.2

AIR 9000 Phase 2/4/6

Multi-Role Helicopter

MRH90 Helicopters

3 747.5

AIR 5349 Phase 3

EA-18G Growler Airborne Electronic Attack Capability

Growler

3 531.4

AIR 9000 Phase 8

Future Naval Aviation Combat System Helicopter

MH-60R Seahawk

3 408.5

LAND 121 Phase 3B

Medium Heavy Capability, Field Vehicles, Modules and Trailers

Overlander Medium/Heavy

3 387.6

JP 2048 Phase 4A/4B

Amphibious Ships (LHD)

LHD Ships

3 091.0

AIR 87 Phase 2

Armed Reconnaissance Helicopter

ARH Tiger Helicopters

2 032.7

AIR 5402

Air to Air Refuelling Capability

Air to Air Refuel

1 822.3

AIR 8000 Phase 2

Battlefield Airlift – Caribou Replacement

Battlefield Airlifter

1 369.2

LAND 116 Phase 3

Bushmaster Protected Mobility Vehicle

Bushmaster Vehicles

1 250.5

LAND 121 Phase 3A

Field Vehicles and Trailers

Overlander Light

1 015.7

SEA 1448 Phase 2B

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2B

678.6

AIR 9000 Phase 5C

Additional Medium Lift Helicopters

Additional Chinook

633.8

JP 2072 Phase 2A

Battlespace Communications System

Battle Comm. Sys. (Land)

461.9

SEA 1439 Phase 4A

Collins Replacement Combat System

Collins RCS

450.4

SEA 1442 Phase 4

Maritime Communications Modernisation

Maritime Comms1

442.1

SEA 1429 Phase 2

Replacement Heavyweight Torpedo

Hw Torpedo

427.9

JP 2008 Phase 5A

Indian Ocean Region UHF SATCOM

UHF SATCOM

420.4

SEA 1439 Phase 3

Collins Class Submarine Reliability and Sustainability

Collins R&S

411.7

SEA 1448 Phase 2A

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2A

386.8

LAND 75 Phase 3.4

Battlefield Command Support System

Battle Comm. Sys.

313.0

JP 2048 Phase 3

Amphibious Watercraft Replacement

LHD Landing Craft

236.2

Total

60 462.4

Note 1: P-8A Poseidon and Maritime Comms are included in the MPR program for the first time in 2014–15.

Note 2: Once a project is selected for review, it remains within the portfolio of projects under review until the JCPAA endorses its removal, normally once it has met the capability requirements of the Australian Defence Force.

Source: See the Project Data Summary Sheets in Part 3 of this report.

Report objective and structure

10. The objective of this report is to provide the Auditor-General’s independent assurance over the status of the selected Major Projects, as reflected in the Statement by the Secretary of Defence, and the Project Data Summary Sheets (PDSSs), prepared by Defence. Assurance from the ANAO’s review of the preparation of the PDSSs is conveyed in the Independent Review Report by the Auditor-General, prepared pursuant to the endorsed Guidelines, and included in Part 3 of this report (pp. 141–144).

11. For the first time in 2014–15, the review’s scope includes the project financial assurance statement within each PDSS, which was first introduced in the 2011–12 review. The Independent Review Report by the Auditor-General now provides assurance over these statements within each PDSS.

12. Excluded from the scope of the ANAO’s review is PDSS data on the identification of Risks and Issues, the Measures of Materiel Capability Delivery Performance, and ‘forecasts’ of future dates and the achievement of future outcomes. Accordingly, the Independent Review Report by the Auditor-General does not provide any assurance in relation to this information.

13. It is intended that all components of the PDSSs will be included within the ANAO’s scope of review, once Defence and ANAO work programs can facilitate the review and conclusion over all of the components of the PDSSs. However, the exclusions to the scope of the review noted above, are due to the lack of a system or systems from which to provide complete and accurate evidence11, in a sufficiently timely manner to facilitate the review. This has been an area of focus of the JCPAA over a number of years12, and will continue to be a part of the Defence and ANAO work program into the future.

14. The ANAO’s analysis on the three key elements of the PDSSs—cost, schedule and the progress towards delivery of required capability, and in particular, longitudinal analysis across these key elements of projects over time, are contained in Part 1 (pp. 1–80). The ANAO’s analysis over other elements of the PDSSs, for example, project maturity and elements excluded from the formal scope of the review, is also included in this part, to provide readers with a balanced perspective over all key acquisition elements.

15. Further insights and context by Defence on issues highlighted during the year are contained in Part 2 (pp. 81–138)—although not included within the scope of the review by the ANAO.

16. Part 4 includes the Guidelines endorsed by the JCPAA (pp. 451–479), which provide the criteria for the compilation of the PDSSs by Defence and the ANAO’s review. Figure 1, below, depicts the key parts of this report.

Figure 1: 2014–15 Report structure

Refer to paragraphs 10 to 16 in Part 1 of this report.

Note: To assist in conducting inter-report analysis, the presentation of data remains largely consistent and comparable with the 2013–14 MPR.

17. For each Major Project, a corresponding PDSS includes unclassified information on project performance, prepared by Defence in accordance with the Guidelines. Additionally, as projects appear in the MPR for multiple years, changes to the PDSS from the previous year are depicted in bold purple text.

18. Each PDSS comprises:

  • Project Header: including name; capability and acquisition type; approval dates; total approved and in-year budgets; stage; complexity; and image;
  • Section 1—Project Summary: including description; current status, financial assurance and contingency statement; context, including background, uniqueness, major risks and issues; and other current sub-projects;
  • Section 2—Financial Performance: including budgets and expenditure; variances; and major contracts in place (in addition to quantities delivered as at 30 June 2015);
  • Section 3—Schedule Performance: providing information on design development; test and evaluation; and forecasts and achievements against key project milestones including Initial Materiel Release (IMR), Final Materiel Release (FMR)13, Initial Operational Capability (IOC) and Final Operational Capability (FOC)14;
  • Section 4—Materiel Capability Delivery Performance: provides a summary of Defence’s assessment of its progress on delivering key capabilities and whether the milestones were achieved15;
  • Section 5—Major Risks and Issues: outlines the major risks and issues of the project and remedial actions undertaken for each;
  • Section 6—Project Maturity: provides a summary of the project maturity as defined by Defence and a comparison against the benchmark score;
  • Section 7—Lessons Learned: outlines the key lessons that have been learned at the project level (further information on lessons learned by Defence are included in Defence’s Appendix 3); and
  • Section 8—Project Line Management: details current project management responsibilities within Defence.

The role of the Joint Committee of Public Accounts and Audit

19. Influential in establishing the MPR, the JCPAA has taken an active role in the development and review of the MPR program. Each year, the Committee considers the draft Guidelines, incorporating the selection of projects for review, and provides the Committee’s views in relation to the Guidelines’ content and development, prior to their endorsement. Following endorsement by the Committee, the Guidelines provide the criteria for Defence’s preparation of the PDSSs and the ANAO’s review.16

20. The main changes to the 2014–15 MPR Guidelines follow on from the recommendations made by the JCPAA in Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014. Recommendations 2 and 7 from this report requested changes to the PDSS template to increase analysis and disclosure in relation to published estimates figures and 30 June actual expenditure17, and additional detail supporting the capability performance information. Implementation of these changes became effective in 2014–15.

21. Following the tabling of the 2013–14 MPR, the Committee published Report 448, Review of the 2013–14 Defence Materiel Organisation Major Projects Report, in May 2015. The JCPAA’s recommendation is set out below18:

Recommendation 1

The Committee recommends that the reformed Department of Defence continues to provide the same priority and appropriate resources to the Major Projects Report in the future as DMO have done in the past so that the achievements of the past eight years are not lost. The same level of effort should also apply to the future development of sustainment reporting.

22. As noted previously, the Committee’s recommendations contribute to the development of the MPR each year and the ANAO was advised that the formal response by Defence to this recommendation was provided to the Committee in late November 2015. At the time of preparing this report actioning of the recommendation is summarised below:

  • Recommendation 1—to progress the enhancement of Defence sustainment reporting, an in-camera briefing was provided by Defence in November 2015. This provided the JCPAA with the opportunity to seek answers to (classified) sustainment matters directly from the relevant Capability Managers, expanding on information already provided in publicly available documents including the Defence Capability Plan, Portfolio Budget Statements, the Defence Annual Report and the MPR. To support this process, the JCPAA requested a sustainment brief from the ANAO, which was provided on 7 October 2015, and discussed at a private briefing of the JCPAA on 22 October 2015.

23. While Defence sustainment19 projects are generally outside the scope of the review, the Collins R&S project (which is defined as a sustainment project by Defence) has been included within the review at the request of the JCPAA since 2009–10. In addition, while ARH Tiger Helicopters and Collins RCS have been transferred to ‘sustainment’ by Defence, they remain included within the 2014–15 review, following the endorsement of the 2014–15 Guidelines.

24. In 2014–15, the ANAO also completed two performance audits on the former DMO’s outcomes from its Programme 1.2: Management of Capability Sustainment. The performance audit examining the management of the disposal of specialist military equipment was tabled in February 201520, and the performance audit examining the contribution made by Materiel Sustainment Agreements to the effective sustainment of specialist military equipment, was tabled in April 2015.21

Overall outcomes

25. This eighth report continues to review four of the nine Defence Major Projects which were initially introduced in the 2007–08 MPR22, and has continued to introduce new projects up to the originally agreed maximum of 30 projects. In 2014–15, 25 projects are reviewed. Maintaining a stable portfolio of projects over time has facilitated transparency and accountability for performance relating to cost, schedule and progress towards delivering the key capabilities of Major Projects, and provides opportunities for further longitudinal and other analysis into the future.

The 2014–15 Major Projects review (Chapter 1)

26. Under section 19A(5) of the Auditor-General Act 1997, the ANAO has reviewed the PDSS data as a priority assurance review and presents the Independent Review Report by the Auditor-General. As part of this review, the ANAO assessed the progress of Defence in addressing previously raised issues in relation to the administration of Major Projects.

27. In 2014–15, issues were noted within the following areas of project management:

  • budget and project management, with acknowledgement by AWD Ships that the project did not have sufficient funds to complete delivery of the approved capability and required $1.2 billion in additional funding.23 See further explanation in paragraph 1.24;
  • price indexation and budget allocations, and inconsistency in the determination and recording of contingency funds (Section 1 of the PDSS). See further explanation in paragraphs 1.32 to 1.35;
  • variability in the interpretation of project progress towards delivering required capability (Section 4 of the PDSS). See further explanation in paragraph 2.60;
  • inconsistency in the recording and reporting of major risks and issues by project offices, and reporting within the mandated Predict! and Excel risk management systems24 (Section 5 of the PDSS). See further explanation in paragraphs 1.40 to 1.44; and
  • inconsistency in the application of the project maturity framework25 (although improved from 2013–14), which is weighted towards pre Second Pass Approval processes, reducing the ability to adequately indicate progress during the acquisition phase (Section 6 of the PDSS). See further explanation in paragraphs 1.48 to 1.58.

28. In 2014–15, the results of the ANAO’s priority assurance review of the 25 PDSSs, was that nothing has come to the attention of the ANAO that causes us to believe that the information and data in the PDSSs, within the scope of our review, has not been prepared, in all material respects, in accordance with the Guidelines.

Analysis of projects’ performance (Chapter 2)

29. The data reviewed in the PDSSs covers the three major dimensions of project performance: cost, schedule, and progress towards delivering the planned capability. Table 2 below, provides summary data on the Defence approved budget, schedule performance and progress toward delivering capabilities for the Major Projects covered in this report, and compares data against that reported in previous MPR editions.

30. A significant contributor to the reduction of slippage shown in the 2014–15 MPR (1 115 months to 768 months), is the removal of a number of projects which had not reached Final Operational Capability (FOC) by 30 June 2014. These projects are Hornet Upgrade (39 months), FFG Upgrade (132 months), HF Modernisation (147 months) and SM-2 Missile (26 months), totalling 344 months of the total net decrease shown of 347 months.

31. Of the above projects, Hornet Upgrade achieved FOC for Phase 2.3 in October 2014. The HACTS/HIP component, which will upgrade aircraft simulators, was transferred out of the project, and is currently due to achieve FOC in January 2017. In addition, SM-2 Missile, which was reported in the 2013–14 MPR with a forecast FOC date of February 2015, achieved FOC in June 2015. The FFG Upgrade and HF Modernisation projects have not achieved FOC as yet, with current forecast dates of March 2016 and December 2016 respectively.

Table 2: Summary longitudinal analysis

 

2012–13 MPR

2013–14 MPR

2014–15 MPR

Number of Projects

29

30

25

Total Approved Budget

$44.3 billion

$59.4 billion

$60.5 billion

Total Budget Variation since Second Pass Approval

$6.5 billion (14.7 per cent)

$16.8 billion (28.3 per cent)

$18.5 billion (30.6 per cent)

In-year Approved Budget Variation

-$1.5 billion (-3.4 per cent)

$12.8 billion (21.5 per cent)

$2.9 billion (4.9 per cent)

Total Schedule Slippage1,2

957 months (36 per cent)

1 115 months (36 per cent)

768 months (28 per cent)

Average Schedule Slippage per Project

35 months

38 months

31 months

In-year Schedule Slippage3

147 months (5 per cent)

205 months (7 per cent)

41 months (2 per cent)

Expected Capability4

High level of confidence of delivery (Green)

95 per cent

96 per cent

97 per cent

Under threat, considered manageable (Amber)

5 per cent

4 per cent

3 per cent

Unlikely to be met (Red)

0 per cent

0 per cent

0 per cent

Refer to paragraphs 29 to 44 in Part 1 of this report.

Note 1: The data for the 25 Major Projects in the 2014–15 MPR compares the data from projects in the 2013–14 MPR and 2012–13 MPR.

Note 2: Slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project.

Note 3: Based on the 29 projects from the 2011–12 MPR, 26 projects from the 2012–13 MPR and 23 projects from the 2013–14 MPR respectively.

Note 4: The grey section of the table is excluded from the scope of the ANAO’s priority assurance review. See further explanation in paragraph 12 in Part 1 of this report.

Cost

32. Within the review period, all projects except for AWD Ships, reported that they could continue to operate within the total approved budget of $60.5 billion.26 The joint Ministerial announcement on 22 May 2015, by the then Minister for Defence and Minister for Finance, stated that the AWD Ships project would require additional approved funding of $1.2 billion to deliver the required capability (approved July 2015).27

33. The total budget for Major Projects included in this MPR has increased by $18.5 billion (44.0 per cent) since Second Pass Approval. Refer to Table 3, below.

Table 3: Budget variation post Second Pass Approval by Variation type

Project

Variation

Explanation

Year

Amount $b

Joint Strike Fighter

Scope increase

58 additional aircraft

2013–14

10.5

 

MRH90 Helicopters

Scope increase/budget transfers

34 additional aircraft

2005–06

2.4

 

Overlander Medium/Heavy

Scope increase/budget transfers

General program supplementation

2013–14

0.7

 

Bushmaster Vehicles

Scope increase

715 additional vehicles

Various

0.8

 

Other

Scope increase/budget transfers (net)

Other scope changes and transfers

Various

(2.5)

 

 

Sub-total

 

11.9

Price Indexation – materials and labour (net)

 

5.1

Exchange Variation – foreign exchange (net)

 

1.5

 

Total

18.5

Note: Variations greater than $500 million are depicted in this table. For the breakdown of in-year variation, refer to Table 6 of this report.

Source: ANAO analysis of the 2014–15 PDSSs.

Schedule

34. Maintaining Major Projects on schedule remains an ongoing challenge for Defence28; in turn affecting when the capability is made available for operational release and deployment by the ADF, and increasing the cost to delivery.29 In the 2014–15 MPR, the total schedule slippage for the 25 Major Projects as at 30 June 2015 is 768 months (2013–14: 1 115 months) when compared to the initial schedule first approved by government. This represents a 28 per cent (2013–14: 36 per cent) increase on the originally approved schedule. Refer to Table 4, below for details of in-year and total schedule slippage by project, for projects in the 2014–15 MPR.

Table 4: Schedule slippage from original planned Final Operational Capability

Project

In-year

(months)

Total

(months)

Project

In-year

(months)

Total

(months)

Joint Strike Fighter

2

2

Overlander Light

0

9

AWD Ships

12

34

ANZAC ASMD 2B

0

57

P-8A Poseidon

0

0

Additional Chinook

0

0

Wedgetail

0

78

Battle Comm. Sys. (Land)

0

4

MRH90 Helicopters

0

60

Collins RCS

0

109

Growler

0

0

Maritime Comms

0

0

MH-60R Seahawk

0

0

Hw Torpedo

5

63

Overlander Medium/Heavy

2

2

UHF SATCOM

0

0

LHD Ships

0

0

Collins R&S

0

99

ARH Tiger Helicopters

0

79

ANZAC ASMD 2A

0

72

Air to Air Refuel

5

62

Battle Comm. Sys.

1

24

Battlefield Airlifter

9

9

LHD Landing Craft

5

5

Bushmaster Vehicles

0

0

 

 

 

Total

 

 

 

41

768

Source: ANAO analysis of the 2014–15 PDSSs.

35. While it should be noted that platform availability can contribute to the slippage within some projects, significant delays relate to those projects with the most developmental content. For example, AWD Ships, Wedgetail, MRH90 Helicopters and ANZAC ASMD 2B.

36. Additional ANAO analysis (refer to Figure 8) presents project slippage as reported in each of the eight reports against the Defence classification of projects as Military-Off-The-Shelf (MOTS), Australianised MOTS or developmental.30 These classifications are a general indicator of the difficulty associated with the procurement process. This figure highlights, prima facie, that the more developmental in nature a project is, the more likely it will result in project slippage, as well as demonstrating one of the advantages of selecting MOTS acquisitions.31 First included in the 2013–14 report, Figure 9 provides analysis of projects either completed, or removed from the review. While initial suggestions by Defence were that analysis in Figure 8 did not allow for future slippage in MOTS projects, Figure 9 shows that MOTS acquisitions assist in reducing schedule slippage.

37. It should also be noted that the impact of a MOTS focused acquisition strategy has received significant attention since the inception of the 2007–08 review and Figure 9 shows the benefit of collating data over a consistent portfolio of projects across time, to allow the longitudinal analysis first sought by the JCPAA. Further development and results in this area will be covered in the 2015–16 report.

38. The reasons for schedule slippage vary but primarily reflect the underestimation of both the scope and complexity of work, particularly for Australianised MOTS and developmental projects (see paragraphs 2.29 to 2.31 in Part 2).

Capability

39. The third major aspect of project performance examined in this report is progress towards the delivery of capability required by government and specified by the Australian Defence Force. Assessment of expected capability delivery by Defence is outside the scope of the Auditor-General’s formal review conclusion, but is included in the analysis to provide an overall perspective of the three major components of project performance.

40. The Defence PDSSs reflect that the 25 projects in this year’s report will deliver all of their key capability requirements, recognising that some elements of the capability required may be under threat, but are considered manageable (assessed as either green or amber). This is consistent with the 2013–14 presentation, and reflects only one project office currently having significant challenges (2013–14: five). Refer to Table 5, below.

Table 5: Longitudinal Expected Capability Delivery

Expected Capability

2012–13 Major Projects Report

%

2013–14 Major Projects Report

%

2014–15 Major Projects Report

%

High Confidence (Green)

All Projects

95

All Projects

96

All Projects

97

Under Threat, considered manageable (Amber)

Joint Strike Fighter (Developmental)

Wedgetail (Developmental)

MRH90 Helicopters (AMOTS)

Air to Air Refuel (Developmental)

FFG Upgrade (Developmental)

155mm Howitzer (MOTS)1

5

Joint Strike Fighter (Developmental)

Wedgetail (Developmental)

MRH90 Helicopters (AMOTS)

Air to Air Refuel (Developmental)

FFG Upgrade (Developmental)2

4

Joint Strike Fighter (Developmental)

3

Unlikely

(Red)

 

0

 

0

 

0

Total

 

100

 

100

 

100

Note 1: 155mm Howitzer was removed from the MPR in 2014–15. However, the Course Correcting Fuze element of the project has been transferred to another project and will be delivered by LAND 17 Phase 1C1 Lightweight Howitzer.

Note 2: FFG Upgrade was removed from the Major Projects Report in 2014–15 prior to achieving Final Operational Capability.

Source: PDSSs in published Major Projects Reports.

41. However, the results of analysis by the ANAO show that some project managers may have taken different perspectives in assessing future achievements in relation to delivering final capability. For example, the ARH Tiger Helicopters project, faces significant capability risks and issues in relation to delivering the required Rate of Effort (flying hours), and technological obsolescence caused by delays in delivery, which impact future use. The expected impact of these risks and issues has not translated into Defence’s assessment of future capability performance, although it could reasonably be assumed to have a long term capability effect.

42. Similarly, the initial results of testing for the LHD Landing Craft project, highlight issues of significance to be addressed prior to project conclusion, not disclosed as impacting expected capability delivery. This is reflective of reporting against arrangements, post government approval, where responsibilities have been allocated within the acquisition and capability delivery framework, to the Chief of Navy post FMR (see also paragraph 2.60).

43. This year, as reported by Defence, the delivery of only three per cent (2013–14: four per cent) of the key capabilities is considered to be under threat but manageable, which as noted above, may be overly optimistic. The project with some elements under threat but considered manageable is Joint Strike Fighter. Further details are outlined at paragraph 2.65.

44. In addition to expected capability delivery, Defence has continued the practice of including declassified information on settlement actions for projects, in the interests of providing greater transparency to readers of this report. As well as prior settlements for projects included in this report32, in 2014–15, five project offices reported receiving goods and services as a result of liquidated damages. These are LHD Ships33, ARH Tiger Helicopters34, Air to Air Refuel35, Maritime Comms36 and Battle Comm. Sys.37

Developments in acquisition governance (Chapter 3)

45. Consistent with previous years, developments in acquisition governance processes are covered in the ANAO’s review, with general indications of positive impacts from some of the more recent initiatives. As might be anticipated, while some initiatives continue to mature, others require further progress prior to achieving their intended impact. These developments broadly relate to the following key acquisition governance areas.

Gate Review Boards

46. First introduced in 2008, the Gate Review (acquisition) process38 was designed to provide the Senior Executive with assurance that all identified risks for a project are manageable, and that costs and schedule are likely to be under control prior to a project passing various stages of its life cycle. The Gate Review process continues to evolve, with additional types of Gate Reviews introduced in 2013–14, as fewer acquisition projects will be eligible for (acquisition) Gate Reviews, and more projects are transferred into sustainment. These include Project Initiation Gate Reviews, Acquisition and Support Concept Gate Reviews (pre First Pass), and Sustainment Gate Reviews. The introduction of Sustainment Gate Reviews in particular, provides continuing oversight of sustainment capabilities, which was previously not available.

Projects of Concern

47. First established in 2008, the Projects of Concern process was implemented to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. The process has continued to play an important, although limited role, across the portfolio of MPR projects. As at 30 June 2015, two MPR projects were continuing projects of concern: AWD Ships and MRH90 Helicopters. The Air to Air Refuel project was removed from the Projects of Concern list in March 2015.

Quarterly Project Performance Report

48. The Quarterly Project Performance Report (QPPR) was implemented in 2014 to replace the Early Indicators and Warnings system (first announced in May 2011), to identify problems with projects in the formative stages of the life cycle. The QPPR aims to provide senior stakeholders within government and Defence with a clear and timely understanding of key risks facing projects, and their progress against cost, schedule and capability performance. In 2014–15, the QPPR identified three MPR projects for continuing underperformance: Battlefield Airlifter, Overlander Light and UHF SATCOM. The Overlander Light project was removed from the list of underperforming projects in the April–June 2015 QPPR. The ongoing issues highlighted for Battlefield Airlifter and UHF SATCOM align with the results of the ANAO’s review, where delays to progress have impacted the delivery schedule of the two projects.

Joint Project Directives

49. There has been a longstanding issue for Defence in maintaining complete and accurate records of government approvals for Major Projects39, which led to the development of Joint Project Directives (JPDs). The implementation of JPDs is intended to provide an appropriately declassified means of promulgating, accurately and completely, the details of project approvals by government.

50. The introduction of a requirement for JPDs40 in 2009–10, for all projects approved by government from March 2010, is maturing and expected to have a greater influence over the portfolio of MPR projects in future years. To date, all of the eleven MPR projects, approved from 1 March 2010, have completed a JPD.41 The ANAO will continue to take JPDs into account in its review program in future years, where these have been prepared. However, this will be dependent on the completeness and accuracy of JPDs, in relation to recording the detail of government approvals.

Business systems rationalisation

51. Defence’s business systems42 rationalisation is aimed at consolidating processes and systems in order to provide a more manageable system environment. The new Capability Acquisition and Sustainment Group’s Business Information Management Branch advised that it is making progress in this area through the development of an Integrated Project Management System, which was being progressively built into the then DMO corporate data warehouse in 2014–15.

Project management and skills development

52. Project management and skills development within Defence and the Defence Industry is a key challenge for the Government and industry alike. Over the last decade, more than $300 million has been provided by government to assist with professionalising Defence staff and up-skilling participants within the Defence Industry. While it is widely believed that Defence activities have increased professional competencies held by Defence staff, the measurement of the impact within industry has been limited. The ANAO will continue to review project management and skills development programs in the 2015–16 MPR.

53. Consistent with previous years, the ANAO’s detailed assessment of these governance initiatives is contained in Chapter 3 of Part 1.

1. The 2014–15 Major Projects Review

Introduction

1.1 This chapter provides an overview of the 2014–15 Major Projects Report (2014–15 MPR) review’s scope and approach, implemented by the Australian National Audit Office (ANAO) for the review of the Project Data Summary Sheets, and the subsequent results of the MPR review.

1.2 During the 2014–15 review period the Government released the First Principles Review: Creating One Defence (First Principles Review), a major government review of the Australian Defence Organisation. As a result, the Defence Materiel Organisation (DMO), the focus of the majority of this review, was delisted from 1 July 2015, and merged back into the Department of Defence (Defence).

1.3 The re-merger of the DMO, a large project management organisation, into the Capability Acquisition and Sustainment Group of Defence, is complex, and expected to take time to fully complete. Consequently, broader managerial changes are in the process of being developed and implemented within Defence, at the time of this report. The ANAO has referred to the effects of the change process where appropriate, however, the development and implementation of changes within Defence following the recommendations of the First Principles Review are not expected to be completed until 2017–18.

The impact of the First Principles Review on materiel acquisition

1.4 The First Principles Review was released on 1 April 2015. The First Principles Review is the thirty-sixth substantive government review of Defence since the 1973 Tange Review, Australian Defence, Report on the reorganistion of the Defence group of departments.43 The First Principles Review’s approach to reforming Defence includes addressing ‘waste, inefficiency and rework’44 by looking holistically at Defence’s business structures, materiel acquisition and sustainment capability, and the efficiency and effectiveness of practices within the department.

1.5 The review made 76 recommendations, and supports a business model which is comprised of three identifiable key features:

  • a stronger and more strategic centre within the department;
  • an end-to-end approach for capability development with Capability Managers assigned clear authority and accountability; and
  • enablers that are integrated and customer-centric.45

1.6 The First Principles Review has recommended that the DMO’s functions, with part of the Capability Development Group’s previous responsibilities46 be remerged into the Department of Defence in the newly created group, led by a single Deputy Secretary, who reports directly to the Secretary of Defence. The appointment of the permanent Deputy Secretary of the Capability Acquisition and Sustainment Group, occurred on 31 August 2015. Defence has provided advice on impacts to their organisation as a result of the First Principles Review in Part 2.

Review scope and approach

1.7 In 2012 the Joint Committee of Public Accounts and Audit (JCPAA) identified the review of the Project Data Summary Sheets (PDSSs) as a priority assurance review under section 19A(5) of the Auditor-General Act 1997, providing the ANAO full access to the information gathering powers under the Act. The ANAO’s review of the individual project PDSSs, which are contained in Part 3 of this report, was conducted in accordance with the Australian Standard on Assurance Engagements (ASAE) 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information issued by the Australian Auditing and Assurance Standards Board. In addition, and for the first time in 2014–15, review of the project financial assurance statement, included within each PDSS, has been included in the scope of the Independent Review Report by the Auditor-General. See paragraph 1.23 below, for more detail.

1.8 Excluded from the scope of the ANAO’s review is PDSS data on the identification of Risks and Issues, the Measures of Materiel Capability Delivery Performance, and ‘forecasts’ of future dates and the achievement of future outcomes, due to the lack of a system or systems from which to provide complete and accurate evidence47, in a sufficiently timely manner to complete the review. Accordingly, the Independent Review Report by the Auditor-General does not provide any assurance in relation to this information.

1.9 While the ANAO’s work is appropriate for the purpose of providing an Independent Review Report in accordance with ASAE 3000, the review of individual PDSSs is not as extensive as individual performance and financial statements audits conducted by the ANAO, in terms of the nature and scope of issues covered, and the extent to which evidence is required by the ANAO. Consequently, the level of assurance provided by this review in relation to the 25 Major Projects is less than that provided by our program of audits.

1.10 However, the Major Projects Report examines systemic issues and provides longitudinal analysis for the 25 projects reviewed, and may also reflect on, or have implications for, general project management practices, including overall performance, or financial matters.

1.11 The review was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $2.2 million.

Areas of review focus

1.12 The ANAO’s review of the information presented in the individual PDSSs included:

  • examination of each PDSS and the documents and information relevant to them;
  • review of relevant processes and procedures used in the preparation of the PDSSs;
  • assessment of the systems and controls that support project financial management, risk management, and project status reporting;
  • interviews with persons responsible for the preparation of the PDSSs and those responsible for the management of the 25 projects;
  • taking account of industry contractor comments provided to the ANAO and Defence regarding draft PDSS information;
  • assessing the assurance by Defence managers attesting to the accuracy and completeness of the PDSSs;
  • examination of the representations by the Chief Finance Officer of Defence supporting the project financial assurance and contingency statements, and the independent third-party review of the project financial assurance statements;
  • examination of representations, provided by the Capability Managers, relating to each project’s progress toward Initial Materiel Release (IMR) and Final Materiel Release (FMR), and Initial Operational Capability (IOC) and Final Operational Capability (FOC); and
  • examination of the ‘Statement by the Secretary of Defence’, including significant events occurring post 30 June, and any supporting management representations.

1.13 The ANAO’s processes and procedures to provide independent assurance over the PDSSs also focused on reviewing project management and reporting arrangements in place that contribute to the overall governance of Major Projects. These included:

  • the financial framework, particularly as it applies to the project financial assurance and contingency statements and managing project budgets in the out-turned budget environment (Section 2 of the PDSSs);
  • schedule management and test and evaluation processes (Section 3 of the PDSSs);
  • the capability assessment framework, as it relates to Defence’s evaluation of the likelihood of delivering key capabilities (Section 4 of the PDSSs);
  • ongoing review of the implementation of the Enterprise Risk Management Framework and major risk and issue data (Section 5 of the PDSSs);
  • the project maturity framework and reporting and the systems in place to support the provision of this data (Section 6 of the PDSSs); and
  • developments in the areas of acquisition governance (Chapter 3 in Part 1).

1.14 This review informed the ANAO’s understanding of the systems and processes supporting the PDSSs for the 2014–15 review period, and highlighted issues in those systems and processes that could be beneficially addressed in the longer term.

Results of the review

1.15 The following sections outline the results of the ANAO’s review, which contribute to the overall conclusion in the Independent Review Report by the Auditor-General for 2014–15.

Financial framework

1.16 The project financial assurance statement was introduced in the 2011–12 Major Projects Report and the contingency statements were introduced for the first time in the 2013–14 report. Together, they are aimed at providing greater transparency of projects’ financial status in the out-turned environment, following the move to out-turned budgeting in 2010, and highlight the use of contingency funding to mitigate projects risks.

1.17 In 2013–14, the ANAO reviewed the financial framework, as it applied to managing project budgets and expenditure in the out-turned budget environment, although the project financial assurance statements were formally excluded from the scope of the review and the Independent Review Report by the Auditor-General. The review indicated that all project offices expected to deliver all required capabilities within the allocated budget. However, for AWD Ships, the CEO DMO indicated that the project may have insufficient approved funds to complete the program.48

1.18 In addition, in 2013–14, a number of project offices added additional disclosures to their PDSSs, and in particular, AWD Ships, LHD Ships and ANZAC ASMD Phase 2B, recognised that available funding for price indexation was a key concern. Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.49 This change in supplementation policy has meant that price Indexation has emerged as a risk for some projects, which is required to be managed individually, by each project office.

1.19 In effect, projects which slip past original delivery dates must now access contingency funding where pre-calculated indexation is insufficient. Previously, the separation of yearly indexation funding from other budget components allowed for greater transparency in reporting and fewer risks for project offices to manage.50

1.20 A project’s total approved budget comprises:

  • the programmed budget, which covers the project’s approved activities, for both contractual and departmental aspects of each project; and
  • the contingency budget, which is established to provide adequate budget to cover the inherent cost, schedule and technical risks involved in managing complex acquisitions.51

1.21 The then DMO’s management of financial risk was based on a portfolio management approach, within the responsibilities of the Chief Finance Officer of DMO (refer to paragraph 1.12 in Part 2 of the 2011–12 Major Projects Report). It is expected that Defence will review this approach as part of the implementation of the First Principles Review, and the ANAO will consider any proposed or implemented changes in 2015–16.

1.22 In 2014–15, the ANAO reviewed the financial framework as it applied to managing project budgets and expenditure, including contingency, in the out-turned budget environment, and the project financial assurance and contingency statements.

Project financial assurance statement

1.23 The project financial assurance statement was added to the PDSSs to provide readers with an articulation of a project’s financial position in relation to delivering project capability and to provide transparency in regard to whether there is ‘sufficient remaining budget for the project to be completed’.52

1.24 In 2014–15, while most projects again continued to operate within their total approved budget, the AWD Ships, LHD Ships and ANZAC ASMD 2B project offices continued to recognise that available funding may be insufficient as contracted indices escalation may be greater than the approved project budget. The AWD Ships53 project also included reference in its project financial assurance statement to the announcement by the Ministers for Finance and Defence on 22 May 2015, that the project would require an additional $1.2 billion to be completed, and that this would be funded at the expense of other Defence acquisitions.54 The 2014–15 Statement by the Secretary of Defence disclosed the approval of this real cost increase by government in July 2015. In addition, Battlefield Airlifter disclosed a cost risk for contracts yet to be executed.

1.25 As noted previously, for the first time in 2014–15, the ANAO has included the project financial assurance statements within the scope of the Independent Review Report by the Auditor-General, reflecting the increased level of confidence sought by stakeholders in the accuracy and completeness of the related disclosures and the work completed by the ANAO and the then DMO in this area, since the introduction of the statement.

1.26 Defence has continued to support the project financial assurance statements with an independent third-party review, which considered the following in selecting the review sample: remaining budget, Projects of Concern listing, complexity, diversity across divisions and past history. This independent review remains critical to the ongoing validation of the project financial assurance statements.

1.27 In reality, until systems and processes are improved, from end-to-end in the project acquisition life cycle, the ANAO’s assurance in this area will remain one of the higher risk elements of the review. For example, while Joint Project Directives55 mature and until they can be relied upon to completely and accurately reflect the approval outcome of government, the ANAO will require access to original approval documents to validate the requirements of projects. At this time, validation by internal Defence documentation is not always possible.

1.28 In addition, the current status of performance information, including for contingency management, project maturity scores, and capability delivery (excluded from the scope of this review) are being impacted by inconsistent application and supporting systems, and lack of management review. All of the above are discussed in more detail elsewhere in this report, and directly relate to the validation of expected project delivery, within budget.

1.29 Projects selected for the 2014–15 third-party review, in support of the financial assurance statement assurance process included:

  • detailed review—ANZAC ASMD 2B and ANZAC ASMD 2A; and
  • compliance review—P-8A Poseidon, MRH90 Helicopters and Maritime Comms.

1.30 Observations from the review included that there were inconsistent approaches to the weighting of risks and assigning of contingency against risks, with only two of the five project offices being assessed as compliant with the Defence Materiel Instruction for project financial assurance statements.56 However, it was determined that the three project offices in question had sufficient contingency despite this non-compliance and that price escalation risk was manageable for ANZAC ASMD 2B. The ANAO’s review also noted a number of issues in terms of compliance with Project Risk Management Manual version 2.4 across project offices, which have been reported to management.

1.31 In conclusion, while for the 2014–15 Major Projects Report, the Chief Finance Officer’s representation letter to the Secretary on the project financial assurance statements was unqualified (except for the known injection of $1.2 billion required for AWD Ships and the affordability risk for Battlefield Airlifter), the project financial assurance statement is restricted to the current financial contractual obligations of Defence for these projects, including the result of settlement actions and the receipt of any liquidated damages; and current known risks and estimated future expenditure as at 30 June 2015.

Contingency statements and contingency management

1.32 As noted above, the 2013–14 Guidelines introduced the requirement for a ‘contingency statement’ within each PDSS. PDSSs are now required to include a statement as to whether contingency funds have been applied during the year, as well as disclosing the risks mitigated by the application of those contingency funds. The five project offices which had contingency funds applied in 2014–15 were AWD Ships (indexation funding shortfall and Counter Measure Lockers for storing explosive ordnance), MRH90 Helicopters (technical and integration risks), ARH Tiger Helicopters (for discounts on upgrades to Ground Mission Equipment received as liquidated damages), Air to Air Refuel (risks related to the modification program and spares) and Battlefield Airlifter (divestiture and contracting risk).57

1.33 The examination of the contingency statements as at 30 June 2015 also highlighted that:

  • where project offices had contingency funds applied, the purpose was within the approved scope of the project;
  • the clarity of the relationship between contingency application and identified risks varied. Of the 24 project offices that have a formal contingency allocation58, two did not explicitly align their contingency log with their risk log, and of the 22 project offices that could demonstrate alignment, 10 did not always meet all the requirements of Project Risk Management Manual (PRMM) version 2.4; and
  • the method for applying contingency varied, with only four project offices using the ‘expected costs’ of the risk treatment (as required by PRMM version 2.4), seven for which no application of contingency was necessary (as there were no high/extreme risks or no cost implications), and the remaining 13 using either a proportionate allocation of the likelihood of the risk eventuating (the method outlined in PRMM version 2.2), an alternate method, or having no application of contingency against risk.

1.34 Although the ANAO found that all project offices tracked their contingency budget in some form, the methods of recording the balance of contingency budgets and application of contingency funds differed between projects. For example, project offices varied in whether they maintained an up to date record of reviews of their contingency log, and its adequacy, or included the expected cost for all high/extreme risks in the risk log with corresponding entries in the contingency log, or risk identifiers and descriptors for allocations of their contingency budget, all of which are requirements outlined in PRMM version 2.4.

1.35 The purpose of the project contingency budget is identified as being ‘to provide adequate budget to cover the inherent risk of the in-scope work of the project’.59 Defence policy requires project offices to maintain a contingency budget log to identify and track components of the contingency budget. However, the lack of oversight of compliance with this policy has resulted in inconsistent approaches taken to contingency allocation. For example, in 2014–15, the ANAO observed that half of the project offices were unable to demonstrate clear links in compliance with PRMM version 2.4 for the contingency allocation to individual risks.

Enterprise Risk Management Framework

1.36 In 2013–14, the ANAO’s review concluded that while the standards of risk management arrangements applying to Major Projects have continued to improve, the inherently uncertain nature of risks and issues meant that PDSS data could not be considered complete due to unknown risks and issues that may emerge in the future. In 2014–15, major risks and issues data in the PDSSs continues to remain out of scope of the Independent Review Report by the Auditor-General under arrangements for the priority assurance review, but is included in the analysis to provide an overall perspective of how risks and issues are managed within Defence.

1.37 The ANAO monitors developments in risk management at the enterprise and project level, in order to maintain its understanding of the new Capability Acquisition and Sustainment Group’s risk management systems and processes. Organisationally, the development of the former DMO’s Enterprise Risk Management Framework, was identified by the ANAO in 2008–09, as a challenging but necessary step for the then DMO in achieving its goal of improving project management.60 In 2014–15, the ANAO again monitored the progress of the Enterprise Risk Management Framework and its associated policies and guidance, which continues to mature.

1.38 Finalised in July 2014, Defence conducted an internal audit on Risk Management and the Enterprise Risk Management Framework in Defence. With a broader scope than the ANAO’s examination of project level risk management, the findings of the audit were as follows:

  • risk management in Defence is inadequately mandated and implemented and also has deficient senior ownership;
  • risk management in Defence is inadequately integrated with other Defence processes61;
  • the enterprise risk deep dive process is incomplete and the enterprise risks are not widely communicated or fully understood;
  • the application of risk management in Defence is inconsistent, lacks quality and fails to cascade through the organisation; and
  • many Defence risk managers are inadequately and inconsistently skilled.62

Defence advised in August 2015 that work on the Enterprise Risk Management Framework continues in the new Capability Acquisition and Sustainment Group.

1.39 The Interdependent Mission Management System (IMMS) was developed in response to a recommendation from the 2011 Plan to Reform Support Ship Repair and Management Practices (Rizzo Report) to ‘establish an integrated risk management system’.63 The Rizzo Report stated that in relation to risk management, ‘Navy and DMO need to improve coordination and integrate their interdependent activities more effectively’.64 The IMMS is intended to provide joint visibility of risks at the enterprise level, and facilitate greater accountability in relation to risk management. Following the Navy Reform Board’s May 2014 endorsement of the use of IMMS to manage interdependent risks in Navy, in January 2015, a Materiel Sustainment Agreement65 was signed between Navy and the then DMO, providing the governance framework for interdependent risk management. Currently, IMMS is in the pilot stage with the Guided Missile Frigate (FFG) System Program Office, with final testing forecast for completion in October 2015.66

1.40 In 2014–15, the ANAO again examined project offices’ risk and issue logs, which are created and maintained utilising the new group’s mandated Excel or Predict! software.67 The ANAO’s review indicates that the majority of project offices maintained risk and issue logs appropriately, but not consistently. The ANAO has continued to note inconsistencies in the practices of various project offices, including:

  • where Excel and Predict! logs are maintained concurrently for the same project, risks are not always consistently recorded; and
  • where Excel is used, there is variability in the information content of the log and in some cases, evidence that it has not received appropriate management attention.

1.41 While some project offices will experience greater challenges with risks and issues administration, considering project complexity, scale and timing, it is important that Defence provide systems and processes to ensure risks are appropriately managed and reviewed across the life of the project. Large complex projects, which are more likely to have a higher number of risks and issues, face the challenge of putting internal systems in place to meet organisational requirements. Particularly in the case of higher cost developmental projects, which also tend to exist in an environment of often considerable scrutiny and political risk.

1.42 In this context, the Joint Strike Fighter project is an example where the project has developed a hierarchical view of risks, due to the number of risks in existence. For example, the project has 51 ‘high’ rated risks (pre-mitigation). By necessity, these are summarised in the Joint Strike Fighter PDSS at a strategic level. However, consideration of the Joint Strike Fighter’s risks is made more difficult given that the project has not finalised its Risk Management Plan, which has remained in draft since 2014.

1.43 Smaller projects, while typically having fewer risks and issues, also have fewer and less experienced staff, and are less likely to have experienced dedicated risk managers as part of their staffing profile. The Battlefield Airlifter project is an example where the ANAO was provided with both Predict! and Excel risk logs, which were inconsistent.

1.44 Overall, the issues with risk management that the ANAO observed related to:

  • variable compliance with corporate guidance to ensure complete, timely and accurate representation of project risks and issues;
  • the currency of Risk Management Plans and the frequency of their update to capture changes in policy and practice68;
  • the frequency with which risk and issue registers are reviewed to ensure risks and issues are appropriately managed and reported to senior management; and
  • risk management logs and supporting documentation of variable quality, particularly when Excel is used.

1.45 The Standardisation Office is the corporate area responsible for the development, amendment and publishing of corporate risk management policy within the Capability Acquisition and Sustainment Group. Gate Reviews held by the Independent Project Performance Office also have a degree of oversight over project risk management processes.69 In 2014–15, both areas confirmed that they provide guidance and advice only. Neither have the mandate or resources for systematic compliance monitoring of risk management.

1.46 Since 2007–08, risks and issues have been a consistent focus of review, although excluded from the formal scope of the Independent Review Report by the Auditor-General, due to issues with completeness, supporting systems and executive oversight. Throughout this time the ANAO has identified deficiencies and worked with project offices to improve compliance with the risk management policies applicable in the then DMO. It is clear that increased scrutiny and accountability of project performance is required to identify shortcomings in corporate performance to support project offices manage their risks, and deficiencies in local project risk management performance.

1.47 To achieve greater consistency in the approach to risk management and in response to the release of a Commonwealth Risk Management Policy on 1 July 2014, the Capability Acquisition and Sustainment Group is developing a single Risk Management Manual, which is expected to be finalised at the end of 2015.70

Project maturity framework

1.48 Initially introduced as Project Risk Scores in 2004, and later renamed Project Maturity Scores in 2005, they have been a feature of the Major Projects Report since inception in 2007–08. The DMO Project Management Manual 2012, defines a maturity score as:

The quantification, in a simple and communicable manner, of the relative maturity of capital investment projects as they progress through the capability development and acquisition life cycle.71

1.49 Maturity scores are a composite indicator, constructed through the assessment and summation of seven different attributes, which cumulatively form a project ‘maturity score’. The attributes are: Schedule, Cost, Requirement, Technical Understanding, Technical Difficulty, Commercial, and Operations and Support, which are assessed on a scale of one to 10.72 Project Maturity is a composite performance indicator available for all Major Projects, for decision making, and to assess their overall status.73

1.50 Historically, while the DMO had raised some doubts about the effectiveness of their maturity score framework, they agreed to retain maturity scores following a JCPAA recommendation that ‘DMO maintain the ability to publish maturity scores in future Major Projects Reports until these are no longer required by the guidelines endorsed by the JCPAA’.74 The Committee viewed the retention of maturity scores as important in relation to providing a measure of capability delivered for each project, until a measure equal to or better than current arrangements is available. Recently, the decision to maintain maturity scores, while seeking to develop an improved measure, was again reaffirmed by Defence to the ANAO in the context of this 2014–15 review.

1.51 In 2014–15, Defence also indicated that the organisation is relying less on project maturity scores and are instead moving towards other project management tools, such as the Materiel Implementation Risk Assessment (MIRA). The MIRA is used during the First Pass Approval stage for projects and is designed to assist project offices in submitting details of their top five risks in the acquisition business case for cabinet submission. The DMO Project Risk Management Manual 2013 defines MIRA as:

A summary of the most significant risks (as recorded in the project risk register) that will impact on DMO’s ability to deliver the Materiel System (Mission and Support System) outcomes on time, within budget, and to the required scope and quality.75

1.52 As the MIRA outlines a project’s key risks at only one point in time, government First Pass Approval, the ANAO notes that for reporting purposes, the MIRA does not provide the same level of oversight on a project’s delivered capability as maturity scores. During the course of the review, the ANAO reviewed the MIRA for new projects, to ensure that the risks disclosed in the MIRA were included in the project risk registers. The results of which were consistent with general alignment with current PDSS disclosures, with any differences due to the passage of time, increased project knowledge, and risk management efforts.

1.53 However, comparing the maturity score against its expected life cycle gate benchmark provides internal and external stakeholders with an indication of a project’s progress. This may trigger further management attention or provide confidence that progress against the appropriate maturity score benchmark is satisfactory.

1.54 While the ANAO has previously raised inconsistency in the application of project maturity scores as an issue, and as maintained in this review, the ANAO noted that project offices were more consistently assigning maturity scores than in previous years. While some subjectivity remains, in the context of a framework that relies upon the application of professional judgement, across a diverse range of project circumstances, with the detailed guidance available, assigning a maturity score is a repeatable process, and is appropriate for external review or audit.

1.55 As previously noted by the ANAO, the guidance underpinning the attribution of maturity scores would benefit from a review for internal consistency and relationship to the Defence’s contemporary business. For example, allocating approximately 50 per cent of the maturity score at Second Pass Approval, regardless of acquisition type, is often inconsistent with the proportion of project budget expended, and the remaining work required in order to deliver the project.

1.56 Further, the existing project maturity score model does not always effectively reflect a project’s progress during the often protracted build phase, particularly for developmental projects. During this phase it can be expected that maximum expenditure will occur, and risks realised, some of which will only emerge as test and evaluation activities are pursued through to acceptance into operational service.

1.57 Finally, while the guidance underpinning maturity scores was due for review in September 201276, this review is not yet finalised. The ANAO was advised that while work had occurred to review the guidance, the release of the First Principles Review meant that the guidance would require further consideration.

1.58 The ANAO will continue to review the framework and attribution of maturity scores in subsequent reviews.

Efficiency of the Major Projects Report process

1.59 As in previous years, project offices prepared indicative PDSSs and supporting evidence prior to 30 June 2015 to support the initial stage of ANAO fieldwork for the 2014–15 review. In general, project offices provided high quality PDSSs, with a few exceptions.

1.60 However, while more recent requirements of the review, such as the provision of the contingency statement, are still maturing, the provision of core project information on cost, schedule and the progress towards delivery of required capability, remains a largely manual process. Additionally, a significant amount of project information is not centrally maintained, requiring extensive contact with individual project offices to obtain evidence to assure the information in the PDSSs. Further, inconsistent application of Defence policies for project management increases these difficulties.77

1.61 The ANAO will continue to assess the application of policy and maintain its focus on the efficiency and consistency of the production and assurance of the PDSSs.

Review conclusion

1.62 The Independent Review Report by the Auditor-General takes into account the overall governance of Major Projects, the results of the ANAO’s examination of the then DMO’s project management and reporting arrangements, and the results of the ANAO’s substantive procedures to gain assurance in relation to key information reported in PDSSs. In 2014–15, the results of the ANAO’s priority assurance review of the 25 PDSSs, was that nothing has come to the attention of the ANAO that causes us to believe that the information and data in the PDSSs, within the scope of our review, has not been prepared, in all material respects, in accordance with the Guidelines.

2. Analysis of Projects’ Performance

Introduction

2.1 Performance information is important in the management and delivery of major Defence equipment acquisition projects (Major Projects) as it informs decisions about the allocation of resources, supports advice to government on project progress and performance, and allows for the Parliament and the public to assess the progress of projects. Project performance information has been the subject of many of the reviews of Defence and a consistent area of focus of the Joint Committee of Public Accounts and Audit (JCPAA) over the time of the Major Projects Report (MPR). This chapter progresses previous Australian National Audit Office (ANAO) analysis of project performance, and utilises data collected over the life of the MPR, to provide updated analysis across the portfolio of MPR projects.

Project performance analysis by the ANAO

2.2 The ANAO utilises three key performance indicators to lead analysis over the three major dimensions of projects’ progress and performance against Defence’s Project Maturity scores.78 These indicators are the:

  • percentage of budget expended (Budget Expended)—which measures the total expenditure as a percentage of the total current budget;
  • percentage of time elapsed (Time Elapsed)—which measures the percentage of time elapsed from original approval to the forecast Final Operational Capability (FOC)79; and
  • percentage of key materiel capabilities expected to be delivered (Expected Capability)—which is Defence’s assessment of the likelihood of delivering the required level of capability.

These are measured in percentage terms, to enable comparisons between projects, and to provide a portfolio view across project progress and performance.

2.3 Following this analysis, each section then provides additional analysis of each of the three major dimensions, with key in-year information, longitudinal analysis compiled since the inception of the review, and results of project progress for the year-ended 30 June 2015.

2.4 Prior to disaggregation and analysis against Project Maturity scores, Figure 2 below, provides an overview of the three major dimensions of project performance, and sets out Budget Expended, Time Elapsed80 and Expected Capability.81

Figure 2: Budget Expended, Time Elapsed and Expected Capability

Note: The Expected Capability for Wedgetail has been assessed against the Supplies section of the Materiel Acquisition Agreement, which lists the equipment to be delivered.

Source: The ANAO’s analysis of Budget Expended and Time Elapsed of the 2014–15 PDSSs. Refer to paragraphs 2.4 to 2.9 in Part 1 of this report. Expected Capability is as presented in the PDSSs.

2.5 The figure shows that for most projects (19 of 25), Budget Expended is broadly in line with, or lagging, Time Elapsed.82 This relationship is generally expected in an acquisition environment predominantly based on milestone payments. However, due to the varying complexities, stages and acquisition approaches across the portfolio of projects, further analysis of these simple performance measures provides an overall picture of key variances.

2.6 Where Budget Expended is lagging Time Elapsed the project schedule may be at risk or milestones not met. For example, projects where the Budget Expended is approximately 20 per cent less than the Time Elapsed, include:

  • Joint Strike Fighter (Budget Expended five per cent, Time Elapsed 40 per cent)—a large scope increase ($10.5 billion) for the purchase of additional aircraft was approved in April 2014 with the project yet to enter main production contracts;
  • Battle Comm. Sys. (Budget Expended 72 per cent, Time Elapsed 100 per cent)—the project achieved FOC in April 2015, with some items still awaiting delivery, and final contract acceptance payments dependant on these deliveries; and
  • LHD Landing Craft (Budget Expended 64 per cent, Time Elapsed 85 per cent)—expenditure for delivery and acceptance of the final batch of landing craft is forecast for January 2016, which is only one month prior to the scheduled Final Materiel Release (FMR) and FOC date of February 2016, placing additional pressure on schedule.

2.7 Where Budget Expended leads Time Elapsed the project budget may be at risk. Projects where Budget Expended leads Time Elapsed (from government approval to FOC) by more than 10 per cent, include:

  • AWD Ships (Budget Expended 76 per cent, Time Elapsed 58 per cent)—contracted indexation rates have been higher than the out-turning applied in the project budget while productivity has been lower than expected. This has resulted in the need for an additional $1.2 billion to complete the project and an extension of the project schedule83;
  • Growler (Budget Expended 47 per cent, Time Elapsed 23 per cent)—most of the expenditure on equipment is in line with aircraft production in advance of training;
  • MH-60R Seahawk (Budget Expended 47 per cent, Time Elapsed 32 per cent)—the project has taken delivery of 12 aircraft to June 2015, with further aircraft in production and all aircraft forecast to be delivered by August 2016. The variance is also exacerbated by the long time between final aircraft delivery and FOC, with this time being used to implement Australian unique modifications and modify navy vessels to operate with the MH-60R;
  • Collins RCS (Budget Expended 97 per cent, Time Elapsed 78 per cent) and Reliability and Sustainability (Budget Expended 86 per cent, Time Elapsed 68 per cent) projects—most of the materiel has been acquired and expenditure undertaken. In addition, originally planned installation dates have been extended, based on submarine availability, reducing the proportion of total Time Elapsed; and
  • UHF SATCOM (Budget Expended 80 per cent, Time Elapsed 68 per cent)—while significant milestones have been achieved, recent problems associated with software development have delayed FMR, and the project anticipates these will affect FOC.

2.8 In each case, reviewing the performance information highlights projects where reasonable explanations for variances or anomalies exist, however, there are situations where the review of performance information shows projects which require additional Defence or government attention.

2.9 Defence expects that all 25 projects in this year’s report will deliver all of their key capability requirements, recognising that some elements of the capability required for some projects may be under threat, but are considered manageable (assessed as either green or amber). While this is consistent with the 2013–14 Defence assessment, it may be considered overly optimistic and is not further analysed here.

Cost performance analysis

Budget Expended and Project Maturity

2.10 Figure 3, below, sets out each project’s Budget Expended against Project Maturity84 and shows that Budget Expended lags Project Maturity for the majority of the projects (19 of 25). This relationship is to be expected as greater numbers of Military-Off-The-Shelf (MOTS) projects are included in the sample of Major Projects, as MOTS products are in-service, and have generally had significant development and testing prior to selection for acquisition.

2.11 Where Budget Expended lags Project Maturity with an approximate differential of 20 per cent or more, the project is classified as either MOTS or Australianised MOTS, except Joint Strike Fighter, which is expected to be MOTS by the time of aircraft delivery. Unlike the relationship between Budget Expended and Time Elapsed, and consistent with prior years, there are no instances where Budget Expended leads Project Maturity by greater than 20 per cent.

2.12 These variances, in part, are the result of Defence’s project maturity framework attributing approximately 50 per cent of the total project maturity at Second Pass Approval. This reduces the value of project maturity assessments within the acquisition phase, which is predominantly the longest and most expensive component of Major Project acquisition.

2.13 Again, analysis of the performance information available highlights useful information for Defence and government, i.e. the selection of MOTS projects significantly reduces risk during project acquisition, where Project Maturity is far more advanced at approval than developmental projects.

2.14 The Defence Materiel Organisation’s (DMO’s) focus on MOTS acquisitions has seen reduced slippage across projects, but may not provide acceptable solutions in all cases. The balance across the portfolio, of MOTS versus developmental acquisitions, should reflect the risk profile of government and Defence, to allow appropriate mitigation strategies to be employed.

Figure 3: Budget Expended and Project Maturity

Note: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Analysis for the 2014–15 Major Projects Report continues to highlight inconsistencies within the application of project maturity, reducing the level of reliability of maturity assessments for key decision makers and other stakeholders; however, improved focus and review by project offices was noted by the ANAO during 2014–15 fieldwork. Refer to paragraphs 2.10 to 2.14 in Part 1 of this report.

Second Pass Approval and 30 June 2015 approved budget

2.15 For the 25 Major Projects, Figure 4, below, compares each project’s approved budget at Second Pass Approval (the main investment decision by government) and their approved budget at 30 June 2015.

2.16 The total approved budget for the 25 projects at 30 June 2015 was $60.5 billion, a net increase of $18.5 billion, when compared to their budget at Second Pass Approval of $42.0 billion (analysis of this variance is included in Figure 5).85

2.17 Figure 4 indicates relative budget variations from Second Pass Approval of 150 per cent or greater for the following projects:

  • Joint Strike Fighter—increase of $12.4 billion, ($10.5 billion for 58 additional aircraft in 2013–14, cumulative exchange rate variations of $1.6 billion and cumulative price variations of $0.3 billion – total increase of 451.7 per cent);
  • MRH90 Helicopters—increase of $2.8 billion, ($2.3 billion for 34 additional aircraft in 2005–06 and other minor scope changes, cumulative exchange rate variations of -$0.2 billion and cumulative price variations of $0.7 billion – total increase of 291.5 per cent);
  • Bushmaster Vehicles—increase of $955.5 million, ($832.2 million for 715 additional vehicles in 2007–08 (437 vehicles), 2011–12 (70 vehicles) and 2012–13 (208 vehicles) and other minor scope changes, cumulative exchange rate variations of -$1.3 million, and cumulative price variations of $124.6 million – total increase of 323.9 per cent);
  • ANZAC ASMD 2B—increase of $429.8 million, ($363.4 million for replacing the Very Short Air Defence system with the CEA Technologies Phased Array Radar system in 2005–06, approval of the rollout of upgrades for ships 2–8 in 2011–12 and other minor scope changes, cumulative exchange rate variations of -$9.7 million, and cumulative price variations of $76.1 million – total increase of 172.7 per cent); and
  • Collins R&S—increase of $339.7 million, ($271.2 million for additional scope in 2001–02, cumulative exchange rate variations of -$5.9 million, and cumulative price variations of $74.4 million – total increase of 471.8 per cent).

2.18 Figure 4 also highlights projects which have movements in excess of $500 million where the movement as a percentage of the total approved project budget is less than 150 per cent. In these instances the movements are predominantly related to price variation and exchange rate movements but also include various minor scope variations. These projects include:

  • AWD Ships—$683.7 million86;
  • Wedgetail—$623.7 million; and
  • Growler—$863.3 million.

Figure 4: Projects’ Second Pass Approval and 30 June 2015 approved budget ($m)

Note 1: [vertical white-filled rectangle] indicates that the budget for the project at 30 June 2015 is less than the original budgeted cost. However, for Overlander Light this reflects a transfer of $2.2 billion to Overlander Medium/Heavy on separation of the original project into two phases in December 2011.

Note 2: The Second Pass Approval amount for the Overlander Medium/Heavy project includes an amount of $0.7 billion for general program supplementation, which was provided as part of the revised Second Pass Approval in July 2013.

Note 3: On 22 May 2015, the then Minister for Defence and the Minister for Finance announced there would be further delays to the delivery of the Air Warfare Destroyers and a further $1.2 billion would be required to complete the project. This budget increase had not been incorporated into the approved project budget at 30 June 2015. This budget increase will be analysed in the 2015–16 Major Projects Report.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Refer to paragraphs 2.15 to 2.18 in Part 1 of this report.

Budget performance

2.19 The following figures and tables illustrate the budget variations for the 25 projects in the 2014–15 report across three differing methodologies:

  • the three main factors contributing to budget variations in each of the last seven years, presented longitudinally from 2008–09 to 2014–15;
  • the in-year budget variations by project; and
  • the expenditure forecasting performance against the actual expenditure for the 2014–15 financial year.
Longitudinal and in-year budget variance analysis

2.20 As noted in paragraph 1.18, from 1 July 2010 price indexation is now provided as part of a project’s budget at Second Pass Approval. This is defined as the budget being ‘out-turned’.87

2.21 Figure 5, below, examines the three main factors contributing to budget variations in each of the last seven years, and highlights the in-year impact of variations for the 25 projects in the 2014–15 report. The 2014–15 net life to date budget variation of $18.5 billion for the 25 Major Projects, when compared to their Second Pass Approval budget, comprises price variation increases of $5.1 billion, foreign exchange rate movement increases of $1.5 billion and real variation increases of $11.9 billion.

2.22 Exchange rate variations in project budgets are a result of projects’ exposure to foreign currencies and movement in foreign exchange rates against the Australian dollar.88 The exposure of specific projects to currency exchange variations is established through the initial government procurement decision and contractual agreement. The US dollar and the Euro are the main influences, although other currencies also have an impact. This year the Australian dollar generally weakened against the US dollar, while against the Euro, similar to last year, it varied significantly across the year but had stabilised to a similar level to where it began by the end of the financial year.

2.23 The large in-year exchange variation of $2.8 billion is due to adjustments in December 2014, February 2015 and May 2015 totalling $0.2 billion, $2.1 billion and $0.5 billion respectively.

2.24 Real variations in project budgets primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability, and budgetary adjustments such as administrative savings decisions. The in-year variation of $0.2 billion reflects an increase in the total approved budget for the Growler project for the Mobile Threat Training Emitter System, air-to-air and anti-radiation weapons, partially offset by a small scope reduction in the Battle Comm. Sys. project, for work being transferred to the LAND 75 Phase 4 Battle Management System project (not included in the 2014–15 Major Projects Report).

Figure 5: Longitudinal budget variations for 2008–09 to 2014–15 ($m)

Note: The significant In-year Real Variation recorded in 2013–14 reflects total approved budget increases of $10.5 billion for the Joint Strike Fighter project for additional aircraft and $0.7 billion for the Overlander Medium/Heavy project for general program supplementation which was provided as part of the revised Second Pass Approval in July 2013.

Sources: The ANAO’s analysis of the 2014–15 PDSSs and Project Budget Approval Histories. Refer to paragraphs 2.20 to 2.24 in Part 1 of this report.89

2.25 Table 6, below, sets out the 2014–15 in-year budget variations for each of the 25 Major Projects. Overall, the 30 June 2015 approved budget for the 25 projects in the 2014–15 report increased by $2 958.2 million in 2014–15 or 5.5 per cent, compared to their 30 June 2014 approved budget. As discussed above, this was driven by net real increases of $192.3 million (mostly for scope increases), and exchange variation increases of $2 765.9 million.

2.26 In 2014–15, only two projects had real variations, being Growler, which received $200.6 million for the Mobile Threat Training Emitter System, air-to-air and anti-radiation weapons, and Battle Comm. Sys., which had a scope reduction of $8.3 million for work that was transferred to the LAND 75 Phase 4 Battle Management System project.

2.27 Projects with larger movements in foreign exchange, as a result of the depreciating Australian dollar, include:

  • Joint Strike Fighter—$1 725.6 million, or 12.8 per cent increase in budget;
  • P-8A Poseidon—$472.8 million, or 11.9 per cent increase in budget;
  • Growler—$294.2 million, or 9.6 per cent increase in budget; and
  • MH-60R Seahawk—$211.6 million, or 6.6 per cent increase in budget.

Table 6: In-year (2014–15) budget variations by project

Project

Approved Budget 2013–14 $m

Approved Budget 2014–15 $m

In-year Exchange Variation $m

In-year Real Variation $m

Total Variance $m

Total Variance (per cent)

Joint Strike Fighter

13 455.5

15 181.1

1 725.6

-

1 725.6

12.8

AWD Ships

7 847.9

7 891.1

43.2

-

43.2

0.6

P-8A Poseidon

-

3 977.8

472.8

-

472.8

11.9

Wedgetail

3 873.1

3 893.2

20.1

-

20.1

0.5

MRH90 Helicopters

3 785.1

3 747.5

(37.6)

-

(37.6)

(1.0)

Growler

3 036.6

3 531.4

294.2

200.6

494.8

16.3

MH-60R Seahawk

3 196.9

3 408.5

211.6

-

211.6

6.6

Overlander Medium/Heavy

3 469.0

3 387.6

(81.4)

-

(81.4)

(2.3)

LHD Ships

3 089.4

3 091.0

1.6

-

1.6

0.1

ARH Tiger Helicopters

2 033.0

2 032.7

(0.3)

-

(0.3)

-

Air to Air Refuel

1 821.4

1 822.3

0.9

-

0.9

-

Battlefield Airlifter

1 289.5

1 369.2

79.7

-

79.7

6.2

Bushmaster Vehicles

1 250.4

1 250.5

0.1

-

0.1

-

Overlander Light

1 020.5

1 015.7

(4.8)

-

(4.8)

(0.5)

ANZAC ASMD 2B

678.4

678.6

0.2

-

0.2

-

Additional Chinook

617.2

633.8

16.6

-

16.6

2.7

Battle Comm. Sys. (Land)

460.1

461.9

1.8

-

1.8

0.4

Collins RCS

450.1

450.4

0.3

-

0.3

0.1

Maritime Comms

-

442.1

16.0

-

16.0

3.6

Hw Torpedo

426.6

427.9

1.3

-

1.3

0.3

UHF SATCOM

419.1

420.4

1.3

-

1.3

0.3

Collins R&S

411.7

411.7

-

-

-

-

ANZAC ASMD 2A

386.9

386.8

(0.1)

-

(0.1)

-

Battle Comm. Sys.

314.8

313.0

6.5

(8.3)

(1.8)

(0.6)

LHD Landing Craft

239.9

236.2

(3.7)

-

(3.7)

(1.5)

Total

53 573.1

60 462.4

2 765.9

192.3

2 958.2

5.5

Source: The ANAO’s analysis of the 2013–14 and 2014–15 PDSSs. Refer to paragraphs 2.25 to 2.27 in Part 1 of this report.

In-year forecast and actual expenditure

2.28 Accurately forecasting and managing budget expenditure is an important element in the effective management of a portfolio of projects and understandably receives close attention within Defence and by key stakeholders, including the JCPAA. Figure 6 sets out the expenditure forecasting performance of each project against the actual expenditure for the 2014–15 financial year. In total, the actual expenditure for the 25 projects at 30 June 2015 was $4.8 billion, against an initial (PBS) forecast expenditure of $4.3 billion, a mid-year (PAES) forecast of $4.2 billion, and a final forecast of $5.0 billion (Final Plan). The main factors contributing to the variances were changes to delivery and payment schedules, and foreign exchange fluctuations.

2.29 As part of its review of the 2012–13 report, the JCPAA recommended that expanded information regarding projects’ budget estimates and actual expenditure be included in the review. The review now includes projects’ PBS, PAES and Final Plan estimates, projects’ actual expenditure, and explanations for the variances between each of these amounts.90 This information is set out below in Figure 6, in Table 2.2 of Part 2 of this report and in Sections 2.2A and 2.2B of the individual Project Data Summary Sheets (PDSSs).

2.30 The ANAO’s analysis at Figure 6 highlights that notable in-year underspends occurred in the following projects:

  • Joint Strike Fighter (expenditure of $233.2 million compared to $237.9 million PBS, $277.9 million PAES and $296.5 million Final Plan estimates)—the variance is primarily due to the unpredictability of the United States Government F-35 Joint Program Office invoicing;
  • LHD Ships (expenditure of $81.3 million compared to $142.6 million PBS, $85.6 million PAES and $86.6 million Final Plan estimates)—the variance is primarily due to the delay in production and testing milestones after ship acceptance. Expected reductions were reflected in the updated estimate figures; and
  • Battlefield Airlifter (expenditure of $158.5 million compared to $313.8 million PBS, $255.4 million PAES and $271.5 million Final Plan estimates)—the variance is due to schedule slippage, delays to deliveries of equipment and a Foreign Military Sales (FMS) case payment of $54.0 million that was paid early, in the 2013–14 financial year rather than the 2014–15 financial year, which was not known at the time of PBS compilation.

2.31 Notable in-year overspends occurred in the following projects:

  • AWD Ships (expenditure of $734.2 million compared to $615.6 million PBS, $759.3 million PAES and $763.2 million Final Plan estimates)—the variance is primarily due to increases associated with shipbuilding activities and cost over-runs, which was reflected in the updated PAES and Final Plan estimates. The project underspent against the PAES and Final Plan estimates, primarily due to further schedule slippage for Ship 1;
  • P-8A Poseidon (expenditure of $531.5 million compared to $324.0 million PBS, $450.8 million PAES and $516.4 million Final Plan estimates)—the variance is primarily due to payments being brought forward into the 2014–15 financial year from the 2015–16 financial year and the associated foreign exchange impacts. The increase in payments was largely reflected in the updated estimates figures;
  • Growler (expenditure of $1 241.9 million compared to $797.4 million PBS, $728.5 million PAES and $1 202.5 million Final Plan estimates)—the variance is primarily due to payments being brought forward into the 2014–15 financial year from the 2015–16 financial year for aircraft production and Airborne Electronic Attack kits and the associated foreign exchange impacts. The increase in payments was largely reflected in the updated Final Plan estimate figure; and
  • MH-60R Seahawk (expenditure of $685.5 million compared to a $504.7 million PBS, $511.7 million PAES and $670.8 million Final Plan estimates)—the variance is primarily due to the acceleration of FMS payments for the MH-60R Seahawk helicopters and the associated foreign exchange impacts. The increase in payments was largely reflected in the updated Final Plan estimate figure.

2.32 The overall project expenditure in the portfolio of projects was comparable to the final plan budget, however, it should be noted that the then DMO budget management was on a portfolio basis, limiting the performance information value of project by project analysis. For example, individual project underspends in one project are offset by bringing forward payments in other projects for cash management purposes.

Figure 6: In-year (2014–15) projects’ forecast expenditure performance compared to actual expenditure ($m)

Sources: The ANAO’s analysis of the 2014–15 PDSSs and Defence Portfolio Budget Statements. Refer to paragraphs 2.28 to 2.32 in Part 1 of this report.

Schedule performance analysis

2.33 Defence continues to acknowledge that schedule performance is the key issue in delivering and sustaining equipment for the Australian Defence Force.91 Further, in 2013, the then Minister for Defence, Senator the Hon. David Johnston stated:

We need to be able to do things much more time effectively, because time is money and this is one of the things we see as the principal causes of problems inside Defence. We want to know when things are beginning to lag, schedule is starting to slip… being forewarned we can give the project appropriate attention.92

2.34 In addition, project slippage could effectively introduce or exacerbate an existing capability gap or require extension to the planned withdrawal date for those platforms being replaced. For example, the withdrawal dates for the Sea King and Black Hawk helicopter fleets included consideration of the introduction of replacement capability.

Time Elapsed and Project Maturity

2.35 Figure 7, below, sets out each projects’ Time Elapsed against Project Maturity93, and shows that Time Elapsed lags Project Maturity for 19 of 25 projects with no exceptions greater than 20 per cent. Projects where Time Elapsed lags Project Maturity with an approximate differential of 20 per cent, similar to Figure 3, and as noted in paragraph 2.11, are either MOTS or Australianised MOTS, except Joint Strike Fighter, which is expected to be MOTS by the time of aircraft delivery. Again, analysis of the performance information available highlights useful information for Defence and government, i.e. the selection of MOTS projects significantly reduces risk during project acquisition.

2.36 Time Elapsed lagging Project Maturity by 20 per cent or more reflects that projects are at an early stage of the capability acquisition life cycle and awaiting significant amounts of their major equipment to be constructed and delivered.

Figure 7: Time Elapsed and Project Maturity

Note: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Analysis for the 2014–15 Major Projects Report continues to highlight inconsistencies within the application of project maturity, reducing the level of reliability of maturity assessments for key decision makers and other stakeholders; however, improved focus and review by project offices was noted by the ANAO during 2014–15 fieldwork. Refer to paragraphs 2.35 to 2.36 in Part 1 of this report.

Schedule slippage and acquisition type by approval date

2.37 Figure 8, below, illustrates the total schedule slippage since Second Pass Approval for projects in the 2014–15 report, demonstrating how this key performance indicator has altered over time. It also depicts the acquisition type and places projects in order of government approval, allowing for more detailed analysis. In addition, Figure 9 illustrates the total schedule slippage for projects which have exited the review to allow further analysis.

2.38 Figures 8 and 9 continue to show that a greater focus on MOTS and Australianised MOTS acquisitions is, prima facie, reducing the slippage in the Major Projects profile. However it is not always possible to acquire the necessary capability in this manner and should be considered with government and Defence’s preferred risk profile and resources in mind.

2.39 The data in Figures 8 and 9, illustrates that older projects, which achieved Second Pass Approval prior to 2005, generally experienced the most slippage. These projects tend to be more developmental (complex) in nature and typically experienced schedule slippage in the past, and have continued to do so. This shows an ongoing trend of slippage in historically late projects.

Figure 8: Current Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to Final Operational Capability (FOC) dates.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s Major Projects Report analysis is Production Period Five.

Source: The ANAO’s analysis of the PDSSs in published Major Projects Reports. Refer to paragraphs 2.37 to 2.40 in Part 1 of this report.

2.40 The challenge of gaining a full understanding of the complexities of developmental aspects of projects at Second Pass Approval is evident by the extent of slippage over time. The 2008 Audit of the Defence Budget (Pappas Review) identified technical risk as the largest source of post Second Pass Approval schedule slippage for ‘post Kinnaird’ projects94, and also observed that schedule slippage causes cost escalation.95

2.41 In JCPAA Report 442, the Committee flagged its intention to continue to monitor activity in this area, and the ANAO will continue to compile data for further longitudinal analysis in the future.96

Figure 9: Exited Major Projects—Total slippage post Second Pass Approval and acquisition type by approval date (years)

Note 1: The order of the projects is from latest to earliest approved. All project slippage relates to FOC dates.

Note 2: Does not include the AIR 5376 Phase 3.2 Hornet Refurb project which exited in 2012 as this project did not introduce a new capability and so did not have an FOC date.

Source: The ANAO’s analysis of the PDSSs in published Major Projects Reports. Refer to paragraphs 2.37 to 2.40 in Part 1 of this report.

Schedule performance

2.42 The subsequent figures and tables illustrate the following for the 25 projects in the 2014–15 report:

  • the original and 30 June 2015 forecasts for achieving FOC;
  • in-year schedule changes to achieving FOC;
  • total schedule slippage across the Major Projects; and
  • total slippage according to a project’s Second Pass Approval date.
Original and 30 June 2015 Final Operational Capability forecasts

2.43 Figure 10, below, presents information on the projects’ original and 30 June 2015 forecasts for achieving FOC. The total schedule slippage for the 25 Major Projects to date is 768 months when compared to the initial prediction when first approved by government. This slippage represents a 28 per cent increase on the expected schedule since the main investment decision.97 Of the 25 projects in the 2014–15 report, 17 have experienced schedule slippage.

2.44 The total schedule slippage across the 2014–15 Major Projects of 768 months, is 347 months lower than the figure of 1 115 months reported in the 2013–14 report. As noted earlier, the difference is mainly due to projects with large amounts of accumulated schedule slippage exiting the review at the end of 2013–14, partially offset by in-year schedule slippage.

2.45 The reasons for schedule slippage often include underestimation of the difficulties associated with technical factors such as design problems, industry capacity and capability, difficulties in system integration to achieve the required capability, or emergent work associated with upgrades.98 In other cases, a project office’s ability to gain access to the platform for upgrading can delay the schedule (for example, the two Collins submarine projects and Hw Torpedo).99

2.46 A closer examination of the reasons for schedule slippage demonstrates the importance of initial assessments of the purchase type, i.e. MOTS, Australianised MOTS or developmental.100 Two projects, MRH90 Helicopters101 and ARH Tiger Helicopters102, were misclassified as MOTS when the projects were both actually Australianised MOTS (i.e. more developmental), which has resulted in extended schedule slippage.

Figure 10: Projects’ original and 30 June 2015 FOC forecasts

Note 1: indicates that the forecast FOC date for the project at 30 June 2015 is earlier than the original FOC date.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s Major Projects Report analysis is Production Period Five.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Refer to paragraphs 2.43 to 2.46 in Part 1 of this report.

In-year schedule performance

2.47 In 2014–15, there was a total schedule slippage of 41 months in the forecast achievement of FOC for the 25 Major Projects. In-year project performance, measured by slippage over the last 12 months, may not reflect the project trend, however, Figure 11 below, shows recovery of previously anticipated slippage for three projects:

  • Wedgetail—achieved FOC in May 2015, one month ahead of the revised forecast schedule;
  • Overlander Light—currently expects to achieve FOC in October 2016, three months later than originally planned, but five months ahead of the 2013–14 forecast schedule; and
  • Collins R&S—changes in the Full Cycle Docking Schedule have resulted in the project now predicting the achievement of FOC in May 2022, four months ahead of the 2013–14 forecast schedule of September 2022.

2.48 In-year schedule slippage involved the following eight projects103 (the explanation provided, drawn from the 2014–15 PDSSs, may also include the reasons for prior slippage):

  • Joint Strike Fighter—minor delay resulting from the reassessment of the projected schedule;
  • AWD Ships—delays reflected in the Comprehensive Cost Review104, which indicated further delays in the delivery of the three ships by 15, 12 and 12 months respectively105, this impact has flowed through to the subsequent major milestones;
  • Overlander Medium/Heavy—there has been minor delays of two months this year, however the project still expects to achieve FOC in March 2023, nine months ahead of the reapproved schedule;
  • Air to Air Refuel—delays resulting from issues around the Aerial Refuelling Boom System, in particular the fleet modification program to upgrade all aircraft to the final boom configuration and service release of the boom capability;
  • Battlefield Airlifter—delays resulting from aircraft production setbacks, the acquisition of mature training system devices and delays to approvals for construction of facilities;
  • Hw Torpedo—slippage resulting from changes to the Full Cycle Docking schedule affecting the installation schedule based on submarine availability;
  • Battle Comm. Sys.—minor delay resulting from the certification of the achievement of FOC; and
  • LHD Landing Craft—delays resulting from the supply of supporting products, such as training and spare parts being delivered to the contracted schedule rather than earlier than contracted as reported in the 2013–14 report.

Figure 11: In-year (2014–15) schedule changes to achieving FOC

Note: Defence’s PDSSs indicate that 14 of the 25 Major Projects Report projects did not record changes to their Final Operational Capability dates this year.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Refer to paragraphs 2.47 to 2.48 in Part 1 of this report.

Longitudinal schedule performance

2.49 Figure 12, below, shows the schedule slippage of the Major Projects included in the 2007–08 to 2014–15 reports (Table 7 provides the details of the projects included in the analysis). The figure shows that the total cumulative schedule slippage for the:

  • four projects in 2007–08 is 11.3 years or 135 months over seven years to 2014–15;
  • eight projects in 2008–09 is 18.0 years or 216 months over six years to 2014–15;
  • 13 projects in 2009–10 is 24.2 years or 290 months over five years to 2014–15;
  • 17 projects in 2010–11 is 23.0 years or 276 months over four years to 2014–15;
  • 18 projects in 2011–12 is 20.5 years or 246 months over three years to 2014–15;
  • 19 projects in 2012–13 is 15.0 years or 180 months over two years to 2014–15; and
  • 25 projects in 2013–14 is 3.4 years or 41 months over one year to 2014–15.

2.50 The figure also shows that 34.6 per cent (22.2 years or 266 months) of the total schedule slippage across the Major Projects covered in the 2014–15 report (64.0 years or 768 months) is made up of the slippage from the four remaining projects reported in the 2007–08 report.

Figure 12: Longitudinal schedule slippage across years (in years)

Note 1: The total schedule slippage in 2014–15 across the 25 projects is 768 months. P-8A Poseidon and Maritime Comms, which are new to this year’s Major Projects Report, have not experienced slippage against Final Operational Capability (FOC) according to Defence’s PDSSs.

Note 2: Bushmaster Vehicles has an FOC date for each Production Period (discrete order). The FOC used for this year’s Major Projects Report analysis is Production Period Five.

Source: The ANAO’s analysis of the PDSSs in published Major Projects Reports.

Table 7: Projects included in Figure 12 analysis by Major Projects Report

Project

2007–08

2008–09

2009–10

2010–11

2011–12

2012–13

2013–14

2014–15

Joint Strike Fighter

 

 

 

Y

Y

Y

Y

Y

AWD Ships

 

Y

Y

Y

Y

Y

Y

Y

P-8A Poseidon

 

 

 

 

 

 

 

Y

Wedgetail

Y

Y

Y

Y

Y

Y

Y

Y

MRH90 Helicopters

 

Y

Y

Y

Y

Y

Y

Y

Growler

 

 

 

 

 

 

Y

Y

MH-60R Seahawk

 

 

 

 

Y

Y

Y

Y

Overlander Medium/Heavy

 

 

 

 

 

 

Y

Y

LHD Ships

 

Y

Y

Y

Y

Y

Y

Y

ARH Tiger Helicopters

Y

Y

Y

Y

Y

Y

Y

Y

Air to Air Refuel

 

Y

Y

Y

Y

Y

Y

Y

Battlefield Airlifter

 

 

 

 

 

 

Y

Y

Bushmaster Vehicles

Y

Y

Y

Y

Y

Y

Y

Y

Overlander Light

 

 

Y

Y

Y

Y

Y

Y

ANZAC ASMD 2B

 

 

Y

Y

Y

Y

Y

Y

Additional Chinook

 

 

 

Y

Y

Y

Y

Y

Battle Comm. Sys. (Land)

 

 

 

 

 

Y

Y

Y

Collins RCS

Y

Y

Y

Y

Y

Y

Y

Y

Maritime Comms

 

 

 

 

 

 

 

Y

Hw Torpedo

 

 

Y

Y

Y

Y

Y

Y

UHF SATCOM

 

 

 

Y

Y

Y

Y

Y

Collins R&S

 

 

Y

Y

Y

Y

Y

Y

ANZAC ASMD 2A

 

 

Y

Y

Y

Y

Y

Y

Battle Comm. Sys.

 

 

 

Y

Y

Y

Y

Y

LHD Landing Craft

 

 

 

 

 

 

Y

Y

Source: The ANAO’s analysis of the PDSSs in published Major Projects Reports.

2.51 Further disaggregation according to a project’s Second Pass Approval date in Table 8, below, shows that 73 per cent (2013–14: 80 per cent) of the total schedule slippage across the 2014–15 Major Projects is made up of projects approved prior to the DMO’s demerger from the Department of Defence, in July 2005. This is a positive indicator of the benefits that the then DMO, as a specialist acquisition organisation, was able to bring to complex Defence procurement and the impact on schedule performance (and cost) during the transition to higher levels of MOTS acquisitions following the Defence Procurement Review 2003 (Kinnaird Review).106

Table 8: Project slippage by Project approval

Project

No. of months between Approval and Original FOC date

No. of months between Approval and 30/6/15 FOC date

No. of months slippage between Original FOC and 30/6/15 FOC date

Projects Approved pre July 2005

Wedgetail

96

173

781

ARH Tiger Helicopters

123

202

79

Air to Air Refuel

94

156

62

Bushmaster Vehicles

217

217

0

Collins RCS

88

197

109

Hw Torpedo

148

211

63

Collins R&S

165

260

991

ANZAC ASMD 2A

97

167

721

Sub Total – Projects Approved pre July 2005

1 028

1 583

562

Percentage of Total – Projects Approved pre July 2005

38%

46%

73%

Projects Approved post July 2005

Joint Strike Fighter

169

169

21

AWD Ships

131

165

34

P-8A Poseidon

71

71

0

MRH90 Helicopters

119

179

60

Growler

111

111

0

MH-60R Seahawk

150

150

0

Overlander Medium/Heavy

125

116

21

LHD Ships

113

113

0

Battlefield Airlifter

68

77

9

Overlander Light

54

58

91

ANZAC ASMD 2B

90

145

571

Additional Chinook

83

83

0

Battle Comm. Sys. (Land)

55

59

4

Maritime Comms

125

125

0

UHF SATCOM

111

111

0

Battle Comm. Sys.

41

65

24

LHD Landing Craft

53

53

51

Sub Total – Projects Approved post July 2005

1 669

1 850

206

Percentage of Total – Projects Approved post July 2005

62%

54%

27%

Total – All Projects With Slippage

2 697

3 433

768

Note 1: These figures do not add precisely due to the exclusion of schedule reductions over the life of the project.

Source: The ANAO’s analysis of the 2014–15 PDSSs. Refer to paragraph 2.51 in Part 1 of this report.

Capability performance analysis

2.52 The Australian Defence Force defines capability as the power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.107 An operational effect is achieved by combining the eight Fundamental Inputs to Capability: personnel; organisation; collective training; major systems; supplies; facilities and training areas; support; and command and management.108

2.53 In acquiring Defence platforms and systems, a range of documentation (including capability definition, operational concept, function and performance specification, and test concept documents) is developed and sets out the detailed requirements/performance attributes to be achieved. In the case of an aircraft, for example, this would include elements such as its range and speed, handling characteristics and self protection abilities. Depending on the nature of this information, it can be classified for national security reasons and therefore cannot be made public.

2.54 Assessment of capability performance (Expected Capability) is measured against the Materiel Release Milestones (MRMs) and Completion Criteria specified in each project’s Materiel Acquisition Agreement (MAA). The exception is Wedgetail, where the assessment is against the Supplies section of the MAA which lists the equipment to be delivered, consistent with prior year reports. However, these measures do not encompass the full delivery of capability, as they were designed to reflect the subset of the then DMO’s undertakings in relation to project acquisition.

2.55 In general, MRMs and Completion Criteria (previously Measures of Operational Effectiveness) were designed to define, at a strategic level, the key elements required to achieve IMR and FMR.109 They focus on the achievement of technical, regulatory or operational requirements to be delivered by the then DMO. Where key requirements are not achieved, this could have a significant effect on a system’s likely suitability for acceptance into operational service.

2.56 The assessment against MRMs is based on a forecast of future events. In the JCPAA’s Report 442, the Committee recommended that the ANAO and the DMO consult to apply a more objective method of assessing capability performance and distinguish capability achieved, capability yet to be achieved, capability unlikely to be achieved and capability exceeded.110

2.57 However, while the then DMO advised there was no system that universally tracks the progress of inputs to capability111, it has included information regarding the key elements which constitute IMR and FMR, as stipulated in the MAA for each project, in the PDSSs for the 2014–15 report. These changes have assisted in providing greater clarity regarding the proportion of capability delivered at these two key milestones.

2.58 The ANAO continues to observe that there is considerable diversity across the projects in the number, level of specification, articulation and focus of MRMs and Completion Criteria in project MAAs. For example, AWD Ships (total budget $7.9 billion) has four MRMs and Completion Criteria, while ANZAC ASMD 2A (total budget $0.4 billion) reports on 15 MRMs and Completion Criteria. To enable appropriate accountability and performance reporting, the number and level of specificity of performance measures should encompass all key components of acquisition, not just the finalisation of each platform (and supplies) as is the case in AWD Ships.

2.59 The MRMs and Completion Criteria are assessed using traffic light indicators by the project managers, and reported monthly within the Australian Defence Organisation. The June 2015 traffic light assessment is represented in the PDSSs as Measures of Materiel Capability Delivery Performance. Due to national security considerations, only the overall status from each project office’s assessment of the likelihood of delivering the required materiel capability is disclosed in the 2014–15 Major Projects Report and earlier reports.

2.60 However, these results at times contrast with other available information and highlight the difficulties in estimating future capability. For example, the PDSS does not disclose the capability impacts related to technological obsolescence and Rate of Effort (flying hours) in the case of the ARH Tiger Helicopters project, which are caused by delays in delivery and the availability/quality of the helicopters. In addition, the PDSS for the LHD Landing Craft project does not disclose the integration issues for introduction into service accepted by Chief of Navy, which are due to risks that were not mitigated through the acquisition phase of the project.

Expected Capability and Project Maturity

2.61 Figure 13, below, sets out Defence’s assessment of the likelihood of delivering all of the key capabilities (Expected Capability) and Project Maturity.112 Defence expects that all 25 projects in this year’s report will deliver all of their key capability requirements, recognising that some elements of the capability required for some projects may be under threat but considered manageable (assessed as either green or amber).

2.62 The complexity and type of acquisition (MOTS, Australianised MOTS or developmental) are significant factors affecting the Expected Capability; the risk is appreciably higher for more developmental projects. Defence’s increased expectations of delivery of key capabilities is in line with the increase in MOTS or Australianised MOTS projects selected for acquisition in recent years. Notwithstanding, the following analysis on pages 61 to 63, highlights a prediction of 100 per cent capability delivery and an ongoing trend of improvement, in contrast to the issues noted at paragraph 2.60. To that end, in some circumstances, Defence’s assessment of expected capability delivery is overly optimistic, which impacts the following analysis.

2.63 In addition, the project’s current stage within the capability development and acquisition life cycle has an impact on the maturity score provided by Defence, with all variances greater than 20 per cent reflecting projects that are either at an earlier stage of the life cycle or have experienced schedule delays, which impacts on the ability of the project to gain maturity. Defence’s assessment of Expected Capability should become better informed as a project matures along its capability development and acquisition life cycle.

Figure 13: Project snapshot—Expected Capability and Project Maturity

Note: ANZAC ASMD 2B’s Project Maturity is based on the progress of the lead ship, not on the current eight ship program.

Source: 2014–15 PDSSs. Analysis for the 2014–15 Major Projects Report continues to highlight inconsistencies within the application of project maturity, reducing the level of reliability of maturity assessments for key decision makers and other stakeholders; however, improved focus and review by project offices was noted by the ANAO during 2014–15 fieldwork. Refer to paragraphs 2.61 to 2.63 in Part 1 of this report.

Capability performance

2.64 Figures 14 and 15, below, present Defence’s assessment of the percentage of materiel capability delivery that:

  • has a high level of confidence will be met (green);
  • are under threat but still considered manageable (amber); and
  • at this stage are unlikely to be met (red).
In-year capability performance

2.65 Figure 14, below, highlights the assessment that there is one project facing challenges in delivering elements of their system’s planned capability.113 This shows an improvement from 2013–14, where five projects had challenges in this area. The project currently highlighted is:

  • Joint Strike Fighter—where there are a number of risks to achieving some of the materiel capabilities required for Initial Operational Capability (IOC) and FOC including final aircraft software builds meeting IOC and FOC requirements and establishing the sustainment of the aircraft.

Figure 14: Expected Capability at 30 June 2015

Note 1: The Expected Capability assessment for Wedgetail has been against the Supplies section of the MAA, which lists the equipment to be delivered.

Note 2: Expected Capability is as per paragraph 2.2 in Part 1 of this report.

Source: 2014–15 PDSSs. Refer to paragraph 2.65 in Part 1 of this report.

Longitudinal capability performance

2.66 A multi-year comparison of Expected Capability should be considered with caution as a project’s Measures of Materiel Capability Delivery (MRMs and Completion Criteria) can change from year to year as Materiel Acquisition Agreements are updated. Therefore, any comparison of an individual or a group of projects’ Measures of Materiel Capability Delivery may not involve comparing ‘like with like’. Changes in anticipated capability outcomes should be read in conjunction with the information in the PDSSs.

2.67 Figure 15, below, examines the assessment of Expected Capability for the same projects from 2008 to 2015 as reported in the PDSSs using traffic light indicators as defined above.114

2.68 As reflected in the portfolio of projects in the 2014–15 report, the assessment is comparable to the 2013–14 report, that it has a ‘high level of confidence’ in delivering 97 per cent (2013–14: 96 per cent) of the key capabilities associated with the Major Projects in this report. While the delivery of the remaining three per cent (2013–14: four per cent) of the key capabilities is considered to be ‘under threat’, the risk is still considered to be ‘manageable’.

Figure 15: Expected Capability across the 2007–08 to 2014–15 Major Projects Reports

Note 1: * The ANAO did not examine the accuracy of the recording of this data in previous Major Projects Reports.

Note 2: ^ Super Hornet did not have Measures of Operational Effectiveness, but does have Measures of Materiel Capability Delivery Performance in the latest MAA.

Note 3: # Hornet Refurb was removed from the PDSSs in 2011–12.

Note 4: ∆ Joint Strike Fighter is excluded from this analysis due to a lack of data.

Note 5: Ω C-17 Heavy Airlift was removed from the PDSSs in 2012–13.

Note 6: ∞ Super Hornet was removed from the PDSSs in 2013–14.

Note 7: √ Armidales was removed from the PDSSs in 2013–14.

Note 8: ∑ C-RAM was removed from the PDSSs in 2013–14.

Note 9: † HF Modernisation, FFG Upgrade and Hornet Upgrade were removed from the PDSSs in 2014–15.

Note 10: Ɏ Next Gen Satellite and Stand Off Weapon were both removed from the PDSSs in 2014–15.

Note 11: Ⱡ 155mm Howitzer and SM-2 Missile were both removed from the PDSSs in 2014–15.

Note 12: The Expected Capability assessment for Wedgetail has been against the Supplies section of the MAA, which lists the equipment to be delivered.

Source: PDSSs in published Major Projects Reports. Refer to paragraphs 2.66 to 2.68 in Part 1 of this report.

3. Developments in Acquisition Governance

Introduction

3.1 Major Defence equipment acquisition projects (Major Projects) are large, complex, high-cost procurement activities, characterised by risk and long timeframes between concept and acceptance into service. Combined with varying degrees of required interoperability, these characteristics pose significant challenges to the governance of projects and highlight the importance of applying a robust governance and performance information framework. Such frameworks have the capacity to support project offices during acquisition and provide for the consistent assessment of a project’s progress and the transparency and accountability across Major Projects as a whole.

3.2 As noted earlier, the former Defence Materiel Organisation (DMO) was delisted from 1 July 2015, and its functions merged back into the Department of Defence, as part of the implementation of the recommendations from the First Principles Review: Creating One Defence (First Principles Review).115 The newly formed Capability Acquisition and Sustainment Group will now manage the process of bringing new capabilities into service including Fundamental Inputs to Capability, for example the provision of personnel, training and command.116 While progress on the implementation of the First Principles Review recommendations is ongoing117, the Australian National Audit Office (ANAO) will continue to review the governance over Defence acquisition, including Defence’s review of prior DMO policy and its application within the new Defence structure.

3.3 In addition to the key acquisition governance aspects discussed in Chapter 1 of Part 1 (financial assurance, contingency management, enterprise risk management and project maturity), the ANAO’s review included assessing specific developments in: Gate Review Boards; Projects of Concern; Quarterly Project Performance Reporting; Joint Project Directives; business systems rationalisation; and project management and skills development, to gain a greater understanding of the Capability Acquisition and Sustainment Group’s business and to assist in planning the approach to the review.

Additional areas of review focus—Acquisition Governance

3.4 Part of the ANAO’s planning for the Major Projects Report (MPR) each year focuses on reviewing the processes, including developments, that contribute to the overall governance of Major Projects. This informs the ANAO’s assessment of the governance initiatives supporting the PDSSs for the review period, and highlights areas in those frameworks that could also be the subject of the ANAO’s future focus.

3.5 These include:

  • the Gate Review process, which is intended to provide the Defence Senior Executive with assurance that projects and products will deliver approved objectives and focus on preparedness to progress to the next stage of activity;
  • the Projects of Concern process, which is designed to address project issues, which are of concern to the Senior Executive and government, relating to cost, schedule and capability;
  • the Quarterly Project Performance Report, which is designed to identify emerging issues and risks facing projects, for remediation;
  • the introduction of a requirement for Joint Project Directives118 in 2009–10, for all projects approved by government from 1 March 2010, to capture the requirements from government approval;
  • business systems119 rationalisation, which is aimed at consolidating processes and systems in order to provide a more manageable system environment; and
  • the project skills professionalisation and development program within Defence and industry support, which is directed at enhancing the skill sets available to manage Defence’s Major Projects.

Gate Review Boards

3.6 The Gate Review (acquisition) process120 was designed to provide assurance that identified risks for a project were manageable, and that costs and schedule were under control, prior to a project passing the various stages of its life cycle. In practice, Gate Reviews (acquisition) involve a periodic assessment of a project in advance of key ‘gates’121 by a Gate Review (acquisition) Board, appointed by Defence122, to provide the Defence Senior Executive with assurance that project offices will deliver on government approved objectives, and focusing on the project office’s preparedness to progress to the next stage of activity, ability to deliver the required capability, and the continued validity of projects’ business cases.123

3.7 Gate Reviews were first held in 2008, during the period of the Mortimer Review, Going to the Next Level: the report of the Defence Procurement and Sustainment Review,124 which believed that the Gate Review process should be expanded. The Mortimer Review recommended establishment of an Independent Project Performance Office (IPPO)125 to oversee project reviews. IPPO also facilitates the Projects of Concern process (discussed further at paragraphs 3.13 to 3.20), and is implementing the Quarterly Project Performance Report (discussed further at paragraphs 3.21 to 3.24).126

3.8 When first introduced, Defence’s Gate Reviews (acquisition) covered four specified gates (First Pass consideration; Second Pass consideration; Contract Solicitation; and Contract Negotiation).127 Gate Reviews can also be undertaken in response to government or management concerns, or to enlist senior management assistance to solve significant issues.128 On completion of reviews, Gate Review Boards report on the health and outlook of a project, and its readiness to proceed to the next stage of activity.129

3.9 In addition, in 2013–14, three new Gate Reviews were introduced for the Project Initiation; Acquisition and Support Concept (pre First Pass); and Sustainment stages130, with a Sustainment Gate Review conducted on three of the projects in the 2014–15 report.131 Previously, project offices that had undergone a Gate Review in the lead up to Final Materiel Release (FMR) did not undergo further Gate Reviews, despite residual issues.132 However, Sustainment Gate Reviews are able to provide continuing oversight by bringing a valuable perspective to the consideration of key issues and risks in reviewing the health and outlook of sustainment capabilities.133

3.10 The increased scope of the Gate Review Program is consistent with the continued confidence of the Senior Executive that Gate Reviews are assisting project offices to manage.134 IPPO undertakes systematic surveys of the views of officers involved in Gate Reviews, and the 2014–15 results confirmed to IPPO that the program is generally viewed to provide benefit to the project/s.135 However, as identified in the ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, tabled in June 2012, an objective assessment of the program’s success is difficult to measure due to the qualitative nature of Gate Reviews and the lengthy project lifecycles.136

3.11 During 2014–15 the ANAO accessed the Gate Reviews (acquisition) and their outcomes where available, to gain additional information on projects’ progress and assurance that the information in this year’s PDSSs was complete and accurate. Of the Gate Reviews (acquisition) conducted, 17 were on 16 of the projects in the 2014–15 report137 (2013–14: 22 MPR projects were subjected to Gate Reviews). However, Defence has now formalised the practice where projects deemed ‘low risk’ as a result of preliminary IPPO analysis, are not subject to a Gate Review. For example, in 2014–15, the Collins R&S project did not proceed to a Gate Review.

3.12 The First Principles Review report released on 1 April 2015 recommended that IPPO be significantly enhanced and strengthened to provide a robust and disciplined contestability function.138 The ANAO appreciates that the Gate Review process is continuing to evolve and will monitor its progress in 2015–16.

Projects of Concern

3.13 The Projects of Concern list was established in 2008 to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. Projects identified as having significant issues that are beyond the normal capacity of a project team to manage, may be recommended by the Defence Senior Executive for inclusion on the list. Defence then makes a recommendation to the Minister for Defence for final determination.139

3.14 Projects are removed from the Projects of Concern list once government is satisfied that remediation activity has been completed successfully, or government has decided to cancel the project.140 Project of Concern summits141, implemented in February 2011 to further increase focus from government, Defence and industry, to remediate Projects of Concern, have recommenced with the most recent summit held in July 2015. Project of Concern reports continue to be provided to the Minister for Defence on a monthly basis.

3.15 The IPPO, who is responsible for overseeing and administering the remediation process for all Projects of Concern, has advised the ANAO that updated policy and procedures for Projects of Concern remain in draft.142

3.16 During 2014–15, three MPR projects were continuing Projects of Concern, these being AWD Ships, MRH90 Helicopters and Air to Air Refuel. However, in March 2015, Defence announced that the Air to Air Refuel project was to be removed from the list following formal acceptance of the Aerial Refuelling Boom System capability.143 In June 2014, it was announced144 that the AWD Ships project was being placed on the list to better address the increasing commercial, schedule and cost risks identified in ANAO Report No.22 2013–14, Air Warfare Destroyer Program, tabled in March 2014, and following the earlier announcement of a government review.145 The project was added as a Project of Concern to ‘ensure that this important project is delivered successfully’.146 As at June 2015, the AWD Reform program was underway.147

3.17 In May 2013, the then Government announced that the MRH90 Helicopters project would be considered for removal from the Projects of Concern list by the end of 2013, following the signing of a Deed of Variation (Deed 2), to the original contract, to address commercial, technical and schedule issues.148 As at June 2015, the MRH90 Helicopters project is still listed as a Project of Concern149 as remediation actions are yet to be completed.150

3.18 As noted by the Australian Strategic Policy Institute in August 2013:

The introduction in 2008 of the ‘Projects of Concern’ management process has seen some projects that faced serious challenges remediated (and in a couple of instances terminated).151

3.19 Since the Projects of Concern process was established, 10 MPR projects have been on the list and eight have been removed due to government decisions that the projects have been remediated.152 The eight projects considered remediated are: Wedgetail; Overlander Medium/Heavy; ARH Tiger Helicopters; Air to Air Refuel; FFG Upgrade; ANZAC ASMD 2B; HF Modernisation; and Stand Off Weapon.

3.20 While specific concerns may have been addressed for some ‘remediated’ projects, it should be noted that residual or emergent risks and issues exist post FMR and may even impact the achievement of Final Operational Capability (FOC). For example, while the ARH Tiger Helicopters project was removed from the Projects of Concern list in April 2008153, it only achieved FMR with significant caveats in March 2014154, and has yet to achieve FOC (currently 79 months behind schedule). Similarly, and as noted previously, FFG Upgrade and HF Modernisation have still not reached FOC. More robust and objective capability assessments by Defence across the life of Major Projects, from Second Pass Approval to Final Operational Capability, would better support the Projects of Concern governance process.

Quarterly Project Performance Report

3.21 In May 2011, the then Government announced that it would implement an Early Indicators and Warnings (EI&W) system designed to help identify and correct potential problems in the formative stages of the project life cycle.155 In December 2013, the then DMO advised that while the EI&W reporting system had provided benefits across Defence, it was no longer providing the required outcomes in its current form.156 Consequently, the Quarterly Project Performance Report (QPPR) was implemented in 2014.

3.22 The QPPR aims to provide senior stakeholders within government and Defence with a clear and timely understanding of emerging issues and risks in the delivery of capability to end-users, with the focus on highlighting underperforming projects.157 Defence has advised that analysis and data for the QPPR are sourced from existing internal reporting systems including Projects of Concern, the Monthly Reporting System and other assessment tools such as Gate Review Boards. The report is submitted to the Minister for Defence on a quarterly basis.

3.23 During 2014–15, three projects from the MPR were identified as key underperforming projects, for internal management attention within Defence:

  • Battlefield Airlifter, due to uncertainties arising from the US Air Force leaving the C-27J program (a risk known at Second Pass Approval), which has also impacted on the amount of effort and resources required to deliver project outcomes;
  • Overlander Light, due to delays caused by the definition and achievement of the Ambulance and Command Post module requirements, however, by June 2015 this project was removed as an underperforming project; and
  • UHF SATCOM, due to ongoing issues with the modification of Commercial-Of-The-Shelf software (now considered developmental), which has caused a further slip of 38 months to FMR from May 2015 to July 2018.

3.24 The ANAO appreciates that the QPPR process is still relatively immature and will monitor its progress in 2015–16.

Joint Project Directives

3.25 Joint Project Directives (JPDs) were introduced by Defence for all projects approved by government from 1 March 2010 to identify the scope and limits of acquisition projects and the party responsible for the major components of the work.158 They are also intended to be an authoritative document which enables any other necessary project documentation, including Materiel Acquisition Agreements (MAAs)159, Capability Definition Documents and acquisition strategies to be based on a reliable source. Therefore, JPDs and MAAs are intended to aid sound governance and accountability in Defence acquisition projects.160

3.26 During the 2014–15 review, the ANAO continued to examine a range of governance documents which define project deliverables, with the Capability Acquisition and Sustainment Group currently providing JPDs for all of the eleven MPR projects approved from 1 March 2010.161 An area of focus is the completeness and accuracy of the JPDs, with some difficulties associated with recording the full detail of government approvals.

3.27 The ANAO will continue to take JPDs into account in its review program in future years, where these have been prepared. With another three post 2010 projects to be included in the 2015–16 report, better alignment with government approved requirements is expected and an initial internal audit would assist in determining that the predefined impacts of JPDs are being met.

Business systems rationalisation

3.28 In previous Major Projects Reports, the ANAO reported that the control environment of each project differed, due to the large range of corporate and project management Information Technology (IT) applications being employed by the different project offices. During the 2014–15 review, the same observations apply across the 25 Major Projects. This has again resulted in an inconsistency between the information produced by each of the project’s IT systems (i.e. risk management, financial management, and document management systems) and highlights an issue for Defence in ensuring reliable and consistent information to properly inform project management and decision making. However, the Capability Acquisition and Sustainment Group’s Business Information Management Branch advised that progress is being made in this area through the development of an Integrated Project Management System (IPMS) which was being progressively built into the then DMO corporate data warehouse in 2014–15. The ANAO was advised that the IPMS interfaces with current IT systems to provide reports to management drawn from the Monthly Reporting System, Project Performance Reporting System, and Predict! data.

3.29 In its Report 436, Review of the 2011–12 Defence Materiel Organisation Major Projects Report, the Joint Committee of Public Accounts and Audit (JCPAA) expressed its ongoing concern with the consistency of information in the then DMO’s business systems, and recommended that ‘the DMO develop a business systems improvement plan which prioritises projects, assigns completion dates and allocates senior level ownership for implementation. A progress update on achievements against the plan should be included in the 2012–13 Major Projects Report’.162

3.30 The new Capability Acquisition and Sustainment Group has since advised of the development of an Information Management (IM) Investment Plan to prioritise investment in business systems and other IT applications and the development of an overarching IM Framework and Strategic Plan, including an IM Governance Model. However, the ANAO was advised that the group’s work on the IM Investment Plan, and IM Framework and Strategic Plan was suspended with the implementation of the First Principles Review. The Chief Information Officer Group has taken the lead in developing an IM framework, with the Capability Acquisition and Sustainment Group providing input.

3.31 In 2012–13 the then DMO advised that there were 540 applications in use, and that the organisation was identifying and confirming where duplicate licenses and other inefficiencies existed, as well as researching software and systems, in order to achieve greater visibility over Information and Communications Technology expenditure. As a result of this process, the total number of applications in use as at June 2014 was reduced to approximately 300. During 2014–15, post remerging into Defence, the Capability Acquisition and Sustainment Group advised that 3 393 separate applications were identified as in use, with 1 108 recommended for removal.163

3.32 As part of a program to rationalise its information and communications technology systems, the single records management system (Objective) was rolled out in 2013, which centralised the then DMO’s electronic document management.164 Guidance and training is being offered to all employees, along with regular Communities of Practice on records management, in order to ensure expected National Archives and Defence records management obligations are met. The group advised in 2014–15 that staff familiarity and use of Objective is improving.

Project management and skills development

Within Defence

3.33 Key challenges for both Defence and the Australian Defence Industry include improving the project management, scheduling, logistics, procurement and engineering services provided to the Australian Government, within current and future workforce constraints. Previously, the then DMO’s professionalisation initiative was a central strategy in meeting those challenges into the future165, as well as to attract and retain a skilled workforce.

3.34 To assist with the professionalisation of staff, the then DMO’s Directorate of Professionalisation Policy was responsible for the development of certification programs that focused on developing specific competencies and gaining professional qualifications. This supported the core business of acquisition and sustainment across the Procurement, Financial Management, Project and Sustainment Management, Materiel Engineering and Materiel Logistics streams.

3.35 In November 2012, the then DMO began the transition of personnel and responsibilities to the Defence Learning Branch as part of a movement to a shared services arrangement. With the delisting of the DMO from 1 July 2015 and the merging of the DMO’s functions back into the Department of Defence, greater emphasis has been placed on the One Defence system, with the First Principles Review noting:

It is critical for the development of the One Defence system that the shared services model is fully implemented. This will mean standardising services, removing duplication of functions, professionalising the workforce and ensuring there are single, clear lines of ownership and accountability… We recommend that the service delivery reform program, including full integration of the current Defence Materiel Organisation corporate functions, be completed.166

3.36 Additionally, in regard to a plan to deliver a professional workforce, the First Principles Review noted:

While the Navy, Army and Air Force have detailed workforce plans, Defence does not have a strategic workforce plan for its enabling workforce. Without it, Defence struggles to identify skills gaps across the agency and place the right people with the right skills in the right roles at the right time to deliver Defence’s mission.

A strategic workforce plan is a necessary prerequisite for building a highly professional workforce with the necessary skills and balance of public servants and Australian Defence Force personnel.167

3.37 Based on this, the First Principles Review recommended:

… Defence build a strategic workforce plan for the enabling functions, and incorporate workforce plans for each job family in order to drive recruitment, learning and development, performance and talent management.168

3.38 The new Capability Acquisition and Sustainment Group’s Directorate of Professionalisation Policy advised that the administrative function for several courses are yet to transition to the Defence Learning Branch, with the Directorate providing a supporting role to ensure the group’s professionalisation requirements continue to be met. The Directorate also advised that the transition of responsibility will continue under the First Principles Review’s renewed focus, noting that First Principles Review implementation plans are still being established.

3.39 The Directorate advised in 2014–15 that approximately 1 077 Capability Acquisition and Sustainment Group staff have either been certified or are enrolled in a certification program with a professional body compared with 1 050 staff in 2013–14.169 The ANAO will continue to monitor the transition of responsibility from the group’s Directorate of Professionalisation Policy to the Defence Learning Branch during 2015–16.

Industry support

3.40 To assist with the development of participants within the Defence Industry, the Skilling Australia’s Defence Industry (SADI) program was established by the Australian Government in 2005. The aim of the SADI program is to up-skill existing employees, improve the quality and quantity of skills training, and provide funding support to the Defence industry for training activities where there is an identified skills shortage in technical, trade and professional skill sets.170

3.41 The SADI program provides funding support to companies and industry associations for training and skilling activities where that training is linked to a Defence capability. The Capability Acquisition and Sustainment Group advised that since 2005, the program has funded more than 30 000 training places. Also, approximately 200 industry participants have been provided with funding support in trade, technical and other professional skill sets.171 To meet the needs of the Australian Defence Force, SADI aims to invest $215 million172 within the period 2005–06 to 2015–16, with more than $5.6 million173 allocated in 2015–16 under one round of the SADI program, to boost the skills of workers in the Defence industry.174

3.42 For 2015–16, administration and delivery of the SADI Grant Program has been transferred to the Department of Industry, Innovation and Science, who will undertake this on behalf of Defence. This change in responsibility has required the transfer of documentation, policies and procedures along with relevant training. The Department of Industry, Innovation and Science will continue to report to Defence on the status of the program.

3.43 Additionally, the 2015–16 SADI Program Guidelines have been updated and now include reworked merit criteria (eligibility requirements remain unchanged) which include five equally weighted criteria to prioritise funding allocations175: support to Defence capability; meeting programme objectives; value for money; strategically targeted skill sets; and the extent to which applicants are Australian small and medium enterprises.

3.44 The SADI program is currently due to close following the 2015–16 round. The ANAO will continue to monitor the program during this time, noting that Defence is expected to release a Defence Industry Policy Statement in late 2015, with an ANAO performance audit currently underway.

3.45 Other government approved programs aimed at up-skilling participants within the Defence industry include the Industry Skilling Program Enhancement (ISPE) package and the Priority Industry Capability Innovation Program (PIC IP). ISPE, which was announced in 2008, provides up to $60.8 million.176 In 2012–13, the then DMO advised that there was no funding programmed past 30 June 2014. Since then, the group has advised that some initiatives under this program have been extended until the new Defence Capability Plan, Defence White Paper and Defence Industry Policy Statement have been prepared and released (expected in early 2016). This program incorporates a number of initiatives aimed at enhancing work and future career pathways in the sector by implementing programs at schools and offering Defence engineering internships.177

3.46 PIC IP, which was implemented in 2011–12, provides up to $45 million over seven years178, and is aimed at providing Australian Defence companies direct support in the form of repayable and matched grants (the recipient matches the funds provided by the Australian Government on a dollar for dollar basis).179 The program particularly focuses on assisting small to medium sized enterprises to pursue innovative Defence industry projects.

3.47 The Capability Acquisition and Sustainment Group has since advised that the PIC IP will cease in 2017 with no further funding rounds to be offered under this program, and the PIC Development Fund (PIC DF) is planned to be instituted in its place. It is currently anticipated that PIC DF will be funded from the Defence Capability Plan and will be administered by the new group, with additional oversight responsibility to be confirmed. However, this transition is still in development.

3.48 In addition, on 5 September 2011, the then Minister for Defence Materiel underlined the need for the replacement or upgrade of up to 85 per cent of its military equipment over the next 15 years, and asked Skills Australia to work with Defence and the Defence industry to develop a plan to ensure Australian industry has the skills to meet this challenge.180 Skills Australia noted the following in their September 2012 report, Building Australia’s Defence Supply Capabilities – Main Report for the Defence Industry Workforce Strategy:

…the current skilling programs offered by the DMO suffer from a lack of longitudinal evaluation processes or strategies in place to assess the effectiveness of these programs. The lack of these mechanisms for either SADI or ISPE means that the DMO and Government are not in a position to determine when success is achieved in these programs and whether they have provided value for money.181

3.49 Skills Australia made three recommendations regarding this issue aimed at undertaking performance evaluations of SADI and ISPE, developing key performance indicators for ISPE, and incorporating performance measures and reporting requirements into SADI funding agreements.182 In response, the then DMO advised that they have considered outcome focused reporting for new and upcoming programs however, they advised that it would be onerous to re-establish reporting requirements for existing programs in the current climate.

3.50 In June 2014, the Productivity Commission released the Trade & Assistance Review 2012–13 which also commented on Defence industry assistance programs, noting that:

… defence industry assistance programs do not appear to have been reviewed (externally and using an economic framework) to the same extent as many other industry assistance programs… All defence programs with material industry assistance objectives, explicit or not, should be separately reported to Parliament, and from time to time reviewed to ensure relevance to strategic and government priorities.183

3.51 The Capability Acquisition and Sustainment Group advised that reviews of industry assistance programs will likely be encapsulated in the new Defence Capability Plan, Defence White Paper and Defence Industry Policy Statement (expected in early 2016). The ANAO will continue to monitor the SADI and other industry assistance programs in 2015–16. Further information about the programs can be found in the upcoming ANAO performance audit on Defence Industry Support and Skill Development Initiatives, due to be tabled in January 2016.184

!Part 2. Defence Major Projects Report

Secretary of Defence Foreword

I am pleased to present the 2014-15 Major Projects Report (MPR), which reports on 25 major Defence equipment acquisition projects (major projects) managed by the Defence Materiel Organisation185 (DMO) – now the Capability Acquisition and Sustainment Group (CASG).

Since last year, seven major projects have been removed following achievement of Final Operational Capability (FOC) or based on a post-Final Materiel Release (FMR) risk assessment of the timely achievement of FOC by the Capability Manager within the Department of Defence (Defence). These projects are the F/A-18 Hornet Upgrade; Guided Missile Frigate Upgrade; Next Generation Satellite Communications Capability; High Frequency Modernisation; SM-1 Missile Replacement; Artillery Replacement; and, Follow-on Stand Off Weapon. Two new major projects included this year are Maritime Communications Modernisation and Maritime Patrol and Response Aircraft (Boeing P-8A Poseidon).

The 25 major projects reported this year have a combined budget of more than $60 billion. The MPR is an authoritative source of quantitative data that has been assured by the Australian National Audit Office (ANAO) and increases Defence’s transparency to the Parliament on project performance.

As demonstrated in this report, major Defence equipment acquisition project financial and capability delivery performances are well ahead of our allies and other sectors. While I note that schedule performance has improved over time, more rigour in schedule estimation and other measures will need to be implemented to improve on-time delivery.

The format of this MPR remains largely the same as last year. Chapter 1 provides an overview of the performance of the DMO (now CASG) and discusses key issues, such as finance, schedule and risk management. Chapter 2 provides a detailed analysis of the schedule, cost and delivery of the materiel elements of capability for the 25 major projects.

In 2015-16, Defence will continue to implement reform across all elements of its business operations. With the implementation of the First Principles Review of Defence underway, I look forward to greater opportunities for improvements to manage and deliver major projects.

The Chiefs of Navy, Army, Air Force, the Chief Information Officer and the major contractors for each project have reviewed the relevant project data and their views have been considered in finalising this report.

Defence and the ANAO continue to have a strong professional working relationship and have further improved the MPR consistent with guidance provided by Parliament’s Joint Committee of Public Accounts and Audit.

I thank the outgoing Auditor-General, Mr Ian McPhee, and the incoming Auditor-General, Mr Grant Hehir, and their staff for their contribution to the overall report. I also commend the various Defence staff in bringing this report together.

I look forward to continued collaboration between Defence and the ANAO on future Reports.

Dennis Richardson
Secretary
Department of Defence
18 December 2015

Executive Summary

In 2014-15, the Defence Materiel Organisation (DMO)186 managed over $11 billion187 in expenditure for the Department of Defence (Defence)188 across the acquisition and sustainment programs and other management services. As at 30 June 2015, the DMO managed 181 major Defence equipment acquisition projects (major projects) with an average value of $530.5 million.

The 2014-15 Defence Major Projects Report (MPR) examines 25 of the largest and most technically challenging of these. The key aspects of this MPR are:

  • as at 30 June 2015, all projects are delivering capability within the approved budget. Government is considering a real cost increase for the Air Warfare Destroyer Build (AWD) project;
  • analysis has identified that average schedule slippage to Final Materiel Release (FMR), the point at which the DMO has delivered all of the approved materiel requirements, is 14 per cent for the 25 projects in this year’s sample compared with 11 per cent for the 30 projects in last year’s sample (noting that 23 projects are common to both years);
  • the removal of seven projects reported in the 2013-14 MPR (F/A-18 Hornet Upgrade; Guided Missile Frigate Upgrade; Next Generation SATCOM Capability; High Frequency Modernisation; SM-1 Missile Replacement; Artillery Replacement and Follow-on Stand Off Weapon); and
  • the inclusion of two projects (Maritime Communications Modernisation and Maritime Patrol and Response Aircraft System (Boeing P-8A Poseidon)) bringing the total number of reported projects to 25.
  • The DMO’s budget performance bettered that of large commercial projects, and compared favourably with counterpart agencies in other countries. To put this cost performance in perspective:
  • the United States Government Accountability Office’s 2015 Assessment of Selected Weapons Programs analysed 78 programs and found an average increase in total acquisition costs of 46.8 per cent against first full cost estimates (noting the trend towards developmental projects in the US Department of Defense); and
  • the United Kingdom Ministry of Defence’s 2014 Major Projects Report found an overall cost increase of 11 per cent since approval across its 11 projects.
  • Similar to project delivery organisations around the world, schedule performance remains an ongoing challenge. As noted above, the current average slip to FMR is 14 per cent. The average slip to Final Operational Capability (FOC) - where the Capability Manager declares that the whole capability has been reported - is 28 per cent, an improvement from 2013-14 MPR’s 35.8 per cent. Most of the in-year FMR schedule slip experienced by the projects was caused by factors such as technical problems, platform availability and contractual issues (further detail is in Chapter 2). Defence continues to implement changes to deliver improvements in schedule estimation and management (further detail is in Chapter 1). However, the benchmarks below are useful in setting major project schedule performance in context:
  • the United States Government Accountability Office’s 2015 Assessment of Selected Weapons Programs (78 programs) found an average schedule delay of 29 months, or 36.5 per cent, against Initial Operational Capability (IOC); and
  • the United Kingdom Ministry of Defence’s 2014 Major Projects Report found overall slippage of 22 per cent across its 11 current projects.

For 2014-15, the 25 major projects included in the MPR reported that 234 of the 240 (or 97 per cent) capability elements had a high level of confidence in being delivered. While risks exist, the remaining three per cent of capability elements are still considered achievable. For the third year in a row, no capability elements have been identified as being unlikely to be met.

Reforms stemming from the 2003 Kinnaird review, such as the two-pass Government approval process, have driven measurable improvements in major project delivery performance. This is reflected in the improved schedule performance of post-Kinnaird projects reported in the MPR. Average schedule slippage against FOC for these major projects is 13 per cent189, compared with 60 per cent for major projects approved prior to the Kinnaird reforms. The outcomes from the First Principles Review will provide further opportunities for improvement in managing schedule performance and other areas of project delivery.

1. Defence Strategic Performance in 2014-15

Overview

1.1 On 1 July 2015, the Defence Materiel Organisation (DMO) transitioned to the newly established Capability Acquisition and Sustainment Group (CASG) within the Department of Defence as recommended by the Defence First Principles Review. CASG continues to have the former DMO’s objective to provide the materiel equipment and sustainment elements of capability for the ADF in an effective, efficient, economical and safe manner.

1.2 As at 30 June 2015, the DMO was managing 181 major acquisition projects in support of the Australian Defence Force (ADF) with an average value of $530.5 million. In capability acquisition, the DMO was primarily responsible for managing the acquisition of the materiel systems (the mission and support systems) component of capability.190

1.3 During 2014-15, 18 projects were closed, having met the required capability, on average, five per cent under the approved budget.

First Principles Review: Creating One Defence (FPR)

1.4 A recommendation of the FPR was the formation of CASG under the leadership of a Deputy Secretary reporting directly to the Secretary. Under the FPR, CASG will take on the responsibility for developing and delivering integrated project plans that will encompass all Fundamental Inputs to Capability (FIC) (including personnel, training, supplies, facilities, training areas, logistics, support, command and management).

1.5 The planned FPR disbandment of Capability Development Group (CDG) will also see accountability for requirements setting and management transferred to the Vice Chief of the Defence Force and the Service Chiefs. Strategic, financial and technical contestability will be located within the Strategy Policy and Intelligence Group, and strengthened in the process.

1.6 In addition to its new role, CASG will continue to extend across the totality of the acquisition lifecycle, from contributing to the early stages of project development before formal Government approval, to delivering the materiel elements of major projects as approved by Government, sustaining and upgrading them once in service and eventually managing their disposal at the end of their service life.

1.7 This work will continue to be performed in an environment of significant complexity. In addition to recent independent studies, the Joint Committee of Public Accounts and Audit (JCPAA), in their review of the 2013-14 MPR, noted the range of complexities that affect the completion of major projects; these include:

  • managing induced schedule delays as a result of budgetary constraints;
  • employing and maintaining an appropriately skilled workforce where the skills required are in high demand by other industries;
  • acquiring new equipment presenting multiple integration challenges;
  • contractors overestimating the technical maturity of proposed equipment solutions;
  • contractors underestimating the level of effort and complexity required to deliver new equipment;
  • unavailability of in‐service equipment (due to operational requirements) limiting the ability of projects to install and test new or upgraded equipment in accordance with the original planned project schedule, and;
  • complying with increasingly demanding certification and regulatory requirements, and ensuring access to intellectual property to enable continued further enhancement and improvement of systems.

1.8 To manage the complexity outlined above, Defence is investing in the development of an operational framework for CASG. The framework comprehensively explains how the organisation operates and the roles and responsibilities within it. The framework will detail the life cycle management processes that provide project and engineering discipline to manage complex materiel procurement from initiation to disposal. In addition, Defence will reinforce accountability at all levels and bring together information upon which good management decisions can be made.

1.9 All of these changes resulting from the FPR are expected to improve overall project performance.

Achievements

1.10 An overview of cost and schedule performance is shown in Chapter 2. Examples of significant achievements in 2014-15 are:

  • Joint Strike Fighter: on 1 October 2014, Australia’s first F-35A Lightning II Joint Strike Fighter made its inaugural flight, marking another significant milestone for the Australian F-35A program.191 This milestone follows the official rollout of Australia’s first two F-35A on 24 July 2014.192
  • AWD Ships: the first Air Warfare Destroyer, NUSHIP HOBART, was launched in Osborne, South Australia, on 23 May 2015.193
  • Wedgetail: the Project achieved Final Operational Capability (FOC) in May 2015.194 The Wedgetail airborne early warning and control capability provided more than 1,220 hours directing air strikes in the coalition operations in the Middle East since October 2014.195
  • MH-60R Seahawk: the first of two Navy MH-60R Seahawk ‘Romeo’ flight simulators was commissioned into service on 27 March 2015.196
  • LHD Ships: the first of two amphibious ships, HMAS Canberra, was commissioned into service on 28 November 2014.197
  • Air to Air Refuel: KC-30A tanker planes have flown more than 2,200 hours refuelling coalition aircraft in Middle East operations since October 2014.198 The first air to air refuelling boom contact was made, on 10 June 2015, by an Air Force crew of the KC-30A Multi-Role Tanker Transport (MRTT) aircraft.199
  • Battle Comm. Sys.: The project achieved Final Materiel Release (FMR) on 26 March 2015 and FOC certification on 8 April 2015 (more details can be found in the PDSS).

Projects of Concern

1.11 Projects of Concern (PoC) are those projects identified as having very significant technical, cost and/or schedule difficulties. The primary objective of the PoC regime is to assist with the implementation of an agreed remediation plan. Projects listed as PoC receive a higher level of oversight and management and undertake increased reporting to Government. Since 2008, 23 projects, with a total value of $30.4 billion, have been managed as PoC. There are six active PoC (listed in Table 1.1) with a total value of $12.8 billion as at 30 June 2015. In 2014-15, the Air to Air Refuelling project was removed after successful remediation of technical issues, and the Australian Defence Satellite Communications Terrestrial Enhancement (JP 2008 Phase 3F), was added as a PoC.

Table 1.1 – List of Projects of Concern as at 30 June 2015

Project Name

Project Number

Date Added

Collins Class Submarine Sustainment

CN10200

November 2008

Multi-Role Helicopter

AIR 9000 Phase 2/4/6

November 2011

Mulwala Redevelopment Project

JP 2086 Phase 1

December 2012

Direct Fire Support Weapons

LAND 40 Phase 2

December 2012

Air Warfare Destroyer Build

SEA 4000 Phase 3

June 2014

Australian Defence Satellite Communications Terrestrial Enhancement

JP 2008 Phase 3F

September 2014

Challenges

1.12 In 2014-15, the DMO delivered the materiel elements of capability as described in the Materiel Acquisition Agreements set between the DMO, the CDG and the relevant Capability Manager at project approval. The average schedule slip to FMR is 14 per cent for the 25 projects included in this MPR versus 11 per cent for the 30 projects included in the 2013-14 MPR. Note that seven projects have been removed from, and two new projects added to this MPR. The main factors driving the schedule slip against the FMR milestone are: platform availability; industry delays in resolving technical design issues; unforeseen technical problems and other contractual issues.

Major Acquisition Program – Financial Management

1.13 The Chief Finance Officer within Defence provides financial assurance in relation to individual projects included in the 2014-15 MPR delivering the remaining intended scope within approved major project budgets.

1.14 When considering and approving budgets, the Government takes account of the estimated impact of inflation over the life of a project. This forecasting of future inflation is known as ‘out-turning’. From 1 July 2010, all major Defence equipment acquisition projects have been managed using out-turned budgets. At the time of project approval, project managers estimate the impact of indices tendered (or estimated) for the life of the project. These estimates are built into the project budget as part of the out-turning process.

1.15 Financial assurance statements have been validated by an external independent company for a project sample of five201 of the 25 projects reported in this MPR.

Table 1.2 – List of 2014-15 MPR Projects by Total Approved Budget202

Project Number

Project Name

Project Name Abbreviation

2014-15
In-Year Budget
$m

Total
Approved Project Budget
$m

AIR 6000 Phase 2A/2B

New Air Combat Capability

Joint Strike Fighter

296.5

15,181.1

SEA 4000 Phase 3

Air Warfare Destroyer Build

AWD Ships

763.2

7,891.1

AIR 7000 Phase 2B

Maritime Patrol and Response Aircraft System (Boeing P-8A Poseidon)203

P-8A Poseidon

516.4

3,977.8

AIR 5077 Phase 3

Airborne Early Warning and Control Aircraft

Wedgetail

53.7

3,893.2

AIR 9000 Phase 2/4/6

Multi-Role Helicopter

MRH90 Helicopters

299.4

3,747.5

AIR 5349 Phase 3

EA-18G Growler Airborne Electronic Attack Capability

Growler

1,202.5

3,531.4

AIR 9000 Phase 8

Future Naval Aviation Combat System Helicopter

MH-60R Seahawk

670.8

3,408.5

LAND 121 Phase 3B

Medium Heavy Capability, Field Vehicles, Modules and Trailers

Overlander Medium/Heavy

107.5

3,387.6

JP 2048 Phase 4A/4B

Amphibious Ships (LHD)

LHD Ships

86.6

3,091.0

AIR 87 Phase 2

Armed Reconnaissance Helicopter

ARH Tiger Helicopters

1.2

2,032.7

AIR 5402

Air to Air Refuelling Capability

Air to Air Refuel

107.4

1,822.3

AIR 8000 Phase 2

Battlefield Airlift – Caribou Replacement

Battlefield Airlifter

271.5

1,369.2

LAND 116 Phase 3

Bushmaster Protected Mobility Vehicle

Bushmaster Vehicles

67.6

1,250.5

LAND 121 Phase 3A

Field Vehicles and Trailers

Overlander Light

127.5

1,015.7

SEA 1448 Phase 2B

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2B

75.2

678.6

AIR 9000 Phase 5C

Additional Medium Lift Helicopters

Additional Chinook

137.8

633.8

JP 2072 Phase 2A

Battlespace Communications System

Battle Comm. Sys. (Land)

17.1

461.9

SEA 1439 Phase 4A

Collins Replacement Combat System

Collins RCS

1.4

450.4

SEA 1442 Phase 4

Maritime Communications Modernisation204

Maritime Comms

32.3

442.1

SEA 1429 Phase 2

Replacement Heavyweight Torpedo

Hw Torpedo

5.2

427.9

JP 2008 Phase 5A

Indian Ocean Region UHF SATCOM

UHF SATCOM

5.2

420.4

SEA 1439 Phase 3

Collins Class Submarine Reliability and Sustainability

Collins R&S

13.7

411.7

SEA 1448 Phase 2A

ANZAC Anti-Ship Missile Defence

ANZAC ASMD 2A

26.8

386.8

LAND 75 Phase 3.4

Battlefield Command Support System

Battle Comm. Sys.

21.3

313.0

JP 2048 Phase 3

Amphibious Watercraft Replacement

LHD Landing Craft

57.6

236.2

Total

4,965.4

60,462.4

Major Defence Equipment Acquisition Project Performance

1.16 Based on internal and external analysis, the DMO steadily improved its effectiveness in delivering materiel capability between 2000 and 2015. In the last five financial years, 98 major projects were completed after successfully introducing the required capabilities. The materiel capability for these 98 major projects was delivered on average five per cent under budget, resulting in over half a billion dollars in savings.

1.17 As in previous years, the cost performance of five per cent under budget continues to out perform equivalent performance in the private sector, as well as counterpart agencies in other countries. To put this cost performance in perspective:

  • the United States Government Accountability Office’s 2015 Assessment of Selected Weapons Programs analysed 78 programs and found an average increase in total acquisition costs of 46.8 per cent against first full cost estimates (noting the trend towards developmental projects in the United States Department of Defense); and
  • the United Kingdom Ministry of Defence’s 2014 Major Projects Report found an overall cost increase of 11 per cent since approval across 11 major projects.

1.18 The current average schedule slip to FOC of post-Kinnaird205 projects of approximately 13 per cent compared with 60 per cent for pre-Kinnaird projects indicates consistent project performance improvements have been realised and continue over time. Even so, as is the case for project delivery organisations around the world, schedule performance remains a challenge. The information below puts current major project schedule performance in perspective:

  • the United States Government Accountability Office’s 2015 Assessment of Selected Weapons Programs found an average schedule delay of 29 months, or 36.5 per cent, against Initial Operational Capability (IOC); and
  • the United Kingdom Ministry of Defence’s 2014 Major Projects Report exhibits average slippage of 23 per cent across 11 current approved projects.

1.19 Table 1.3 displays the average FOC slip as published in the MPR since 2007-08. In the first year of publication, the average FOC slip was represented in months. Since then it has been expressed as an average percentage slip.

1.20 Having overcome technical challenges, the exit of the Guided Missile Frigate Upgrade and High Frequency Modernisation projects from the MPR project list has significantly improved average FOC slip in 2014-15. Chapter 2 examines the factors that affect FOC in more detail.

Table 1.3 – Historical Representation of Final Operational Capability

Year

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

No. of projects

9

15

22

28

29

29

30

25

Average FOC Slip

 30 mths

 25%

 30%

 28%

 32%

 35%

 35.8%

 28%

1.21 Despite the complex delivery environment, the DMO’s performance in delivering project scope continued to progress positively. Figure 1.1 tracks reporting against the capability Measures of Effectiveness for every MPR project since the report’s inception and highlights a clear trend of improvement. Materiel capability performance of 2014-15 MPR projects as at 30 June 2015 was:

  • 97 per cent of Measures of Effectiveness are likely to be met (green);
  • three per cent of Measures of Effectiveness are under threat but manageable (amber); and
  • zero per cent of Measures of Effectiveness are considered unlikely to be met (red).

Figure 1.1 – Materiel Capability Performance as at 30 June 2015

Continuous Improvement

Negotiation Cell

1.22 In 2013-14, the Defence Negotiation Cell established a standing offer panel for negotiation services with 20 nominated highly-performing individuals from 17 organisations. In 2014-15, the number of nominated individuals increased to 22. These individuals are supporting various project and sustainment teams to develop their commercial expertise and to achieve the best outcomes from their dealings with industry. This is aimed at securing agreements that deliver more value, leading to better contract performance and ultimately to improved outcomes for Defence and industry. Further, in 2015, Defence established the Evaluation and Negotiation Centre of Excellence aimed at significantly improving the conduct of evaluations for complex procurements and commercial negotiations.

Acquisition and Sustainment Planning Framework

1.23 The 2008 Mortimer Review recommended improvements to Defence’s acquisition strategy development process during the requirements phase of the capability systems life cycle. To support this improvement, Defence has developed Acquisition and Support Implementation Strategy (ASIS) policy and guidance that place greater emphasis on implementation of the support system to ensure whole of life issues are better addressed early in the project life cycle.

1.24 Following extensive stakeholder consultation, the ASIS policy and guidance was formally released in July 2015 and is supported by targeted executive and project practitioner training. Importantly, the ASIS policy framework has been developed to encompass the FPR recommended end-to-end capability development and management processes from needs definition through to disposal, including consideration of all FIC. As Defence works to streamline and improve its capability life cycle processes, the ASIS policy and guidance will form a central pillar of this improvement activity.

Schedule Management

1.25 Accurate schedule estimation is essential for predictable schedule performance. To this end, Defence is undertaking various initiatives to improve schedule estimation across major Defence equipment acquisition projects, including:

  • developing a Tier 1 Defence Cost and Schedule Estimation Manual. Chapter 1 of the Manual has been released. The remainder of the Manual is under review until first quarter of 2016;
  • integrating the Schedule Compliance and Risk Assessment Method (SCRAM) with the Integrated Baseline Review (IBR) process to improve early identification of schedule risks, and piloting the new integrated process on selected high risk projects to refine and prove the process; and
  • releasing a Project Controls Manual in August 2014 that updates and consolidates Work Breakdown Structure, scheduling and earned value management processes into a framework to improve consistency of application.

Schedule Compliance Risk Assessment Method

1.26 In 2014-15, SCRAM reviews continued to be performed on major projects to identify risk, issues and technical debt that drive schedule slippage and to provide recommendations for schedule achievement. Notable among these were Project SEA 4000 – Air Warfare Destroyer Build and AIR 6000 – New Air Combat Capability – Joint Strike Fighter (JSF) Mission Systems and Autonomic Logistic Information System (ALIS) Software development.

1.27 For the first time a SCRAM assessment was conducted on the JSF ALIS development using scientific Software Parametric Analysis techniques. The ALIS is being developed using an Agile software development method called ‘Scrum’. SCRAM modelling was based on ‘Story Point’ work unit sizing instead of the traditional Software Lines of Code (SLOC), which is not representative of work effort in Commercial-Off-The-Shelf (COTS) based systems such as ALIS. The initial modelling trial results appear valid based on comparison with contractor actual performance outcomes.

1.28 Based on systemic issues identified during multiple SCRAM Reviews, Defence is now trialling integration of SCRAM with the IBR process. The IBR process is used to formally establish a Project’s Performance Management (cost/schedule/scope) Baseline to track performance during project execution. The revised approach incorporating SCRAM reviews includes the feasibility of a practical method to identify and eliminate potential schedule pressures prior to contract award. SCRAM integrated IBR Trials will continue in 2015-16.

1.29 Work is also continuing on the development of a standard Technical Implementation Risk Assessment (TIRA) methodology. The intention is for TIRAs to provide a framework and guidance for the conduct of early Defence technical risk assessments prior to the formation of Performance Management Baseline at IBR. Hence, TIRAs would also be an input for Materiel Implementation Risk Assessments, which are developed by Defence’s Project Managers as part of the Project Approvals process.

1.30 Finally, work has commenced on development of a SCRAM extension for manufacturing and production which is not covered in the current SCRAM Process Reference and Assessment Models. This has been introduced following an increasing demand for SCRAM reviews in production related projects such as AWD shipbuilding and ASMD follow-on production programs.

Information and Communications Technology (ICT) Services

1.31 As the Defence ICT capability owner, the Chief Information Officer Group (CIOG) delivers the ICT infrastructure and services to CASG that are necessary to enable materiel capability to the war fighter as well as the corporate and other functions of CASG. In 2014-15, the DMO positioned its Information Management function to ensure close alignment with CIOG service delivery changes and the wider Defence service delivery reform currently underway.

1.32 The DMO (and now CASG) has supported the Program Definition Phase of a major new program—a Defence-wide Enterprise Resource Planning (ERP) system being managed by CIOG.

1.33 The Defence ERP will consolidate Finance, Logistics, Procurement, Engineering and Maintenance and Estate applications into a single ERP solution.

1.34 Significant benefits of this integrated model include increased availability and accuracy of information to support decision making, improved availability of Defence assets and improved information and operation in communications interrupted environments.

1.35 The implementation of a single ERP-enabled operating model will provide an opportunity to streamline and unify business processes throughout Defence.

Professionalisation

1.36 Defence’s performance is critically dependent on the skills, knowledge, expertise, competency, capability and professionalism of its workforce. The professionalisation of the CASG workforce remains central in ensuring committed people with the right skills are in appropriate roles to deliver capability.

1.37 Defence continues to refine education and skilling pathways for its staff. During 2014-15, the DMO facilitated 22 courses in the project management discipline; 17 in the logistics domain; 25 in the commercial and business domain; 22 in the procurement and contracting domain; and 10 engineering courses. The DMO also facilitated and managed 16 leadership programs for 333 candidates to enhance the leadership and management skills of future DMO (now CASG) and Defence senior executives; as well as sponsoring the Executive Masters in Business Administration Programs in Complex Program Leadership and Strategic Procurement conducted by the Queensland Institute of Technology for 19 DMO participants.

1.38 In 2014-15, the DMO sponsored 660 individuals under its Professional Body Sponsorship and Certification Program. A further 417 personnel have undertaken Australian Qualification Framework vocational training packages in the materiel logistics/sustainment management disciplines. The DMO has continued this program as a key strategy in DMO’s initiatives to professionalise the workforce.

Projects Exited the MPR

1.39 Table 1.4 lists all the projects that have exited the MPR since its inception. The Expenditure to Date is as at 30 June 2015.

Table 1.4 – List of Projects Exited from MPR

Project Number

Project

First Reported in MPR

Last Reported in MPR

Level of Development

Government Approved Budget $m

Expenditure to Date $m

Remaining Budget $m

FMR Achieved/Forecast

FOC Achieved/Forecast

Reason for Exit

AIR 5376 Phase 3.2

F/A 18 Hornet Upgrade Structural Refurbishment (Hornet Refurb)

2008-09

2010-11

Australianised MOTS

319.1

319.1

0.000

N/A

N/A

JCPAA Approval206

AIR 8000 Phase 3

C-17 Heavy Airlift

2008-09

2011-12

MOTS

1,519.6

1,420.4

99.2

Dec-11

Dec-11

FOC achieved

AIR 5349 Phase 1/2

Bridging Air Combat Capability

2008-09

2012-13

MOTS

3,627.3

2,964.7

662.6

Dec-12

Dec-12

FOC achieved

SEA 1444 Phase 1

Armidale Class Patrol Boat

2007-08

2012-13

Australianised MOTS

537.2

488.5

48.7

Nov-07

Oct-12

FOC achieved

LAND 19 Phase 7A

Counter-Rocket Artillery and Mortar

2011-12

2012-13

MOTS

261.9

182.5

79.4

Jan-13

Jan-13

FOC achieved

AIR 5376 Phase 2

F/A 18 Hornet Upgrade

2007-08

2013-14

AMOTS

1,882.6

1,655.5

227.1

Sept 12

Oct 14

FMR achieved

AIR 5418 Phase 1

Follow On Stand Off Weapon

2009-10

2013-14

AMOTS

318.6

284.1

34.5

Sept 13

Jan 14

FOC achieved

JP 2008 Phase 4

Next Generation SATCOM Capability

2009-10

2013-14

MOTS

869.5

568.9

300.6

Jun 14

Jul 15

FMR achieved

JP 2043 Phase 3A

High Frequency Modernisation

2007-08

2013-14

Developmental

580.1

469.3

110.8

Dec 16

Dec 16

JCPAA Approval207

LAND 17 Phase 1A

Artillery Replacement

2010-11

2013-14

MOTS

159.5

158.5

1.0

Sept 13

Oct 14

FMR achieved

SEA 1390 Phase 2.1

Guided Missile Frigate Upgrade Implementation

2007-08

2013-14

Developmental

1,453.2

1,373.6

79.5

Mar 16

Mar 16

JCPAA Approval208

SEA 1390 Phase 4B

SM-1 Missile Replacement

2010-11

2013-14

AMOTS

413.7

344.1

69.6

Feb 15

Jun 15

JCPAA Approval209

2. Summary of Major Project Performance in 2014-15

Introduction

2.1 This chapter provides a performance overview of the 25 projects included in the 2014-15 MPR. It includes detailed analysis of the three key variables of cost, schedule and materiel systems delivery against the Government’s approved budget, schedule and scope. The analysis commences at an aggregate level, discussing performance of all 25 MPR projects collectively, before addressing some project specifics.

2.2 The MPR only covers 25 of the 181 major projects as at 30 June 2015 (14 per cent of the Approved Major Capital Investment Program and 61 per cent by value), so caution must be applied when extrapolating any analysis to the entirety of Defence’s acquisition effort. This is because the projects in the MPR are not necessarily representative of all projects: the 25 projects are the largest by budget, at the time of inclusion and, in general, involve higher levels of complexity than other Defence projects with relatively smaller budgets.

Project Performance

2.3 Table 2.1 gives a summary of life-to-date budget approvals and Table 2.2 gives an overview of the in-year budget movements and cost performance. Table 2.3 shows schedule performance for the 25 projects in this year’s MPR and Table 2.4 presents a summary of the key characteristics of each project in terms of maturity and level of development required. Defence’s analysis indicates that, while projects have been managed within approved budgets, schedule performance, as identified in previous MPRs, continues to be the key issue.

Table 2.1 – Project Total Budget Status

Project Number

Project

(a)Government Approved Budget ($m)

(b) Price Indexation (to July 2010) ($m)

(c) Foreign Exchange Variations ($m)

(d) Scope Changes ($m)

(e) Transfers ($m)

(f) Budgetary Adjustments ($m)

(g)Budget Cost Savings ($m)

((f+g)/a) Net Variation (%)

(a+b+c+d+e+f+g) Current Total Budget ($m)

AIR 6000 Phase 2A/2B

Joint Strike Fighter

2,751.6

351.0

1,566.0

10,515.4

0.0

-2.9

0.0

-0.1%

15,181.1

SEA 4000 Phase 3

AWD Ships

7,207.4

1,173.2

-379.6

0.0

-109.9

0.0

0.0

0.0%

7,891.1

AIR 7000 Phase 2B

P-8A Poseidon

3,637.4

20.5

379.5

0.0

-37.9

0.0

-21.7

-0.6%

3,977.8

AIR 5077 Phase 3

Wedgetail

2,170.4

1,111.1

-59.3

225.6

618.6

-173.2

0.0

-8.0%

3,893.2

AIR 9000 Phase 2/4/6

MRH90 Helicopters

957.2

679.8

-247.6

2,597.1

-239.0

0.0

0.0

0.0%

3,747.5

AIR 5349 Phase 3

Growler

1,155.3

0.0

689.4

1,686.7

0.0

0.0

0.0

0.0%

3,531.4

AIR 9000 Phase 8

MH-60R Seahawk

3,029.6

0.1

418.0

0.0

0.0

-39.2

0.0

-1.3%

3,408.5

LAND 121 Phase 3B

Overlander Medium/Heavy

3,284.7

0.0

102.9

0.0

0.0

0.0

0.0

0.0%

3,387.6

JP 2048 Phase 4A/4B

LHD Ships

2,958.3

428.4

-305.0

0.0

9.3

0.0

0.0

0.0%

3,091.0

AIR 87 Phase 2

ARH Tiger Helicopters

1,584.0

418.2

121.5

0.0

-84.3

-6.7

0.0

-0.4%

2,032.7

AIR 5402

Air to Air Refuel

2,076.6

484.1

-449.3

0.0

-135.5

-153.6

0.0

-7.4%

1,822.3

AIR 8000 Phase 2

Battlefield Airlifter

1,156.5

0.0

212.7

0.0

0.0

0.0

0.0

0.0%

1,369.2

LAND 116 Phase 3

Bushmaster Vehicles

295.0

124.6

-1.3

832.2

0.0

0.0

0.0

0.0%

1,250.5

LAND 121 Phase 3A

Overlander Light

3,171.2

0.0

31.0

-2,186.5

0.0

0.0

0.0

0.0%

1,015.7

SEA 1448 Phase 2B

ANZAC ASMD 2B

248.8

76.1

-9.7

214.7

148.7

0.0

0.0

0.0%

678.6

AIR 9000 Phase 5C

Additional Chinook

637.6

46.9

-50.7

0.0

0.0

0.0

0.0

0.0%

633.8

JP 2072 Phase 2A

Battle Comm. Sys. (Land)

436.4

0.0

25.5

0.0

0.0

0.0

0.0

0.0%

461.9

SEA 1439 Phase 4A

Collins RCS

455.3

56.5

-59.7

0.0

-0.9

-0.8

0.0

-0.2%

450.4

SEA 1442 Phase 4

Maritime Comms

385.7

0.0

56.4

0.0

0.0

0.0

0.0

0.0%

442.1

SEA 1429 Phase 2

Hw Torpedo

238.1

99.4

-123.7

213.3

1.0

-0.2

0.0

-0.1%

427.9

JP 2008 Phase 5A

UHF SATCOM

460.9

18.0

-40.5

0.0

0.0

0.0

-18.0

-3.9%

420.4

SEA 1439 Phase 3

Collins R&S

72.0

74.4

-5.9

310.3

-38.3

-0.8

0.0

-1.1%

411.7

SEA 1448 Phase 2A

ANZAC ASMD 2A

449.0

101.3

-3.6

0.0

-159.8

-0.1

0.0

0.0%

386.8

LAND 75 Phase 3.4

Battle Comm. Sys.

332.9

15.6

-20.2

-15.3

0.0

0.0

0.0

0.0%

313.0

JP 2048 Phase 3

LHD Landing Craft

236.4

0.1

8.1

-0.7

-7.7

0.0

0.0

0.0%

236.2

 

Total

39,388.3

5,279.3

1,854.9

14,392.8

-35.7

-377.5

-39.7

-1.1%

60,462.4

Table 2.2 – Project In–Year Budget Status

Project Number

Project

Portfolio Budget Statements ($m)

Portfolio Additional Estimate Statements ($m)

(a) Final Plan ($m)

(b) Actual Spend ($m)

(b-a) Variation ($m)

Variation (%)

AIR 6000 Phase 2A/2B

Joint Strike Fighter

237.9

277.9

296.5

233.2

-63.3

-21.3%

SEA 4000 Phase 3

AWD Ships

615.6

759.3

763.2

734.2

-29.0

-3.8%

AIR 7000 Phase 2B

P-8A Poseidon

324.0

450.8

516.4

531.5

15.1

2.9%

AIR 5077 Phase 3

Wedgetail

63.9

51.1

53.7

49.1

-4.6

-8.6%

AIR 9000 Phase 2/4/6

MRH90 Helicopters

285.7

267.7

299.4

300.5

1.1

0.4%

AIR 5349 Phase 3

Growler

797.4

728.5

1,202.5

1,241.9

39.4

3.3%

AIR 9000 Phase 8

MH-60R Seahawk

504.7

511.7

670.8

685.5

14.7

2.2%

LAND 121 Phase 3B

Overlander Medium/Heavy

118.8

107.4

107.5

106.1

-1.4

-1.3%

JP 2048 Phase 4A/4B

LHD Ships

142.6

85.6

86.6

81.3

-5.3

-6.1%

AIR 87 Phase 2

ARH Tiger Helicopters

3.8

0.2

1.2

1.2

0.0

0.0%

AIR 5402

Air to Air Refuel

141.9

103.7

107.4

103.8

-3.6

-3.4%

AIR 8000 Phase 2

Battlefield Airlifter

313.8

255.4

271.5

158.5

-113.0

-41.6%

LAND 116 Phase 3

Bushmaster Vehicles

68.4

67.3

67.6

68.4

0.8

1.2%

LAND 121 Phase 3A

Overlander Light

170.6

127.3

127.5

127.3

-0.2

-0.2%

SEA 1448 Phase 2B

ANZAC ASMD 2B

77.2

71.9

75.2

72.7

-2.5

-3.3%

AIR 9000 Phase 5C

Additional Chinook

165.9

132.6

137.8

136.4

-1.4

-1.0%

JP 2072 Phase 2A

Battle Comm. Sys. (Land)

73.5

16.8

17.1

20.0

2.9

17.0%

SEA 1439 Phase 4A

Collins RCS

1.4

1.4

1.4

1.1

-0.3

-21.4%

SEA 1442 Phase 4

Maritime Comms

23.6

21.7

32.3

31.5

-0.8

-2.5%

SEA 1429 Phase 2

Hw Torpedo

8.0

5.0

5.2

4.7

-0.5

-9.6%

JP 2008 Phase 5A

UHF SATCOM

10.7

5.0

5.2

1.3

-3.9

-75.0%

SEA 1439 Phase 3

Collins R&S

7.3

13.7

13.7

11.0

-2.7

-19.7%

SEA 1448 Phase 2A

ANZAC ASMD 2A

28.2

24.9

26.8

26.6

-0.2

-0.7%

LAND 75 Phase 3.4

Battle Comm. Sys.

38.2

15.2

21.3

23.8

2.5

11.7%

JP 2048 Phase 3

LHD Landing Craft

55.1

50.3

57.6

56.6

-1.0

-1.7%

 

Total

4,278.2

4,152.4

4,965.4

4,808.2

-157.2

-3.2%

Table 2.3 – Project Schedule Status

Project Number

Project

(a) 2nd Pass Approval

(b) Originally estimated FMR210

Forecast FMR at 30 June 2014

(c) Forecast FMR at 30 June 2015

(c-a)/ (b-a)Variance Factor211 FMR

(d) Originally estimated FOC

Forecast FOC at 30 June 2014

(e) Forecast FOC at 30 June 2015

(e-a)/(d-a) Variance Factor FOC

AIR 6000 Phase 2A/2B

Joint Strike Fighter

Nov-09

Dec-23

Aug-23

Dec-23

1.00

Dec-23

Oct-23

Dec-23

1.00

SEA 4000 Phase 3

AWD Ships

Jun-07

Dec-17

Sep-19

Sep-20

1.26

May-18

Mar-20

Mar-21

1.26

AIR 7000 Phase 2B

P-8A Poseidon

Feb-14

Oct-19

-

Oct-19

1.00

Jan-20

-

Jan-20

1.00

AIR 5077 Phase 3

Wedgetail

Dec-00

Nov-12

Dec-14

Feb-15

1.19

Dec-08

Jun-15

May-15

1.80

AIR 9000 Phase 2/4/6

MRH90 Helicopters

Aug-04

Oct-14

Dec-17

Dec-17

1.31

Jul-14

Jul-19

Jul-19

1.50

AIR 5349 Phase 3

Growler

Apr-13

Jul-22

Jul-22

Jul-22

1.00

Jul-22

Jul-22

Jul-22

1.00

AIR 9000 Phase 8

MH-60R Seahawk

Jun-11

Dec-23

Dec-23

Dec-23

1.00

Dec-23

Dec-23

Dec-23

1.00

LAND 121 Phase 3B

Overlander Medium/Heavy

Jul-13

Dec-22

Jul-22

Jul-22

0.96

Dec-23

Jan-23

Mar-23

0.93

JP 2048 Phase 4A/4B

LHD Ships

Jun-07

Aug-15

Aug-15

Oct-15

1.02

Nov-16

Nov-16

Nov-16

1.00

AIR 87 Phase 2

ARH Tiger Helicopters

Mar-99

Jul-12

Mar-14

Mar-14

1.12

Jun-09

Jan-16

Jan-16

1.64

AIR 5402

Air to Air Refuel

May-03

Feb-13

Dec-15

May-16

1.33

Mar-11

Dec-15

May-16

1.66

AIR 8000 Phase 2

Battlefield Airlifter

Apr-12

Oct-17

Oct-17

Mar-18

1.08

Dec-17

Dec-17

Sep-18

1.13

LAND 116 Phase 3

Bushmaster Vehicles

Nov-98

Sep-16

Sep-16

Sep-16

1.00

Dec-16

Dec-16

Dec-16

1.00

LAND 121 Phase 3A

Overlander Light

Dec-11

Jul-16

Mar-17

Oct-16

1.05

Jul-16

Mar-17

Oct-16

1.05

SEA 1448 Phase 2B

ANZAC ASMD 2B

Sep-05

Jul-17

Sep-17

Oct-17

1.02

Mar-13

Oct-17

Oct-17

1.61

AIR 9000 Phase 5C

Additional Chinook

Feb-10

Jan-17

Jan-17

Jan-17

1.00

Jan-17

Jan-17

Jan-17

1.00

JP 2072 Phase 2A

Battle Comm. Sys. (Land)

Nov-11

Aug-16

Aug-16

Aug-16

1.00

Jun-16

Oct-16

Oct-16

1.07

SEA 1439 Phase 4A

Collins RCS

Sep-02

Jan-16

Oct-18

Oct-18

1.21

Dec-10

Feb-19

Feb-19

1.99

SEA 1442 Phase 4

Maritime Comms

Jul-13

May-23

-

May-23

1.00

Dec-23

-

Dec-23

1.00

SEA 1429 Phase 2

Hw Torpedo

Jul-01

Nov-13

Aug-18

Oct-18

1.40

Nov-13

Sep-18

Feb-19

1.42

JP 2008 Phase 5A

UHF SATCOM

Mar-09

Mar-14

May-15

Jul-18

1.87

Jun-18

Jun-18

Jun-18

1.00

SEA 1439 Phase 3

Collins R&S

Sep-00

Oct-22

Oct-22

Aug-22

0.99

Jun-14

Sep-22

May-22

1.58

SEA 1448 Phase 2A

ANZAC ASMD 2A

Nov-03

Jul-17

Sep-17

Oct-17

1.01

Dec-11

Oct-17

Oct-17

1.72

LAND 75 Phase 3.4

Battle Comm. Sys.

Nov-09

Apr-13

Feb-15

Mar-15

1.56

Apr-13

Mar-15

Apr-15

1.59

JP 2048 Phase 3

LHD Landing Craft

Sep-11

Feb-16

Sep-15

Feb-16

1.00

Feb-16

Sep-15

Feb-16

1.00

 

 

 

 

 

Average

1.14

 

 

Average

1.28

Table 2.4 – Project Characteristics

Project Number

Project

First reported in MPR of

Customer

Purpose of Capability212

Level of Development

Acquisition Category213

Pre-/Post-Kinnaird214

Project Stage

Prime Systems Integrator

AIR 6000 Phase 2A/2B

Joint Strike Fighter

2010-11

Air Force

Replacement

Developmental

I

Post

Enter Contract

US Government

SEA 4000 Phase 3

AWD Ships

2008-09

Navy

New

AMOTS

I

Post

Detailed Design Review

AWD Alliance

AIR 7000 Phase 2B

P-8A Poseidon

2014-15

Air Force

Replacement

MOTS

II

Post

Second Pass Approval

US Government

AIR 5077 Phase 3

Wedgetail

2007-08

Air Force

New

Developmental

III

Pre

Final Contract Acceptance

Boeing Company

AIR 9000 Phase 2/4/6

MRH90 Helicopters

2008-09

Army/Navy

Replacement

AMOTS

I

Post

Initial Materiel Release

Airbus Group Australia Pacific

AIR 5349 Phase 3

Growler

2013-14

Air Force

New

AMOTS

II

Post

Enter Contract

US Government

AIR 9000 Phase 8

MH-60R Seahawk

2011-12

Navy

Replacement

MOTS

II

Post

Initial Materiel Release

US Government

LAND 121 Phase 3B

Overlander Medium/Heavy

2013-14

Army

Replacement

AMOTS

I

Post

Preliminary Design Review

Defence

JP 2048 Phase 4A/ 4B

LHD Ships

2008-09

Joint

New

AMOTS

I

Post

Integration and Test

BAE Systems Australia

AIR 87 Phase 2

ARH Tiger Helicopters

2007-08

Army

New

AMOTS

II

Pre

Acceptance Into Service

Airbus Group Australia Pacific

AIR 5402

Air to Air Refuel

2008-09

Air Force

New

Developmental

II

Pre

Initial Materiel Release

Airbus Defence and Space

AIR 8000 Phase 2

Battlefield Airlifter

2013-14

Air Force

Replacement

MOTS

II

Post

Integration and Test

US Government

LAND 116 Phase 3

Bushmaster Vehicles

2007-08

Army/Air Force

Replacement

AMOTS

III

Pre

Complete Acceptance Testing

Thales

LAND 121 Phase 3A

Overlander Light

2009-10

Army

Replacement

AMOTS

II

Post

Initial Materiel Release

Defence

SEA 1448 Phase 2B

ANZAC ASMD 2B

2009-10

Navy

Upgrade

Developmental

I

Post

Initial Materiel Release

ANZAC Alliance

AIR 9000 Phase 5C

Additional Chinook

2010-11

Army

Replacement

MOTS

III

Post

Initial Materiel Release

US Government

JP 2072 Phase 2A

Battle Comm. Sys. (Land)

2012-13

Joint

Replacement

MOTS

III

Post

Acceptance Into Service

Defence

SEA 1439 Phase 4A

Collins RCS

2007-08

Navy

Upgrade

AMOTS

IV

Pre

Initial Materiel Release

Defence

SEA 1442 Phase 4

Maritime Comms

2014-15

Navy

Upgrade

AMOTS

II

Post

Preliminary Design Review

Selex

SEA 1429 Phase 2

Hw Torpedo

2009-10

Navy

Replacement

MOTS

III

Pre

Initial Materiel Release

US Government

JP 2008 Phase 5A

UHF SATCOM

2010-11

Joint

Upgrade

MOTS

II

Post

Detailed Design Review

Intelsat

SEA 1439 Phase 3

Collins R&S

2009-10

Navy

Upgrade

AMOTS

III

Pre

Integration and Test

ASC

SEA 1448 Phase 2A

ANZAC ASMD 2A

2009-10

Navy

Upgrade

AMOTS

II

Pre

Initial Materiel Release

ANZAC Alliance

LAND 75 Phase 3.4

Battle Comm. Sys.

2010-11

Army

New

AMOTS

II

Post

Final Materiel Release

Defence

JP 2048 Phase 3

LHD Landing Craft

2013-14

Navy

Replacement

AMOTS

III

Post

Initial Materiel Release

Navantia

Budget Performance

2.4 Project budgets, against which cost performance is measured, are subject to variations arising from exchange rate variations, Government approval of changes in scope and transfers to Defence Groups. Following the move to out-turned project budgets in July 2010, there are no further variations to project budgets as a result of inflationary effects.

2.5 Table 2.1 includes a summary of the project budget variations from date of Government approval to 30 June 2015. Two variations in scope occurred during 2014-15:

  • Growler (December 2014): budget increase of $200.6 million for inclusion of Growler Enabling Capabilities; and
  • Battle Comm. Sys. (June 2015): budget decrease of $8.3 million transferred from LAND 75 Phase 3.4 to support design work now conducted as part of LAND 75 Phase 4 work package A.

2.6 Figure 2.1 provides a comparison of expenditure as at 30 June 2015 with the total approved budget at that date. While this provides an indicator of project progress, the percentage of budget spent is dependent on the nature of the project and the level of early investment that may be required for project start-up and non-recurring engineering effort.

2.7 Importantly, as at 30 June 2015 no project has exceeded its total approved budget and the net variation to the original Government approvals is negative 1.1 per cent, reflecting accumulated real savings to date of $417.2 million across the 25 projects.

Figure 2.1: Comparison of Overall Project Budget and Expenditure as at 30 June 2015 (in $m)

2.8 The relationship between project expenditure and project progress is not necessarily linear. The profile of expenditure against total approved budget is determined by several factors including the level of development required for the acquisition. For example, a project with a low level of development acquiring a Commercial or Military Off-The-Shelf (COTS or MOTS) product will generally have an expenditure pattern closely matched to the production of the materiel elements. In comparison, a highly developmental project usually requires a degree of initial ‘seed capital’ on commencement with expenditure declining during the development phase and increasing as the project shifts into the build/integration phase.

2.9 Another factor is the evolution of the project and its performance to date. A project may be well advanced in years but show a low level of expenditure against the total budget. This may result from poor contractual performance culminating in withholding of payments against specific milestones. This is, in effect, a deferral of payments that will be re-instated upon contractor achievement of milestones.

2.10 Table 2.2 provides a summary of the in-year spend against budget for 2014-15. Figure 2.2 compares in-year expenditure as at 30 June 2015 with budget for 2014-15. Most projects have expended close to their given budget with minor variations attributed primarily to exchange rates or rounding issues.

  • Within Table 2.2, two projects had significant budget overachievements in percentage terms:
    • Battle Comm. Sys (Land) – 17.6 per cent overachievement ($2.9 million) is due to accelerated support contract initial payment and delivery of attrition spares for Combat Net radio equipment.
    • Battle Comm. Sys. – 11.7 per cent overachievement ($2.5 million) due to a Contract Change Proposal for the Track Management System.
  • Five projects had significant budget underachievements in percentage terms:
    • Joint Strike Fighter – 21.3 per cent underspend ($63.3 million) is primarily due to United States F-35 Joint Program Office contracted timeframes and unpredictability of expenditure forecasts.
    • Battlefield Airlifter – 41.6 per cent underspend ($113.0 million) driven primarily by delay in billing by the United States and delayed spares and Ground Support Equipment deliveries, productions milestones for aircraft five through 10, training mobilisation and acquisition of mature training system devices. Delays in acceptance of aircraft three and four resulted in non-achievement of technical data milestones and delays in Ground Support Equipment and Tools procurement also contributed.
    • Collins RCS – 21.4 per cent underspend ($0.3 million) is primarily due to delays in programmed work by Australian industry.
    • UHF SATCOM – 75 per cent underspend ($3.9 million) is due to delays with the prime contract milestones (‘Test Readiness Review’ and ‘Spares, Support and Test Equipment’), which have additionally indirectly impacted Outsourced Service Providers’ ability to meet agreed milestone dates.
    • Collins R&S – 19.7 per cent underspend ($2.7 million) is primarily due to delays and cost savings by Australian Industry.

2.11 Figure 2.2 shows three projects (P-8A Poseidon, Growler and MH-60R Seahawk) had significant overachievements in dollar terms. All were due to early Foreign Military Sales case payments – i.e. payments occurred ahead of schedule.

Figure 2.2: Comparison of In-year Project Budget and Expenditure as at 30 June 2015 (in $m)

Contingency Management

2.12 In keeping with standard commercial practice, budgets for major Defence capital investment programs are approved by Government with a contingency provision that varies between projects depending on the complexity and risk of the acquisition. This allows Project Managers to proactively manage risk, and, when necessary, treat risk events that have materialised into issues. Contingency funding provides a limited financial margin for Project Managers against inherent uncertainties, risks and unexpected events that may arise during the course of a project. It is especially important in Defence projects that typically have greater inherent risk, longer timeframes and are generally more complex than private sector projects.

2.13 Previous reports have included the aggregate amount of contingency that has been spent over the life of the relevant MPR projects. Since the 2013-14 MPR reporting period, each Project Data Summary Sheet now includes advice on whether contingency has been applied to the project during the financial year. The application of contingency is assured by the ANAO. Consistent with this, the aggregate amount of contingency reported over the life of the relevant MPR projects in paragraph 2.14 is the aggregate amount of applied contingency rather than contingency actually spent. The term ‘applied contingency’ is the amount of contingency that a project has allocated against identified risks. It is a combination of both the amount of contingency that has already been spent and the amount that may need to be spent to manage or retire risk should it materialise.

2.14 Across the life of the 25 projects in this year’s MPR (that is, from November 1998 to June 2015), the aggregate amount of applied contingency is approximately $1.8 billion. This represents three per cent of the 25 projects combined project approval value ($60.5 billion) or 26 per cent of the combined contingency budget. The areas where risk has been retired using contingency include:

  • systems development;
  • systems integration;
  • logistics and support;
  • schedule constraints; and
  • project resourcing.

2.15 Defence has established a framework which provides full accountability and traceability of all management decisions related to the use of the project contingency budget.

Schedule Performance

2.16 In 2010, Defence introduced Initial Materiel Release and Final Materiel Release (FMR) as more appropriate milestones for measuring materiel acquisition performance in contributing to the Capability Managers’ coordination of all capability elements to achieve Initial and Final Operational Capability (IOC and FOC) milestones. Schedule analysis presented in this and previous MPRs was based on achievement, or expected achievement, of FOC.

2.17 Figure 2.3 represents the schedule performance for each of the projects covered in this year’s MPR. The chart shows the original project approval date, the originally approved FMR estimate and the forecast FMR as at 30 June 2015. The chart also highlights the time it takes to deliver complex Defence acquisition projects.

2.18 Two projects achieved FMR during the 2014-15 financial year: Wedgetail and Battle Comm. Sys. Another three are scheduled to achieve FMR during 2015-16: LHD Ships; Air to Air Refuel; and LHD Landing Craft.

Figure 2.3: Schedule from Approval to Current FMR Estimate

2.19 The period between FMR and FOC indicates the time required by Capability Managers to bring together all the other Fundamental Inputs to Capability. This usually occurs after the final materiel elements of capability have been delivered.215 Figure 2.4 provides an indication of the estimated time required to complete this work.

2.20 Two projects in the MPR – Wedgetail and Battle Comms. Sys. – achieved FOC during 2014-15; another three are scheduled to achieve FOC during 2015-16. The three projects are: ARH Tiger Helicopters; Air to Air Refuel; and LHD Landing Craft. The declaration of FOC marks the formal transition of capability from the acquisition to the sustainment phase of the capability life cycle at that time.

Figure 2.4: Current FMR and FOC Estimates

In-year Final Materiel Release Schedule Variance

2.21 Figure 2.5 illustrates the in-year schedule variance for FMR as a percentage of the total time expected to reach the FMR milestone. In the 2014-15 MPR, 14 projects re-assessed their estimated FMR date.

2.22 The largest individual in-year variance to forecast FMR dates is 63 per cent for UHF SATCOM reflecting an anticipated delay of 38 months. This is because the Network Control System component of the project is no longer a COTS solution and is now deemed as developmental software design. Due to some design faults, ViaSat has taken on elements of work previously contracted to their sub-contractors and are continuing the software design in-house.

2.23 Two projects are forecasting to be ahead of their FMR estimates from the previous years: Overlander Light – nine per cent and Collins R&S improved its FMR forecast by one per cent (more details are available in the Project Data Summary Sheets for each project).

Figure 2.5: In-year FMR Schedule Variance, by Project Type

Total Final Materiel Release Schedule Variance

2.24 Figure 2.6 shows the variance to forecast FMR dates, as a percentage of originally estimated duration, for each of the 25 projects in the MPR. At 30 June 2015, the average FMR variance for all of the MPR projects was 14 per cent.

2.25 Two projects are forecasting early achievement of FMR: Overlander Medium/Heavy and Collins R&S.

2.26 Seven projects reported an FMR schedule variation of 20 per cent or greater across the life of the project. The largest individual variance to forecast FMR dates for the life of the project is 87 per cent for UHF SATCOM reflecting a delay of 52 months due to the Prime Contractor’s decision to develop the software design in-house which was previously contracted out to sub-contractors. Battle Comm. Sys. has a variance of 56 per cent. The delay was due to the need to better align with the Army Brigade Rotation Cycle. Government approved a revised FMR baseline of March 2015 and the project subsequently met this, achieving FMR on 26 March 2015.

2.27 Hw Torpedo has an FMR variance of 40 per cent, across the life of the project, because the FMR date was set before the Full Cycle Docking program had reached maturity in terms of the length of dockings and impact of emergent work and other capability upgrades. As a result, the Heavy Weight Torpedo installation schedule has been delayed. Air to Air Refuel has a variance of 33 per cent due to a combination of impacts from supporting operations and contractor performance.

Figure 2.6: Total Schedule Variance for FMR, by Project Type

In-year Final Operational Capability Schedule Variance

2.28 Figure 2.7 shows in-year schedule variance to forecast FOC dates, as a percentage of the 2014-15 MPR project duration estimate. In the 2014-15 MPR, 11 projects re-assessed their forecast FOC date. Key points to note are as follows:

  • The largest in-year FOC schedule variance was recorded by the Battlefield Airlifter project, which had in-year movement of 13 per cent. The variation was due to delays in aircraft production, to acquisition of Mature Training System devices and construction approval of facilities at RAAF Base Amberley, Queensland.
  • Three projects: Wedgetail (one month); Overlander Light (five months); and Collins R&S (four months) have gained some schedule ahead of their FOC baselines.

Figure 2.7: In-year FOC Schedule Variance, by Project Type

Note: Joint Strike Fighter is currently developmental in nature but should ultimately become MOTS when it enters production line delivery.

Total Final Operational Capability Schedule Variance

2.29 The average variance to forecast FOC dates, as a percentage of originally estimated duration, for the 25 projects in MPR 2014-15, was 28 per cent as at 30 June 2015.

2.30 Figure 2.8 charts the schedule variance as a percentage of the originally estimated project duration from Government approval to FOC. The projects are grouped in the categories of MOTS, Australianised MOTS (AMOTS) and Developmental. The chart shows that, generally, MOTS projects are more likely to be delivered on time while AMOTS and Developmental acquisitions are more prone to underestimating technical complexity and systems integration effort.

2.31 The number of projects reporting an FOC schedule variation of 50 per cent or greater has decreased from 12 in last year’s MPR to nine. Delays to FOC over the life of these projects are as follows:

  • Wedgetail has faced difficulties integrating the phased array radar and other mission critical elements into an operational system. However, the capability was successfully deployed during recent operations in the Middle East and achieved FOC in May 2015.
  • MRH90 Helicopters delays are due to a number of technical and reliability issues.
  • ARH Tiger Helicopter delay to FOC resulted from a reduction in the flying rate of effort experienced by the ARH fleet stemming from less than expected maturity of the contractor’s Tiger helicopter program at the time of acquisition.
  • Air to Air Refuel delays are mainly due to development and improvements to the avionics and the air refuelling boom systems required to meet full capability. As with Wedgetail, the Air to Air Refuel capability has successfully supported the coalition’s recent operations in the Middle East.
  • The two Collins projects – RCS and R&S – have been negatively impacted by limited platform availability due to operational requirements arising from the implementation of the Coles Review and unscheduled maintenance requirements having a detrimental impact on the Full Cycle Docking program.
  • Deferral of FOC for the Anzac Anti-Ship Missile Defence program (Phases 2A and 2B) resulted from a Government approved change of project delivery strategy and scope, including the decision to substitute the Very Short Range Air Defence System option with a phased array radar capability. This is a significant capability advantage over the originally approved scope and allowed leading edge Australian technology to be proven in one ship before committing to the upgrade of the remaining seven ships.
  • Battle Comm. Sys. delayed the FOC date to align installation of equipment with Capability Manager priorities. In December 2011, Government agreed to align the LAND 75 Phase 3.4 FOC with the Army’s Brigade rotation cycle circa December 2013. The approval was linked to a Basis of Provisioning change sought by the Army.

Figure 2.8: Schedule Variance for FOC since Government Approval, by Project Type

Note: Joint Strike Fighter is currently developmental in nature but should ultimately become MOTS when it enters production line delivery.

Effect of Kinnaird Reforms on Schedule

2.32 Implementation of the 2003 Kinnaird reforms has delivered a marked improvement in schedule performance for Defence capability acquisition projects. As shown in Table 2.5, average schedule variance to FOC for pre-Kinnaird MPR projects is 60 per cent, while the average variance for post-Kinnaird projects is 13 per cent. This is indicative of a general shift to more MOTS rather than developmental projects, as well as the age and stage of the post-Kinnaird projects.

Table 2.5 – Pre-Kinnaird and Post-Kinnaird Schedule Variance

 

Project

FOC Variance as at 30 Jun 15

In-year FOC Variance

Variance Factor

FOC Variance as at 30 Jun 15

In-year FOC Variance

Pre-Kinnaird

Wedgetail

80%

-1%

1.80

60%

1%

ARH Tiger Helicopters

64%

0%

1.64

Air to Air Refuel

66%

5%

1.66

Bushmaster Vehicles

0%

0%

1.00

Collins RCS

99%

0%

1.99

Hw Torpedo

42%

3%

1.42

Collins R&S

58%

-2%

1.58

ANZAC ASMD 2A

72%

0%

1.72

Post-Kinnaird

Joint Strike Fighter

0%

1%

1.00

13%

2%

AWD Ships

26%

9%

1.26

P-8A Poseidon

0%

0%

1.00

MRH90 Helicopters

50%

0%

1.50

Growler

0%

0%

1.00

MH-60R Seahawk

0%

0%

1.00

Overlander Medium/Heavy

-7%

2%

0.93

LHD Ships

0%

0%

1.00

Battlefield Airlifter

13%

13%

1.13

Overlander Light

5%

-9%

1.05

Additional Chinook

0%

0%

1.00

ANZAC ASMD 2B

61%

0%

1.61

Battle Comm. Sys. (Land)

7%

0%

1.07

Maritime Comms

0%

0%

1.00

UHF SATCOM

0%

0%

1.00

Battle Comm. Sys.

59%

2%

1.59

LHD Landing Craft

0%

9%

1.00

Schedule Variance Attribution

2.33 Schedule slippage for developmental and AMOTS projects is attributable to the higher levels of technical complexity and system integration risk. The higher the technical challenge, the higher the inherent risk to the schedule. Developmental projects are new and often at the leading edge of available technology. As such, it is not uncommon for projects, as they progress, to encounter unforeseen technical difficulties requiring significant modification that results in delay. Similarly, Australianisation can also encounter unforeseen technical difficulties and have an unanticipated impact on existing features of the baseline MOTS product. Rectifications of such issues often require extensive, time consuming remediation work.

2.34 Further analysis of 2014-15 MPR data has revealed additional drivers of schedule delays as shown in Table 2.6. Of the seven projects that underestimated technical complexity, five could also identify some overestimation of industry capability and performance, and another two were also affected by approved changes to scope (ANZAC ASMD Phases 2A/2B attained Government approval to acquire the more capable phased array radar based solution over the original very short range air defence system solution, though without an increase to schedule). In September 2012, Government announced a re-baselining of the AWD Ships construction schedule216, and limited platform availability was the primary driver for delays of Collins class submarine projects.

2.35 Some projects also have a greater delay to FOC as a result of Capability Manager decisions to adjust schedule in order to align with expected capability outcomes or operational imperatives.

Table 2.6 – Attribution of Schedule Variance Factors

Driver of Schedule Variance

Project

Platform availability

Collins RCS

Collins R&S

Industry Capability/Budget Adjustments

AWD

Technical complexity – underestimation by industry and/or Defence of the complexity of developmental and/or large scale integration projects.

FFG Upgrade

Wedgetail

Air to Air Refuel

ARH Tiger Helicopters

MRH90 Helicopter

Technical complexity and Scope Change

Anzac ASMD 2A

Anzac ASMD 2B

Capability Manager Decisions

Battle Comm. Sys.

Materiel Scope Performance

2.36 Materiel Scope Performance measures represent the key materiel capability performance attributes of a project delivered by Defence and industry, which, if not satisfied, could have a significant detrimental effect on the eventual suitability of equipment for operational service. The Materiel Scope Performance measures for each project, as defined in the Materiel Acquisition Agreement, are identified from the project approval documentation, including the Operational Concept Document and the Function and Performance Specification, which detail the capital equipment assets to be delivered.

2.37 For security classification reasons the MPR does not identify the individual measures for each of the projects. However, each PDSS has a percentage breakdown on how the project is tracking against its particular suite of capability elements.

2.38 The subjective ‘traffic light’ assessment of each element is indicative of:

  • Green: a high level of confidence that the capability outcome sought will be met;
  • Amber: the capability outcome being under threat but still considered manageable and able to be met; and
  • Red: at this stage, the capability outcome is unlikely to be fully met.

2.39 Materiel Capability Performance measures indicate the status of the materiel element of capability against the FMR milestone’s definition as at 30 June 2015 and are not necessarily indicative of each project’s ultimate ability to deliver the final intended scope. For the 2014-15 MPR, there are 240 individual Materiel Capability elements across the 25 projects with 234 green (or 97 per cent), six amber (or three per cent) and zero red.

Figure 2.9: Materiel Capability Performance measures for the 2014-15 Report

Figure 2.10: Trends for Materiel Capability Performance measures

2.40 Figure 2.9 indicates that a high percentage (97 per cent) of Materiel Capability Performance measures are currently reported as green.

2.41 The three per cent reported as amber is attributed to Joint Strike Fighter. The project assesses that Phase 2A/B (Combined Stages 1 and 2) will deliver its materiel requirements, noting there are a number of risks to achieving some of the materiel capabilities required to deliver IOC and FOC. These risks include: integration of the F-35A into Defence’s systems; final software builds meeting required functionality by IOC and FOC; establishing the sustainment capability; and, re-establishing the training systems. Mitigation strategies are in place for all the risks identified (more details are in the project’s PDSS).

2.42 Figure 2.10 shows previous MPRs’ Capability Performance Measures, as at 30 June for the respective years. The number of performance measures has increased from 138 across 22 projects in 2009-10 to 240 across the 25 projects in 2014-15. The increase in average number of Capability Performance Measures per project from 6.3 in 2009-10 to 9.6 in 2014-15, provides increased transparency of scope performance. Analysis of these measures indicates that Defence’s performance has improved over the last seven years, with the majority of current performance measures likely to be achieved.

3. Appendices

Appendix 1: Categorising Acquisitions

The DMO categorises its acquisition projects to enable it to differentiate between the complexities of business undertakings, focus management attention, provide a basis for professionalising its workforce and facilitate strategic workforce planning. Projects are graded into one of four Acquisition Categories (ACATs):

  • ACAT I – These are major capital equipment acquisitions that are normally the ADF’s most strategically significant. They are characterised by extensive project and schedule management complexity and very high levels of technical difficulty, operating, support and commercial arrangements;
  • ACAT II – These are major capital equipment acquisitions that are strategically significant. They are characterised by significant project and schedule management and high levels of technical difficulty, operating, support arrangements and commercial arrangements;
  • ACAT III – These are major or minor capital equipment acquisitions that have a moderate strategic significance to the ADF. They are characterised by the application of traditional project and schedule management techniques and moderate levels of technical difficulty operating, support arrangements and commercial arrangements; and
  • ACAT IV – These are major or minor capital equipment acquisitions that have a lower level of strategic significance to the ADF. They are characterised by traditional project and schedule management requirements and lower levels of technical difficulty, operating, support and commercial arrangements.

As the complexity of a project will vary over its life cycle, Defence reviews project acquisition categories at defined milestones between entry into the Defence Capability Plan and project completion.

The ACAT framework provides a recognised, consistent and repeatable methodology for categorising projects and aligning Project Managers’ certified experience and competencies to the complexity and scale of projects under management. Project Managers are assigned to acquisition projects on the basis that their Certified Professional Project Manager status is consistent with the project’s ACAT level.

The ACAT level of a project is assessed against six project attributes:

  • acquisition cost: the approved budget for the project;
  • project management complexity: the complexity of project management necessary for its execution;
  • schedule complexity: the inherent complexity brought about by delivery pressures on the project;
  • technical difficulty: the complexities associated with technical undertakings such as design and development, assembly, integration, test and acceptance;
  • operation and support: the complexity associated with readying the organisation and environment in which the system will be operated, supported and sustained; and
  • commercial: the readiness and capability of industry to develop, produce and support the required capability, and the complexity of the commercial arrangements being managed.

Appendix 2: Project Maturity Scores – Monitoring Progress

The DMO’s Project Maturity Score quantifies the maturity of a project by way of a score based on the Project Managers’ judgement at defined milestones in its capability development and acquisition phases. This score is then compared against an ideal or benchmark score for that milestone. A project’s maturity is assessed on 16 milestones across its lifecycle and for each of these milestones the ideal or benchmark condition is represented by a benchmark score as shown in Figure 1.

Figure 1: Benchmark Maturity Scores

The Project Maturity Score comprises a matrix of seven attributes:

  • Schedule;
  • Cost;
  • Requirement;
  • Technical Understanding;
  • Technical Difficulty;
  • Commercial; and
  • Operations and Support.

The Project Manager assesses the level of maturity that a project reaches at a particular milestone for each of these attributes on a scale of 1 to 10. Score assessment is made by selecting the most appropriate description that fits the question under the attributes columns. Project Maturity Scores provide a means of communicating in a simple fashion an indicative ‘as is’ versus a ’should be’ condition to inform decision making for each project. The scores are not precise and are not intended to enable exact comparisons across projects. Following is a description of the Project Maturity Score Attributes.217

 

Appendix 3: Lessons Learned

The Joint Committee of Public Accounts and Audit recommended in Report 422: Review of the 2009-10 Defence Materiel Organisation Major Projects Report, that the DMO include a Lessons Learned section in the MPR for projects that have met the exit criteria to report on their lessons learned at both the project level and the whole of organisation level.

Table 1 lists their lessons at the project level, against a whole of organisation level category.

Table 1: Lessons Learned – Project Level

Categories of Systemic Lessons

Project Lesson

Project Learned from

Resourcing

A reasonable presence of Australian Super Hornet Project Staff in the US is required to enable the Commonwealth adequate insight, influence and progress reporting of the USN and Boeing activities.

AIR 5349 Phase 1 – Bridging Air Combat Capability

Resourcing

Personnel resourcing, especially continuity in Business and Finance staff, requires careful management in project wind-down leading to FOC as project reporting and accurate financial accounting remains obligatory and at the same magnitude. Australian Super Hornet Project Office suffered when the business and finance responsibilities were reassigned from the Project Office in Canberra to Tactical Fighter Systems Program Office 12 months before FOC without an associated transfer of personnel. Furthermore, the level of work to account for assets and inventory procured by the project and the finance resource that would be required following FMR was underestimated causing the processing of Assets Under Construction to be adversely affected. This was further exacerbated by increased governance required through the utilisation of Quality Assurance Rollout Assist. To overcome these deficiencies, finance and logistics resources are being shared within Tactical Fighter Systems Program Office.

AIR 5349 Phase 1 – Bridging Air Combat Capability

Resourcing

The level of experience gained as a result of the Joint Standoff Weapon C-1 operational test and evaluation program has provided the DMO with the ability to streamline raise train sustain weapons test programs.

AIR 5349 Phase 2 – Bridging Air Combat Capability

Resourcing

Integrated Product Teams: Integrated product teams for all project disciplines (engineering, logistics, commercial, test and evaluation, and display development) were established with members from all major stakeholders (Commonwealth, prime and sub contractors, US and Canadian Government representatives). These teams met formally on a regular basis and with significant issues being raised with the overarching management integrated product team. As well as ensuring progress towards a common goal, the teams enabled the implementation of many other project initiatives that relied on quick and honest communication between all parties.

AIR 5376 Phase 2.1 – F/A-18 Hornet Upgrade

Resourcing

Sufficient resident project staff is important to ensure US Government and contractors understand our requirements and expectations.

AIR 5418 Phase 1 – Follow On Stand Off Weapon

Resourcing

The DMO needs to work closely with Australian Small to Medium Enterprise (SME) companies to ensure the SME resourcing effort and engineering demands in executing Defence contracts is not underestimated.

JP 2008 Phase 4 – Next Generation SATCOM Capability

Requirements Management

Risks associated with requirements instability, software development and systems engineering were known at the time of contract signature but in the light of subsequent events were clearly not adequately addressed in pre-contract negotiations. The experience underlines the importance of having well-defined and stable requirements at contract award, and of contractors having sound systems engineering and software development processes.

JP2043 Phase 3A – High Frequency Modernisation

Requirements Management

The accelerated procurement of major materiel is possible with off-the-shelf items currently in production, but the establishment of a sustainment solution is a challenge and requires early management oversight.

AIR 5349 Phase 1 – Bridging Air Combat Capability

Requirements Management

Interface Control Documents are not always correct or may not have been interpreted correctly during host platform design.

AIR 5418 Phase 1 – Follow On Stand Off Weapon

Requirements Management

Failure at project inception to articulate, tailor and agree naval standards to be applied to a ship designed and built to commercial ‘Classification Society’ standards has resulted in considerable debate and potential cost increase.

SEA 1444 Phase 1 – Armidale Class Patrol Boat

Requirements Management

The data generated by Defence Science Technological Organisation as part of the centre barrel test-to-destruction programme will result in a considerable cost saving to the project (due to a reduction in the number of aircraft requiring structural refurbishment programs 2) and an increased flexibility in aircraft modification induction dates.

AIR 5376 Phase 3.2 – F/A-18 Hornet Upgrade Structural Refurbishment

Requirements Management

Modifying an ageing weapon system such as the Hornet aircraft can present emergent work such as corrosion and cracking in the aircraft structure which must be rectified while the aircraft is disassembled. Adequate project contingency budget and schedule must be programmed to accommodate such uncertainties.

AIR 5376 Phase 3.2 – F/A-18 Hornet Upgrade Structural Refurbishment

Requirements Management

Resourcing

Increased need for collaboration due to diverse systems integration. As DMO projects become heavily integrated and dependent on one another, such as interoperable battle management systems, the technical challenges to success become frequent. Close collaboration with the customer, supplier and related DMO projects, early in the process, is essential to understanding the interoperability requirements and developing suitable test plans and schedules that achieve the outcomes of the customer. Regular joint working groups are an excellent way to achieve this.

Land 17 Phase 1A – Artillery Replacement

Requirements Management

Resourcing

Close stakeholder engagement – whilst delivering a novel and technically complex system to Army, the project experienced a constantly changing environment in terms of customer requirements. In order to ensure the customer’s needs are met through timely and accurate representation of requirements to suppliers, continuous face to face stakeholder engagement is essential. Regular working groups with both the customer and supplier are an excellent way to achieve this.

Land 17 Phase 1A – Artillery Replacement

Requirements Management

Requirements and specifications must be well defined and agreed before contract signature.

Where detailed specifications cannot be defined fully prior to contract signature, such as when systems definition and new design work must be undertaken within a developmental project phase, then the end capability requirements and priorities must be well defined and agreed.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Requirements Management

Close liaison and communication with Navy stakeholders is required throughout the project life. Navy regulator engagement must be open and transparent from the project commencement to FOC so that the Navy Acceptance Certificate (T1338) residual issues/risks are well understood and easily accepted. Where capability delivered falls short of Navy customer initial expectations as agreed in the MAA, the process of securing concessions/agreement is needed to allow efficient and prompt project closure to avoid/limit inefficient use of resources.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Requirement Management

Contract Management

Two stage contracting – Contract Development Agreements facilitate early positive engagement with the contractor, joint development of the resultant fixed price contract and establishes an effective and cooperative work environment.

SEA 1390 Phase 4B – SM-1 Missile Replacement

Requirement Management

Contract Management

For significant and high technological upgrades to major systems the acquirer (Commonwealth) acting as the Procurement Coordinator managing separate contracts directly with OEMs allows for better risk management, schedule control and influence on the quality of the contracted supplies.

SEA 1390 Phase 4B – SM-1 Missile Replacement

First of Type Equipment; and

Off-The-Shelf Equipment

Weapons acquired under the scope of the Project proved to be cost effective for the Commonwealth as the weapons were USN common and this also assisted in providing common integration and technical input from the USN.

AIR 5349 Phase 2 – Bridging Air Combat Capability

First of Type Equipment; and

Off-The-Shelf Equipment

FMS is a good procurement vehicle when a US Program is truly MOTS. However, FMS provides little ability for DMO to manage capability and associated risk when the US program is less mature.

AIR 5349 Phase 2 – Bridging Air Combat Capability

First of Type Equipment

Stability of interfaces on ageing platforms may not be reliable, leading to an underestimation of integration complexity.

AIR 5418 Phase 1 – Follow On Stand Off Weapon

First of Type Equipment

Host platform upgrades not required in the past may now be required, due to the minimum technical performance requirements of new systems to be integrated.

AIR 5418 Phase 1 – Follow On Stand Off Weapon

First of Type Equipment

FMS is a good procurement vehicle when a US program is mature. However, FMS provides little ability for DMO to manage capability and associated risk when US program is less mature and the Commonwealth is the integrator of project outcomes.

AIR 5418 Phase 1 – Follow On Stand Off Weapon

First of Type Equipment

For a new or significantly modified design there will be a number of design changes emanating from initial sea trials. The aggressive delivery schedule for the Armidale Class Patrol Boat did not allow time for changes from initial sea trials to be built into the follow-on build boats prior to their construction. This resulted in an evolving design baseline throughout the production phase that was not stabilised until after delivery of the last boat. Consequently the redesign, build, test and acceptance aspects of boats built after the first of class became unnecessarily complicated, expensive and inefficient. Time should be allowed after the first (or second depending on the size of the class) boat build to conduct sea trials and modify and stabilise the design as appropriate prior to the main production run.

SEA 1444 Phase 1 – Armidale Class Patrol Boat

First of Type Equipment

Substantial development in the information technology field over the extended term of the project means that some elements of the system could now be delivered via off-the-shelf solutions or by other contemporary production, rather than attracting extended software development, thereby reducing risk, schedule and possibly cost.

The proposed approach for capability development involving substantial software or software systems development over an extended period needs to be considered carefully to enable best use of emerging developments within appropriate risk, schedule and cost constraints.

JP2043 Phase 3A – High Frequency Modernisation

First of Type Equipment

Procurements that include significant change to software-intensive systems and complex system integration have many inherently high-risk activities, which must be analysed and appropriate risk mitigation processes applied. Such risks are often under-estimated in the planning phase.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

First of Type Equipment

Requirements Management

Major maritime software development should be incremental and delivery does not have to be aligned with the platform modification program.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Contract Management

An acquisition strategy combining the acquisition and support of the fleet in one single contract rather than the traditional acquisition model followed by a separate support contract can lead to significant disputation and complications in closing out latent defects where the prime contractor is not also the builder. Invariably, once the capability is delivered and being operated and the contract is into the sustainment phase, there is a greater reluctance on the part of the prime contractor to progress rectification of build-related defects that may result in a cost to the contractor and disputation with the builder.

SEA 1444 Phase 1 – Armidale Class Patrol Boat

Contract Management

The Armidale Class Patrol Boat In Service Support (ISS) contract is principally a 15 year fixed price contract with the option for a five year extension. Existing contract provisions provide no incentive to the contractor to improve or implement changes in the delivery of support activities that would deliver benefits/savings to both the contractor and the Commonwealth. In particular, there is no incentive to make savings over the life of the contract that would generate a reduction in the ISS fee. Incentives need to be built into contracts beyond the acquisition phase.

SEA 1444 Phase 1 – Armidale Class Patrol Boat

Contract Management

Proactive Contract Management: Due to the incremental contracting nature of the project, joint and proactive contract management was essential. Regular commercial integrated product teams provided an effective vehicle to manage the prime integration contract with Boeing and FMS cases with the US Government.

AIR 5376 Phase 2 – F/A-18 Hornet Upgrade

Contract Management

Participation in face to face financial working groups bi-annually resulted in significant financial savings under the WGS MOU. The cost associated with overseas travel was far outweighed by the financial savings and clarity of financial projections.

JP 2008 Phase 4 – Next Generation SATCOM Capability

Contract Management

Best practice would suggest that for a capability acquisition that includes significant software development, a contract that allows for both fixed price elements as well as alternative cost structures which include appropriate controls, incentive and penalty models that can be applied to the highly developmental elements involving significant risk, may be appropriate.

Milestone payments could be selected for those deliverables that have well defined objectives and the alternative payment method with incremental work packages could be applied to the software aspect of the project. This approach would require strict controls and metrics to limit the risk to the Commonwealth.

JP2043 Phase 3A – High Frequency Modernisation

Contract Management

The contract schedule must be accepted by all parties as realistic and achievable from the outset. Each party must be committed to achievement of the schedule and aware of the consequences of non-achievement, plus any provisions for delay outside the contractor’s control.

The contract should contain:

  • milestones which enable the Commonwealth to unambiguously assess Contractor performance from the outset of the Contract;
  • with the exception of non-recurring engineering effort, payment of all or a substantial part of the contract price should be subject to achievement of clear project milestones;
  • milestones should reflect delivery of contracted requirements to the Commonwealth, not just reaching intermediate points on the timeline;
  • milestones which enable use of the equipment and supplies (such as Integrated Logistic System (ILS) and training) should be given similar weight as delivery of the equipment itself;
  • payment on achievement of milestones should be conditional on achievement of previously scheduled milestones;
  • payment of milestones should also be tied to remedies under the contract to allow the Commonwealth to seek redress; and
  • clear entitlements of the Commonwealth to access all contractor project data (including internal workforce planning data) so as to be able to make informed assessments if a milestone is not achieved.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Contract Management

Implement a progressive acceptance methodology from the outset for all project data / documentation supplies and requirements acceptance objective quality evidence in order to progressively increase confidence of all stakeholders involved with regard to project outcomes.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Contract Management

The establishment of commercial contracts were based entirely on deliverable items and artefacts (software build states and/or documentation in electronic format) and progress against agreed milestones. Payments were made on delivery acceptance and milestones achieved in accordance with the contract. Reliance on Contract Earned Value Management requires considerable effort and expertise on the part of the Project authority to adequately assess contractor performance, and was not utilised or necessary to achieve ‘value for money’ project objectives.

SEA 1390 Phase 4B – SM-1 Missile Replacement

Contract Management

Significant efficiencies were achieved for ease of handling, delivery, traceability and tracking of documents through electronic document delivery which was encouraged in all commercial contracts and the primary FMS case.

SEA 1390 Phase 4B – SM-1 Missile Replacement

Contract Management

Schedule Management

Accessibility requirements should be agreed, specified and documented early in the contracting process to minimise risk of incurring excusable delays when access to the system to be upgraded is constrained due to operational reasons.

JP2043 Phase 3A – High Frequency Modernisation

Contract Management

Schedule Management

Foreign Military Sales (FMS) Schedule planning – When factoring FMS related schedules, there is an inclination to schedule the acceptance of the case without allowing sufficient schedule float to accommodate potential delays. Often, there will be a delay post case acceptance whilst the US Government supporting office seeks to contract their suppliers - this delay could be some six to nine months in some instances.

When negotiating lead times, it is essential to gain an understanding of the contracting and procurement processes of the source country.

LAND 19 Phase 7A – Counter-Rocket Artillery & Mortar

Contract Management

Schedule Management

A fundamental issue to consider at the time of capability and project definition is how the capability should be acquired. If the project is developmental, then consideration should be given to methods other than a fixed price contract for achieving the capability.

Contracts should include appropriate clauses that recognise the complexities of verifying and validating a software development project.

Multi platform upgrades should allow for implementation and testing/acceptance of the first platform without committing to a full class upgrade of all platforms.

Conducting an upgrade of an existing capability concurrent with scheduled maintenance availability requires very detailed planning and careful consideration of the supporting contract clauses.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Off-the-Shelf Equipment

Requirements Management

Sole source relationships: In a sole source relationship, projects might consider the Commonwealth of Australia would lack leverage over suppliers when negotiating contractual outcomes due to the absence of supplier competition. In this case, early and strong face-to-face engagement between the project office and FMS staff in the US and Saab staff in Sweden assured professional and outcome focused relationships.

Using other Defence establishments for training, using partner nations to leverage open source commercial information to gain a sense of value for money in Australia’s circumstance, and holding the supplier’s reputation for further business opportunities at risk from poor performance in the current project are options available to the Commonwealth when negotiating sole source contracts.

LAND 19 Phase 7A – Counter-Rocket Artillery & Mortar

Contract Management

Requirements Management

For very large developmental contracts, project managers must ensure that the contractor maintains sufficient focus and resourcing on documenting what is being delivered and how to use it (through ILS, configuration management and training).

Milestones must be structured so that the contractor is not tempted to focus on equipment deliverables only. Payment for equipment milestones should be conditional on achievement of related ILS milestones.

The contractor should be clear on configuration management requirements of ILS products in an incremental delivery software development project. This should align to milestones and remedies in the contract.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Contract Management

Requirements Management

Objective acceptance criteria are required to ensure there is no scope for dispute as to whether the criteria have been met.

Criteria for determining contractual achievement should support those criteria used by Defence for determining achievement by DMO of the measure of effectiveness in the MAA.

SEA 1390 Phase 2.1 – Guided Missile Frigate Upgrade

Off-the-Shelf Equipment

Requirements Management

Resourcing

Support arrangements – Accelerated Acquisitions. Whilst they deliver equipment quickly, Integrated Logistics Support considerations (e.g. Net Personnel and Operating Cost) can take considerable time when implemented retrospectively. Limitations to resources and costs need to be considered at the early stages of the project to enable robust planning.

LAND 19 Phase 7A – Counter-Rocket Artillery & Mortar

Military Off-The-Shelf Equipment

Considerable acceleration of the standard acquisition cycle is possible when the major supplies being procured are off-the-shelf production items. However, acceleration of establishment of support systems may be more difficult and should attract early management focus.

AIR 8000 Phase 3 – C17 Globemaster III Heavy Airlifter

Schedule Management

Closely monitor the return of repairable parts for the production installation phase to ensure no delays are experienced during the rebuild of each aircraft being modified. The more severe action that could be taken is to direct that repairable parts are not removed during the aircraft modification.

Close monitoring of modification kit holdings and subsequent timely procurement is required to ensure kit deficiencies do not arise impacting on production schedule.

AIR 5376 Phase 3.2 – F/A-18 Hornet Upgrade Structural Refurbishment

Schedule Management

Joint Risk and Schedule Management: Through the integrated product teams a common risk and schedule management methodology was implemented for the entire project. Boeing, as the prime integrator, provided a vehicle to manage both risk and schedule in a common framework. Pro-active management of risks was encouraged and many mitigation strategies, particularly in respect to display development, were implemented to avoid schedule delays.

AIR 5376 Phase 2 – F/A-18 Hornet Upgrade

Schedule Management

Resourcing

Governance

International Traffic in Arms Regulations (ITAR) – as the number of ITAR controlled items being acquired by Defence increases, the need for close engagement with the Defence Export and Controls office and a detailed data management plan early in the project becomes essential. The movement and transfer of ITAR controlled items between countries and parties is governed by Technical Assistance Agreements and Third Party Retransfers, these documents are time consuming to develop with the US government and must be commenced early in the project.

Land 17 Phase 1A – Artillery Replacement

Governance

Considerable acceleration of the acquisition cycle for the WGS program necessitated a strengthening of the governance process to ensure lines of authority and responsibility were clear in the definition of business need and option analysis.

JP2008 Phase 4 – Next Generation SATCOM Capability

Lessons Learned – Whole of Organisation Level

The 2014-15 MPR builds on the lessons learned (at the organisational level) reported in last year’s report. Set out below is a summary of progress against the categories of lessons learned.

Military Off-The-Shelf Equipment

The lessons learned in relation to Military-Off-The-Shelf equipment are incorporated in DMO policies.

Schedule Management

The lessons learned in relation to Schedule Management are incorporated in Chapter 1, Schedule Management, 1.26-1.30, page 95.

Requirements Management

To address issues associated with understanding the level of maturity of requirements and progression through the various systems engineering reviews, the standard procedure covering Function and Performance Specification (FPS) development was amended to provide further guidance on the maturity levels for the FPS when it is progressing through the capability development process. Additionally, policy, guidance handbook and a standard procedure was promulgated to assist with assessing the suitability and risks for system reviews conducted in acquisition.

To improve the quality of objective evidence, or data, relating to acquiring and sustaining materiel systems, emphasis has been applied in the areas of configuration management and Verification and Validation (V&V). Improvements in the V&V policy, handbook and a standard procedure were issued to provide a more robust approach in utilising risk-based methodology to ensure that the necessary objective evidence is obtained. To reinforce the data quality of materiel systems an overarching configuration management policy has been promulgated to ensure reliable materiel system status throughout acquisition and sustainment.

Appendix 4: Glossary

Acquisition Category

See Appendix 1.

Additional Estimates

Where amounts appropriated at Budget time are required to change, the Parliament may make adjustments to portfolios through the Additional estimates process.

ASDEFCON

AUStralian DEFence CONtracting suite of contracting templates.

Capability

The power to achieve a desired operational effect in a nominated environment within a specified time and to sustain that effect for a designated period.

Capability is generated by the Fundamental Inputs to Capability.

Capability Manager

A Capability Manager (CM) has the responsibility to raise, train and sustain capabilities. In relation to the delivery of new capability or enhancements to extant capabilities through the Defence Capability Plan, CMs are responsible for delivering the agreed capability to the Government, through the coordination of the Fundamental Inputs to Capability. Principal CMs are Chief of Navy (CN), Chief of Army (CA), Chief of Air Force (CAF), Deputy Secretary Intelligence and Security (DEPSEC I&S) and Chief Information Officer (CIO).

Capital Equipment

Substantial end items of equipment such as ships, aircraft, armoured vehicles, weapons, communications systems, electronics systems or other armaments that are additional to, or replacements for, items in the Defence inventory.

Contract Change Proposal

This is a formal written proposal by the Commonwealth or the contractor, prepared in accordance with the terms and conditions of the contract, to change the contract after the effective date. After agreement by the parties, the contract is amended in accordance with the processes established in the contract.

Corporate Governance

The process by which agencies are directed and controlled, and encompasses; authority, accountability, stewardship, leadership, direction and control.

Defence Procurement Review 2003 (Kinnaird Review)

In August 2003 the Defence Procurement Review 2003 published its findings on the problems associated with major Defence acquisition projects with the review being chaired by Mr Malcolm Kinnaird. This became known as the Kinnaird Review.

DEFPUR 101

DEFence PURchasing (101) contracting template used prior to the formation of the DMO.

Financial Management and Accountability Act 1997

The Financial Management and Accountability Act 1997 (FMA Act) establishes the regulatory framework for financial management within public sector agencies, including the DMO. This was superceded by the Public Governance, Performance and Accountability Act 2013 that came into effect on 1 July 2014.

Firm Price Contract

A firm price contract is unalterable in all respects for the duration of the contract, except where the parties agree to a contract amendment which alters that contract price.

Foreign Military Sales

The US Department of Defense’s Foreign Military Sales program facilitates sales of US arms, Defense services, and military training to foreign governments.

Forward Estimates

The level of proposed expenditure for future years (based on relevant demographic, economic and other future forecasting assumptions).

The Government requires forward estimates for the following three financial years to be published in each annual Federal Budget paper.

Function and Performance Specification

A specification that expresses an operational requirement in function and performance terms. This document forms part of the Capability Definition Document.

Materiel Acquisition Agreement

An agreement between Defence and the DMO which states in concise terms what services and products the DMO (as a supplier) will deliver, for how much and when.

Materiel Capability Performance Measures

The traffic lights, based on a subjective assessment, indicate:

  • Green: There is a high level of confidence that they will be met;
  • Amber: Are under threat but still considered as manageable and able to be met; and
  • Red: At this stage are unlikely to be met.

Memorandum of Understanding (MOU)

A memorandum of understanding is a document setting out an agreement, usually between two government agencies.

Minor Capital Acquisition Project

A Defence project in which the proposed equipment falls within the definition of capital equipment but does not meet the criteria in the definition of a major project.

Off-The-Shelf

A product that is available for purchase, which has been delivered to another military or government body or commercial enterprise.

Operational Concept Document

The primary reference for determining fitness-for-purpose of the desired capability to be developed. This document forms part of the Capability Definition Document.

Operational Test and Evaluation (OT&E)

Test and evaluation conducted under realistic operational conditions with representative users of the system, in the expected operational context, for the purpose of determining its operational effectiveness and suitability to carry out the role and fulfil the requirement that it was intended to satisfy.

Platforms

Refers to air, land, or surface or sub-surface assets that are discrete and taskable elements within the ADF.

Portfolio Budget Statement

A document presented by the Minister to the Parliament to inform Senators and Members of the basis for Defence/DMO budget appropriations in support of the provisions in Appropriation Bills 1 and 2. The statements summarise the Defence/DMO budget and provides detail of outcome performance forecasts and resources in order to justify agency expenditure.

Prime System Integrator

The entity that has prime responsibility for delivering the mission and support systems.

Public Governance, Performance and Accountability Act 2013

The Public Governance, Performance and Accountability Act 2013 came into effect on 1 July 2014 and superseded the Financial Management and Accountability Act 1997. It is a Commonwealth Act about the governance, performance and accountability of, and the use and management of public resources by, the Commonwealth, Commonwealth entities and Commonwealth companies, and for related purposes.

Schedule Compliance and Risk Assessment Method

The DMO developed a Schedule Compliance Risk Assessment Methodology (SCRAM), which provides a framework for identifying and communicating the root causes of schedule slip and the recommendations for going forward to Program and Executive-level management. It is based on a repeatable process that uses a root cause analysis of a schedule slip model to locate factors that impact program schedule along with a “health check” of the documented schedule, assessing its preparation and probability distribution of completion dates.

System Program Office

One of the core business units in the DMO. They provide a crucial link between the DMO and its customers. They provide acquisition and sustainment services to the ADF.

Test Concept Document

The basis for the DMO’s development of the Test and Evaluation Master Plan for a project, and is the highest level document that considers test and evaluation requirements within the capability systems’ life-cycle. This document forms part of the Capability Definition Document.

Variable Price Contracts

Variable price contracts provide for the contractor to be paid a fixed fee for performance of the contract, subject to certain variations detailed in the contract. Variable price contracts may allow for variations in exchange rates, labour and/or material costs.

!Part 3. Project Data Summary Sheets

Assurance by the Auditor-General and the Secretary of Defence

Independent Review Report by the Auditor-General on the Department of Defence’s Project Data Summary Sheets

To the President of the Senate
To the Speaker of the House of Representatives

Scope

The review of the accompanying 25 Project Data Summary Sheets (PDSSs) as at 30 June 2015, including the ‘Statement by the Secretary of Defence’, was undertaken as a priority assurance review under section 19A(5) of the Auditor-General Act 1997.

My review is designed to provide assurance that the information contained in each PDSS has been prepared in accordance with the 2014–15 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit. The 25 projects are listed in Attachment A.

My review encompassed the information in each PDSS, including the cost, schedule performance, and capability to be delivered against approved requirements, but did not include an assessment of the following information, which is outside the scope of the review, as set out in the Terms of the Priority Assurance Review Engagement, agreed with the Secretary of the Department of Defence (Defence):

  1. Section 1.3 Project Context—Major Risks and Issues, Section 5.1 Major Project Risks, and Section 5.2 Major Project Issues;
  2. Section 4.1 Measures of Materiel Capability Delivery Performance; and
  3. ‘forecasts’ of future dates regarding a project’s expected achievement of delivery schedules and capability where included in each PDSS.

The information in clauses (a) to (c) above, has not been included in the scope of the review due to the lack of a system or systems from which to provide complete and accurate evidence, in a sufficiently timely manner to facilitate the review. Accordingly, the conclusion of this review does not provide any assurance in relation to this information.

Secretary’s Responsibility for the Project Data Summary Sheets

The Secretary of Defence is responsible for the preparation and presentation of the unclassified PDSSs for the 25 projects outlined in the scope, in accordance with the Guidelines. This responsibility includes ensuring the completeness and accuracy of each project’s cost and schedule performance, and capability to be delivered against approved requirements, in each PDSS.

Auditor’s Responsibility

My responsibility is to express an independent conclusion based on my review. My review has been conducted in accordance with the Australian Standard on Assurance Engagements, ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information issued by the Australian  Auditing and Assurance Standards Board, which forms part of the Australian National Audit Office Auditing Standards.

My review is designed to enable me to obtain sufficient appropriate evidence to form a conclusion on whether anything has come to my attention to indicate that the information and data in the PDSSs, within the scope of my review, has not been prepared, in all material respects, in accordance with the Guidelines.

Independence

In conducting the review, I have followed the independence requirements of the Australian National Audit Office, which incorporate the requirements of the Australian accounting profession.

Review criteria and methodology

The criteria that have been used to conduct my review are based on the Guidelines and include whether Defence has procedures in place designed to ensure that project information and data was recorded in a complete and accurate manner for each project.

I have conducted the review of the PDSSs, as explained in the above Scope section, for the 25 projects by making such enquiries and performing such procedures as I, in my professional judgement, considered reasonable in the circumstances including:

  • examination of each PDSS and the documents and information relevant to them;
  • a review of relevant processes and procedures used by Defence in the preparation of the PDSSs;
  • an assessment of the systems and controls that support project financial management, risk management, and project status reporting, within the Australian Defence organisation;
  • interviews with persons responsible for the preparation of the PDSSs and those responsible for the management of the 25 projects;
  • taking account of industry contractor comments provided to the ANAO and Defence on draft PDSS information;
  • assessing the assurance by Defence managers attesting to the accuracy and completeness of the PDSSs;
  • examination of the representations by the Chief Finance Officer of Defence supporting the project financial assurance and contingency statements, and the independent third-party review of the project financial assurance statements;
  • examination of representations, provided by the Capability Managers, relating to each project’s progress toward Initial and Final Materiel Release, and Initial and Final Operational Capability; and
  • examination of the ‘Statement by the Secretary of Defence’, including significant events occurring post 30 June, and management representations by the Secretary of Defence.

A limited assurance engagement of this nature provides less assurance than a reasonable assurance engagement.

Conclusion

Based on my review described in this report, nothing has come to my attention that causes me to believe that the information and data in the PDSSs, within the scope of my review, has not been prepared, in all material respects, in accordance with the Guidelines.

Grant Hehir
Auditor-General
Canberra ACT
22 December 2015

Attachment A - List of Projects
  • AIR 6000 Phase 2A/2B New Air Combat Capability
  • SEA 4000 Phase 3 Air Warfare Destroyer Build
  • AIR 7000 Phase 2B Maritime Patrol and Response Aircraft System
  • AIR 5077 Phase 3 Airborne Early Warning and Control Aircraft
  • AIR 9000 Phase 2/4/6 Multi-Role Helicopter
  • AIR 5349 Phase 3 EA-18G Growler Airborne Electronic Attack Capability
  • AIR 9000 Phase 8 Future Naval Aviation Combat System Helicopter
  • LAND 121 Phase 3B Medium Heavy Capability, Field Vehicles, Modules and Trailers
  • JP 2048 Phase 4A/4B Amphibious Ships (LHD)
  • AIR 87 Phase 2 Armed Reconnaissance Helicopter
  • AIR 5402 Air to Air Refuelling Capability
  • AIR 8000 Phase 2 Battlefield Airlift – Caribou Replacement
  • LAND 116 Phase 3 Bushmaster Protected Mobility Vehicle
  • LAND 121 Phase 3A Field Vehicles and Trailers
  • SEA 1448 Phase 2B ANZAC Anti-Ship Missile Defence
  • AIR 9000 Phase 5C Additional Medium Lift Helicopters
  • JP 2072 Phase 2A Battlespace Communications System
  • SEA 1439 Phase 4A Collins Replacement Combat System
  • SEA 1442 Phase 4 Maritime Communications Modernisation
  • SEA 1429 Phase 2 Replacement Heavyweight Torpedo
  • JP 2008 Phase 5A Indian Ocean Region UHF SATCOM
  • SEA1439 Phase 3 Collins Class Submarine Reliability and Sustainability
  • SEA 1448 Phase 2A ANZAC Anti-Ship Missile Defence
  • LAND 75 Phase 3.4 Battlefield Command Support System
  • JP 2048 Phase 3 Amphibious Watercraft Replacement

Statement by the Secretary of Defence

On 1 July 2015, the Defence Materiel Organisation (DMO) transitioned to the newly established Capability Acquisition and Sustainment Group (CASG) within the Department of Defence as recommended by the Defence First Principles Review. CASG continues to  have the  former DMO’s objective to provide the materiel equipment and sustainment elements of capability for the ADF in an effective, efficient, economical and safe manner. The status of the Project Data Summary Sheets (PDSSs) for the 2014-15 Major Projects Report is at 30 June 2015, when DMO was still a listed separate entity.

The attached PDSSs for the 25 major projects included in this report have been prepared in accordance with the Guidelines developed by the former DMO in consultation with the Australian National Audit Office (ANAO) and endorsed by the Joint Committee of Public Accounts and Audit.

Project Status as at 30 June 2015

In my opinion, the Project Data Summary Sheets comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2015. In stating this opinion, and in agreement with the ANAO, I acknowledge that the following sections of each PDSS are not covered in the scope of the Auditor-General’s assessment:

  • Section 1.3 Project Context–Major Risks and Issues, Section 4.1 Measures of Materiel Capability Delivery Performance, Section 5.1 Major Project Risks, Section 5.2 Major Project Issues; and
  • Future dates that are ‘forecasts’ regarding a project’s expected achievement of delivery schedules and capability where included in Sections 1 and 3 of each PDSS.

Significant Events Occurring Post 30 June 2015

In stating  this  opinion,  I  acknowledge  the  following  material  events  have  occurred  post 30 June 2015:

SEA 4000 Phase 3 Air Warfare Destroyer Build

The $1.2 billion Real Cost Increase for the project, advised by government on 22 May 2015218 was approved in July 2015.

AIR 9000 Phase 8 Future Naval Aviation Combat System Helicopter

The project achieved Initial Operational Capability (IOC) on 25 September 2015.

JP 2048 Phase 4A/4B Amphibious Ships (LHD) and JP 2048 Phase 3 Amphibious Watercraft Replacement (LLC)

JP 2048 Phase 4A/4B – NUSHIP Adelaide completed her final sea trials on 28 August 2015 and delivery from BAE Systems Australia to CASG took place on 21 October 2015. Limitations in the current contracting model have led to restructuring to include intermediate milestones. As a result, the forecasted date for Final Materiel Release has been revised to be achieved along similar timeframes to the revised Final Acceptance date of November 2016.

JP 2048 Phase 3 (LLC) achieved IOC in August 2015 and JP 2048 Phase 4A/4B (LHD) achieved IOC in November 2015. These platforms are part of the Amphibious Capability for which the Navy is managing a number of integration risks to achieve Final Operational Capability (FOC).

AIR 87 Phase 2 Armed Reconnaissance Helicopter

The declaration of FOC for the ARH Tiger may be delayed as Defence is awaiting resolution of the electronic self-protection capability.

AIR 8000 Phase 2 Battlefield Airlift – Caribou Replacement

In late 2012, the US Air Force advised their intention to divest their C-27J program and ceased all C-27J activity. US Air Force divestment related risks highlighted at combined pass for the acquisition of the C-27J, have been realised. Defence is preparing formal advice to government on the impacts to project schedule.

SEA 1448 Phase 2A and 2B ANZAC Anti-Ship Missile Defence

The project achieved IOC in September 2015.

JP 2008 Phase 5A Indian Ocean Region UHF SATCOM

ViaSat and the Commonwealth have been working to rebaseline the contract through a Contract Change Proposal. Since 30 June 2015, further work has been undertaken by both parties to establish a solid basis for delivering the agreed capability. The agreed position, subject to formal signature, will allow for the capability to be delivered before the previously advised date of June 2018.

LAND 75 Phase 3.4 Battlefield Command Support System

The project achieved Final Acceptance in October 2015.

Dennis Richardson
Secretary
Department of Defence
18 December 2015

Project Data Summary Sheets

Due to the complex nature of the material, the Project Data Summary Sheets (PDSS) are available in PDF only. You can download the complete PDSS section or individual PDSS:

  1. AIR 6000 Phase 2A/2B – New Air Combat Capability (Joint Strike Fighter)
  2. SEA 4000 Phase 3 – Air Warfare Destroyer Build (AWD Ships)
  3. AIR 7000 Phase 2B – Maritime Patrol and Response Aircraft System (P-8A Poseidon)
  4. AIR 5077 Phase 3 – Airborne Early Warning and Control Aircraft (Wedgetail)
  5. AIR 9000 Phase 2/4/6 – Multi-Role Helicopter (MRH90 Helicopters)
  6. AIR 5349 Phase 3 – EA-18G Growler Airborne Electronic Attack Capability (Growler)
  7. AIR 9000 Phase 8 – Future Naval Aviation Combat System Helicopter (MH-60R Seahawk)
  8. LAND 121 Phase 3B – Medium Heavy Capability, Field Vehicles, Modules and Trailers (Overlander Medium/Heavy)
  9. JP 2048 Phase 4A/4B – Amphibious Ships (LHD)
  10. AIR 87 Phase 2 – Armed Reconnaissance Helicopter (ARH Tiger Helicopters)
  11. AIR 5402 – Air to Air Refuelling Capability (Air to Air Refuel)
  12. AIR 8000 Phase 2 – Battlefield Airlift – Caribou Replacement (Battlefield Airliftery)
  13. LAND 116 Phase 3 – Bushmaster Protected Mobility Vehicle (Bushmaster Vehicles)
  14. LAND 121 Phase 3A – Field Vehicles and Trailers (Overlander Light)
  15. SEA 1448 Phase 2B – ANZAC Anti-Ship Missile Defence (ANZAC ASMD 2B)
  16. AIR 9000 Phase 5C – Additional Medium Lift Helicopters (Additional Chinook)
  17. JP 2072 Phase 2A – Battlespace Communications System (Battle Comm. Sys. Land)
  18. SEA 1439 Phase 4A – Collins Replacement Combat System (Collins RCS)
  19. SEA 1442 Phase 4 – Maritime Communications Modernisation (Maritime Comms)
  20. SEA 1429 Phase 2 – Replacement Heavyweight Torpedo (Hw Torpedo)
  21. JP 2008 Phase 5A – Indian Ocean Region UHF SATCOM (UHF SATCOM)
  22. SEA 1439 Phase 3 – Collins Class Submarine Reliability and Sustainability (Collins R&S)
  23. SEA 1448 Phase 2A – ANZAC Anti-Ship Missile Defence (ANZAC ASMD 2A)
  24. LAND 75 Phase 3.4 – Battlefield Command Support System (Battle Comm. Sys.)
  25. JP 2048 Phase 3 – Amphibious Watercraft Replacement (LHD Landing Craft)

The PDSS are also available in the full report PDF available at Related documents on this page. The full report PDF also includes bookmarks and a linked contents list.

!Part 4. JCPAA 2014–15 Major Projects Report Guidelines

The JCPAA 2014–15 Major Projects Report Guidelines are available in PDF only.

Download the Major Projects Report Guidelines (PDF).

Footnotes

1 The Minister for Defence, Senator the Hon. Marise Payne, Address to the Sea Power Conference, 7 October 2015.

2 The then Minister for Defence, the Hon. Kevin Andrews MP, Address to ADM Cyber Security Summit, 17 June 2015.

3 The then Minister for Defence, the Hon. Kevin Andrews MP, Minister for Defence – First Principles Review of Defence, 1 April 2015.

4 Department of Defence, Defence Annual Report 2014–15, Volume One, Performance, governance and accountability, p. 103.

5 ibid., Volume Two, Audited financial statements, p. 146.

6 Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Chapter 2, pp. 1–2.

7 For further information on the First Principles Review refer to paragraphs 1.4 to 1.6 in Part 1 of this report.

8 JCPAA, Report 448, Review of the 2013–14 Defence Materiel Organisation Major Projects Report, May 2015, Foreword, p. v.

9 The 2014–15 Major Projects Report Guidelines (the Guidelines) were endorsed by the JCPAA in September 2014 and were revised following the release of the First Principles Review. The revised Guidelines received endorsement from the JCPAA in October 2015. Refer to Part 4 of this report.

10 Based on information contained in the Approved Major Capital Investment Program provided to the ANAO by the new Capability Acquisition and Sustainment Group (formerly the DMO).

11 Commonwealth of Australia, JCPAA, Defence major projects report 2012–13, 20 March 2014, pp. 1–3.

12 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 7, p. xv.

13 IMR and FMR are milestones that mark the completion and release of acquisition project supplies required to support the achievement of Initial Operational Release and FOC respectively. They are defined in the MAA [Materiel Acquisition Agreement]. Department of Defence, Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, pp. B2–B3.

14 IOC and FOC are the points when the first or final subset of a capability system that can be operationally employed is realised. They are capability states endorsed at project approval at Second Pass, and reported as having been reached by the Capability Manager. ibid.

15 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2013, pp. 38–39.

16 The 2014–15 Major Projects Report Guidelines are included in Part 4 of this report.

17 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2013, p. 28.

18 JCPAA, Report 448, Review of the 2013–14 Defence Materiel Organisation Major Projects Report, May 2015, p. ix.

19 Sustainment involves the provision of in-service support for specialist military equipment, including platforms, fleets and systems operated by Defence. Typical services include repair and maintenance, engineering, supply, configuration management and disposal action. Department of Defence, Defence Portfolio Budget Statements 2014–15, May 2015, p. 176.

20 Further information on the management of the disposal of specialist military equipment can be found in the ANAO Report No.19 2014–15, Management of the Disposal of Specialist Military Equipment, February 2015.

21 Further information on Materiel Sustainment Agreements can be found in the ANAO Report No.30 2014–15, Materiel Sustainment Agreements, April 2015.

22 These are Wedgetail, ARH Tiger Helicopters, Bushmaster Vehicles and Collins RCS.

23 The then Minister for Defence, the Hon. Kevin Andrews MP and the Minister for Finance, Senator the Hon. Mathias Cormann, Air Warfare Destroyer program still fixing serious legacy issues, 22 May 2015.

24 Department of Defence, DMO, Submission No. 1 to the JCPAA, 6 March 2013, Attachment A, p. 5.

25 The project maturity framework outlined in the Department of Defence, Defence Materiel Standard Procedure (Project Management), DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010, is a methodology used to quantify the maturity of projects as they progress through the acquisition life cycle.

26 Refer to the Statement by the Secretary of Defence in Part 3 of this report.

27 Refer to footnote 23 for more detail.

28 See Defence’s Executive Summary in Part 2 of this report.

29 Source 1: Australian Financial Review, Cost fear sets off $8bn warships review, 7 October 2013.

Source 2: M Thomson, Australian Strategic Policy Institute, Indexation, inflation and the cost of defence projects, 25 June 2015, available from <http://www.aspistrategist.org.au/indexation-inflation-and-the-cost-of-defence-projects/> [accessed 10 November 2015].

30 See Table 2.4 in Part 2 of this report.

31 Off-The-Shelf (OTS): Hardware or software that already exists, is in service with one or more other customers for an equivalent purpose and requires no, or minimal, change. Sometimes expressed as COTS (Commercial Off-The-Shelf) or MOTS (Military Off-The-Shelf). Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 8.

32 Prior settlements for projects within this report include Wedgetail, MRH90 Helicopters, ARH Tiger Helicopters and Air to Air Refuel.

33 See the LHD Ships PDSS (Section 1.2 Current Status—Schedule Performance) in Part 3 of this report.

34 See the ARH Tiger Helicopters PDSS (Section 1.2 Current Status—Cost Performance (In-year)) in Part 3 of this report.

35 See the Air to Air Refuel PDSS (Section 1.2 Current Status—Cost Performance (In-year)) in Part 3 of this report.

36 See the Maritime Comms PDSS (Section 2.1 Project Budget (out-turned) and Expenditure History (Note 3)) in Part 3 of this report.

37 See the Battle Comm. Sys. PDSS (Section 1.2 Current Status—Cost Performance (In-year)) in Part 3 of this report.

38 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 13, pp. 15–16, found that while generally the then DMO had improved the effectiveness of the program, there remain opportunities for further improvement and rigour.

39 For further information on Joint Project Directives see ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 11.1 to 11.54, pp. 219–232.

40 Joint Project Directive: A project-specific directive issued by the Secretary of the Department of Defence and the Chief of Defence Force to the nominated Capability Manager or Project Realisation Manager and other involved action addresses, detailing the basis of project approval and assigning overall responsibility, authority and accountability for realisation of the capability system to an in-service state. Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 6.

41 Joint Strike Fighter (Stage 2), P-8A Poseidon, Growler, MH-60R Seahawk, Overlander Medium/Heavy, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Battle Comm. Sys. (Land), Maritime Comms and LHD Landing Craft.

42 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, paragraph 3.116, p. 39.

43 Sir A Tange, Australian Defence, Report on the reorganistion of the Defence group of departments, November 1973.

44 First Principles Review Team, First Principles Review: Creating One Defence, April 2015, p. 13, available from <http://www.defence.gov.au/publications/reviews/firstprinciples/> [accessed 2 September 2015].

45 ibid., p. 18.

46 The Capability Development Group develops the proposals and business cases to seek Government approval on how the Australian Defence Force will be modernised. This is done through the two pass government approval process and in close consultation with the Services and other Defence organisations supporting the fundamental inputs to capability. Upon approval by Government, the funds are transferred to the acquisition agencies, mainly the Capability Acquisition and Sustainment Group, responsible for the delivery of the fundamental inputs to capability. Adapted from <http://www.defence.gov.au/CDG> [accessed 19 November 2015].

47 Commonwealth of Australia, JCPAA, Defence major projects report 2012–13, 20 March 2014, pp. 1–3.

48 ANAO Report No.14 2014–15, 2013–14 Major Projects Report, December 2014, Statement by the CEO DMO, pp. 171–172.

49 Out-turning a project budget takes into account the planned increases in overall Defence spending due to inflationary pressures. JCPAA, Report 429, Review of the 2010–11 Defence Materiel Organisation Major Projects Report, May 2012, Appendix C, p. 46.

50 M Thomson, Australian Strategic Policy Institute, Indexation, inflation and the cost of defence projects, 25 June 2015.

51 Department of Defence, Defence Materiel Manual Project, DMM (PROJ) 1-0-001, DMO Project Management Manual, April 2012, Chapter 7 – Cost Management, paragraph 7.1.5, p. 39.

52 JCPAA, Report 436, Review of the 2011–12 Defence Materiel Organisation Major Projects Report, May 2013, paragraph 3.4, p. 14.

53 The AWD Ships project was also the subject of an ANAO performance audit. See ANAO Report No.22 2013–14, Air Warfare Destroyer Program, March 2014, for further detailed information on this project.

54 Refer to footnote 23 for more detail.

55 Refer to footnote 39 for further information on Joint Project Directives.

56 Department of Defence, Defence Materiel Instruction (Finance), DMI (FIN) 01-0-044, Project Financial Assurance Statements, February 2015.

57 Additionally, the Wedgetail and ARH Tiger Helicopters projects have acknowledged that their 2013–14 contingency statements, which stated that contingency had not been applied during the financial year, were incorrect. The Wedgetail project applied contingency for salaries related to schedule delay and the ARH Tiger Helicopters project applied contingency for the Deployable Aircraft Maintenance Rig capability. See the Wedgetail and ARH Tiger Helicopters PDSSs in Part 3 of this report.

58 The Collins R&S project does not have a formal contingency allocation.

59 Department of Defence, DMM (PROJ) 11-0-002, DMO Project Risk Management Manual 2013, July 2013, Chapter 9 – Management of Contingency Budgets in DMO Acquisition Projects, p. 108, requires that the Project Contingency Budget Log is kept up to date for the proper overall management of risk and that it is submitted for review at Additional and Budget estimates.

60 ANAO Report No.13 2009–10, 2008–09 Major Projects Report, November 2009, Part 1, paragraphs 2.12 to 2.18, pp. 37–39.

61 More recent ANAO observations are in ANAO Report No.9 2015–16, Test and Evaluation of Major Defence Equipment Acquisitions, November 2015, paragraph 5, p. 8.

62 Department of Defence, Audit & Fraud Control Division, Audit Task: 14-013 – Risk Management and the Enterprise Risk Management Framework in Defence, July 2014, Audit Summary.

63 PJ Rizzo, Plan to Reform Support Ship Repair and Management Practices, July 2011, p. 7, available from <http://www.defence.gov.au/publications/Reviews/Rizzo/Review.pdf> [accessed 24 September 2015].

64 ibid.

65 Materiel Sustainment Agreement: An agreement between the Department of Defence and the Capability Acquisition and Sustainment Group (CASG) with the Defence Capability Manager supplying funding and CASG undertaking the sustainment of specific platforms (such as a ship or aircraft fleet), commodities (such as clothing or combat rations) and services (such as provision of maritime target ranges). ANAO Report No.30 2014–15, Materiel Sustainment Agreements, April 2015, paragraph 1, p. 11.

66 For further information on risk management in maritime sustainment, see the ANAO Report No.30 2014–15, Materiel Sustainment Agreements, April 2015, paragraphs 3.67–3.71, pp. 72–73.

67 Refer to footnote 24.

68 Department of Defence, DMM (PROJ) 11-0-002, DMO Project Risk Management Manual 2013, July 2013, Chapter 2 – Managing Risk in DMO Projects – An Overview, p. 23, requires all projects to develop and follow a project Risk Management Plan.

69 For further information on the Independent Project Performance Office and Gate Reviews, refer to paragraphs 3.6 to 3.12 in Part 1 of this report.

70 This will consolidate existing internal risk management policy including the DMO Project Risk Management Manual 2013, July 2013 and the DMI (PROJ) 11-0-005, Project Risk Management, November 2010.

71 Department of Defence, DMM (PROJ) 1-0-001, DMO Project Management Manual 2012, April 2012, Glossary, p. 75.

72 See Appendix 2 in Part 2 of this report and footnote 25 for further detail.

73 Department of Defence, Defence Capability Plan 2012, July 2012, pp. 3–4.

74 JCPAA, Report 442, Inquiry into the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 8, p. 39.

75 Department of Defence, DMM (PROJ) 11-0-002, DMO Project Risk Management Manual 2013, July 2013, p. 119.

76 Department of Defence, DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010, p. 9, with a stated 24 month review period.

77 During fieldwork, the ANAO observed incompatibilities with Project Risk Management Manual version 2.4 across 12 projects, and with DMI (FIN) 01-0-019, Management of Contingency Budgets in DMO Acquisition Projects, September 2014, across four projects.

78 For further information, see paragraphs 1.48 to 1.58 in Part 1 of this report.

79 Refer to footnote 13 for the definition of IMR and FMR milestones, and footnote 14 for the definition of IOC and FOC milestones.

80 A project’s budgeted cost and schedule data is at 30 June 2015, and may differ from originally approved budgets and schedules.

81 As Defence’s assessment of the likelihood of delivering key capabilities involves high levels of uncertainty which may cause actual outcomes to differ materially from that stated in the PDSSs, this data and Defence’s assessment are outside the scope of the ANAO’s review.

82 A project’s budget expended is cash based. In cases where pre-payments have been made, but have not been expensed/amortised, cash paid by a project will be greater than the accrued expenditure.

83 Refer to footnote 23 for more detail.

84 The JCPAA has recommended that the ability to publish project maturity scores be maintained until they are no longer required by the JCPAA. JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 8, p. 39.

85 Analysis provided within Figures 4 and 5 includes scope variations, foreign exchange and indexation for each Major Project however, Table 3 of this report, separates the variations across all Major Projects.

86 See Note 3 of Figure 4, below, for further information.

87 See paragraph 1.14 in Part 2 of this report.

88 Australian Government arrangements for foreign exchange variation involve ‘no win/no loss’ supplementation. As a matter of policy, unless specifically approved, individual agencies are not permitted to ‘hedge’ against foreign exchange risk.

89 Price variations were factored into budgets following the introduction of out-turning in 2010–11.

90 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendations 1 and 2, p. 28.

91 See the Secretary of Defence Foreword in Part 2 of this report.

92 Australian Financial Review, Cost fear sets off $8bn warships review, 7 October 2013.

93 Refer to footnote 84 for more detail.

94 M Kinnaird, Defence Procurement Review 2003, August 2003.

95 G Pappas, Department of Defence, 2008 Audit of the Defence Budget, April 2009, p. 76.

96 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, pp. 29–31.

97 In instances where a Major Project has multiple segments/capabilities with separate Final Operational Capability (FOC) dates, the ANAO has used the project’s current lead/main capability FOC for calculating schedule performance. Defence’s approach is to use the final FOC date for a project listed in the 2014–15 PDSSs. These approaches, both valid, led to a small difference in the calculated percentage by which the Major Projects’ total schedule has slipped for the 2014–15 MPR (ANAO—28 per cent; Defence—27.8 per cent).

98 See the PDSSs in Part 3 of this report.

99 See the Collins RCS, Hw Torpedo and Collins R&S PDSSs in Part 3 of this report.

100 ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 9.1 to 9.4, pp. 198–199.

101 Further information on MRH90 Helicopters can be found in ANAO Reports No.48 2008–09, Planning and Approval of Defence Major Capital Equipment Projects, June 2009, pp. 84, 90 and 133; No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, pp. 86–87 and pp. 130–133; and No.52 2013–14, Multi-Role Helicopter Program, June 2014.

102 See the ARH Tiger Helicopters PDSS in Part 3 of this report.

103 Further, in the Statement by the Secretary of Defence in Part 3 of this report, the Secretary makes reference to additional information on milestone achievement dates for MH-60R Seahawk, LHD Ships, ANZAC ASMD 2B, ANZAC ASMD 2A, Battle Comm. Sys. and LHD Landing Craft.

104 Air Warfare Destroyer Alliance, AWD Reform – Cost Review Outcomes, April 2015.

105 Source 1: Refer to footnote 23 for more detail.

Source 2: See the AWD Ships PDSS in Part 3 of this report.

106 M Kinnaird, Defence Procurement Review 2003, August 2003.

107 Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, p. B1.

108 Source 1: Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Chapter 2, pp. 1–2.

Source 2: Department of Defence, DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, paragraph 1, p. 1.

109 A number of MRMs and Completion Criteria at 30 June 2015 include milestones relating to schedule, which are not strictly capability measures within the Defence framework.

110 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, pp. 37–39.

111 The then CEO DMO Mr Warren King advised the JCPAA that ‘Landing on a method to have an easily auditable statement of what the capability is that we have delivered is really a complex issue and still there is, I think, work to be done’. Commonwealth of Australia, JCPAA, Defence major projects report 2012–13, 20 March 2014, pp. 1–3.

112 Refer to footnote 84 for more detail.

113 Expected Capability assessments concern forecasting future achievements and are outside the scope of the ANAO’s review.

114 Refer to footnote 113 for more detail.

115 The then Minister for Defence, the Hon. Kevin Andrews MP, Minister for Defence – First Principles Review of Defence, 1 April 2015.

116 Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Chapter 2, pp. 1–2.

117 For further information on the First Principles Review refer to paragraphs 1.4 to 1.6 in Part 1 of this report.

118 Refer to footnote 40 for the definition of a Joint Project Directive.

119 JCPAA, Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, paragraph 3.116, p. 39.

120 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 13, pp. 15–16, found that while generally the then DMO improved the effectiveness of the program, there remain opportunities for further improvement and rigour.

121 The term ‘gate’ refers to a major project decision point or milestone. ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 1.8, p. 29.

122 Each Board includes a Chair (independent to the project under review and with the management level dependent on the complexity of the project, its profile and sensitivity), a number of project managers selected for their expertise, and up to two external members with extensive Defence or commercial experience. To help inform the Board, an independent preliminary analysis is conducted prior to the Board meeting to identify key issues to bring to the Board’s attention. Department of Defence, Defence Materiel Instruction (Executive), DMI (EXEC) 00-0-009, Gate Reviews for DMO Projects and Sustainment Products, August 2014, paragraphs 37 to 41, p. 7.

123 ibid., paragraphs 1 to 4, pp. 2–3.

124 Source 1: D Mortimer, Going to the Next Level: the report of the Defence Procurement and Sustainment Review, September 2008, pp. 35–36.

Source 2: ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 5, p. 14.

125 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Independent Project Performance Office to oversee major Defence projects established, 29 June 2011.

126 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Strategic Reform Program, 6 May 2011.

127 In certain circumstances, the Director General Independent Project Performance may approve that a mandatory gate not be applied.

128 The decision points and their mandatory status have changed over time. These are detailed in DMI (EXEC) 00-0-009, Gate Reviews for DMO Projects and Sustainment Products, August 2014, Annex A, pp. 10–14.

129 ibid., paragraphs 52 to 55, p. 9.

130 The first standalone sustainment Performance Gate Review will normally be held 12 months after the FMR Gate Review. Sustainment products will then be reviewed at one to three year intervals, depending on its level of complexity and associated risks or issues, and in line with major events such as re-contracting or mid-life upgrades. Department of Defence, DMI (EXEC) 00-0-009, Gate Reviews for DMO Projects and Sustainment Products, August 2014, paragraphs 30 to 34, p. 6.

131 This includes MH-60R Seahawk (where the Acquisition and Sustainment review was held jointly), ARH Tiger Helicopters and Bushmaster Vehicles (which also underwent a separate Acquisition Gate Review in 2015).

132 ANAO Report No.14 2014–15, 2013–14 Major Projects Report, December 2014, paragraph 3.12, p. 87.

133 ANAO Report No.30 2014–15, Materiel Sustainment Agreements, April 2015, paragraphs 3.24 to 3.25, pp. 58–59.

134 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 4.26, p. 94.

135 Department of Defence, Gate Review Survey 2015, 2015, Executive Summary, pp. 5–8.

136 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 4.72, p. 105 and paragraphs 4.23 to 4.28, pp. 93–95.

137 This includes the Joint Strike Fighter, AWD Ships, P-8A Poseidon (two Gate Reviews), Wedgetail, MRH90 Helicopters, Growler, MH-60R Seahawk, Air to Air Refuel, Battlefield Airlifter, Bushmaster Vehicles, Additional Chinook, Battle Comm. Sys. (Land), Maritime Comms, UHF SATCOM, Battle Comm. Sys. and LHD Landing Craft projects. Eight projects have Gate Reviews scheduled for later this year. The MH-60R Seahawk, ARH Tiger Helicopters and Bushmaster Vehicles projects were also subject to Sustainment Gate Reviews in 2014–15. Five projects (ANZAC ASMD 2B, Collins RCS, Hw Torpedo, Collins R&S, ANZAC ASMD 2A) are not expected to undergo Gate Reviews in 2015.

138 First Principles Review Team, First Principles Review: Creating One Defence, April 2015, p. 38.

139 This advice was set out in a brief from the then Acting CEO DMO to the then Minister for Defence, Projects of Concern Overview and Summit Arrangements, 3 February 2015.

140 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Reforms to Projects of Concern, 29 June 2011.

141 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Defence Capability Reform, 16 October 2012.

142 Defence advised that this status is not yet expected to change, with the First Principles Review outcomes anticipated to impact departmental documentation processes.

143 Department of Defence, Projects of Concern list update, 6 March 2015.

144 The then Minister for Defence, Senator the Hon. David Johnston, Air Warfare Destroyer added to Projects of Concern list, 4 June 2014.

145 The Minister for Finance, Senator the Hon. Mathias Cormann, and the then Minister for Defence, Senator the Hon. David Johnston, Coalition committed to the efficient delivery of the Air Warfare Destroyer programme, 18 December 2013.

146 The then Minister for Defence, Senator the Hon. David Johnston, Air Warfare Destroyer added to Projects of Concern list, 4 June 2014.

147 For further information on the AWD Ships project see ANAO Report No.22 2013–14, Air Warfare Destroyer Program, March 2014.

148 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Dr Mike Kelly AM MP, MRH90 helicopter Project of Concern progress, 9 May 2013.

149 See Table 1.1 in Part 2 of this report.

150 For further information on the MRH90 Helicopter project see ANAO Report No.52 2013–14, Multi-Role Helicopter Program, June 2014.

151 Australian Strategic Policy Institute, Agenda for change – Strategic choices for the next government, August 2013, p. 24.

152 Source 1: The then Minister for Defence Materiel, the Hon. Dr Mike Kelly AM MP, Minister for Defence Materiel chairs Projects of Concern Summit, 10 May 2013.

Source 2: Department of Defence, Projects of Concern list update, 6 March 2015.

153 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Reforms to Projects of Concern, 29 June 2011.

154 This advice was set out in a minute from the Head Capability Systems, AIR87PH2 ARH – CDG Endorsement of FMR Approval, 17 February 2014, with attached Army caveats and requirements in relation to the approval of FMR from Head Modernisation and Strategic Planning - Army.

155 The then Minister for Defence, the Hon. Stephen Smith MP, and the then Minister for Defence Materiel, the Hon. Jason Clare MP, Strategic Reform Program, 6 May 2011.

156 This advice was set out in a brief from the then CEO DMO to the Secretary of the Department of Defence and the then Chief of the Defence Force, Brief for Secretary/CDF: Early Indicators and Warnings (EI&W) Report – September 2013, 20 December 2013, p. 1.

157 Department of Defence, DMO Quarterly Project Performance Report for April–June 2015, August 2015, p. 2.

158 Source 1: Refer to footnote 40 for the definition of a Joint Project Directive.

Source 2: For further information on Joint Project Directives see ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 11.1 to 11.54, pp. 219–232.

159 Materiel Acquisition Agreement: An agreement that states in concise terms what services and products the Capability Acquisitions and Sustainment Group (as supplier) will deliver, for how much and when, in support of unapproved and approved Major and Minor Capital Equipment projects. Department of Defence, Defence Capability Development Manual, July 2015, Part 1, Glossary, p. 7.

160 The key role of these two documents in Defence’s current acquisition arrangements has been set out in the Senate Foreign Affairs, Defence and Trade References Committee, Procurement procedures for Defence capital projects, Final Report, August 2012, pp. 105–108.

161 Joint Strike Fighter (Stage 2), P-8A Poseidon, Growler, MH-60R Seahawk, Overlander Medium/Heavy, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Battle Comm. Sys. (Land), Maritime Comms and LHD Landing Craft.

162 JCPAA, Report 436, Review of the 2011–12 Defence Materiel Organisation Major Projects Report, May 2013, Recommendation 4, p. 26.

163 The significant increase in identified applications is due to the implementation of the One Defence model under the First Principles Review. As the functions of the DMO have been merged back into the Department of Defence, the identified applications are across the department, not just the former DMO.

164 ANAO Report No.12 2013–14, 2012–13 Major Projects Report, December 2013, Part 2, paragraph 1.34, p. 111.

165 Department of Defence, Introducing the Defence Materiel Organisation, June 2014, p. 7.

166 First Principles Review Team, First Principles Review: Creating One Defence, April 2015, pp. 50–51.

167 ibid., p. 56.

168 ibid., p. 57.

169 The ANAO notes that average APS full time equivalent staffing numbers for the Capability Acquisition and Sustainment Group decreased from 5 389 at 30 June 2013, to 4 812 at 30 June 2014 and to 4 075 at 30 June 2015.

Source 1: Department of Defence, Defence Annual Report 2013–14, October 2014, Table 8.7, p. 132.

Source 2: Department of Defence, Defence Annual Report 2014–15, October 2015, Table 8.7, p. 131.

170 Department of Defence, Skilling Australia’s Defence Industry (SADI) Programme, 2015, available from <http://www.defence.gov.au/dmo/DoingBusiness/Industry/SkillingDefenceIndustry/SkillingAustralianDefenceIndustry/Default.aspx> [accessed 2 September 2015].

171 ibid.

172 ibid.

173 Department of Industry, Innovation and Science, Skilling Australia’s Defence Industry (SADI), 2015, available from <http://www.business.gov.au/grants-and-assistance/DefenceIndustry/SADI/Pages/default.aspx> [accessed 2 September 2015].

174 The then Minister for Defence, the Hon. Kevin Andrews MP, Funding to skill Australia’s defence industry, 27 July 2015.

175 Department of Defence, Skilling Australia’s Defence Industry Programme Guidelines 2015–16, July 2015, pp. 11–12.

176 Department of Defence, Industry Skilling Program Enhancement package (ISPE), 2015, available from <http://www.defence.gov.au/dmo/DoingBusiness/Industry/SkillingDefenceIndustry/ IndustrySkillingProgramEnhancement/> [accessed 2 September 2015].

177 ibid.

178 The then Minister for Defence Materiel, the Hon. Jason Clare MP, More than $12 million investment in priority industry capabilities, 30 April 2012.

179 Department of Defence, Priority Industry Capability Innovation program (PICIP), 2015, available from <http://www.defence.gov.au/dmo/DoingBusiness/Industry/IndustryPrograms/PriorityIndustryCapabilityInnovationProgram/> [accessed 2 September 2015].

180 The then Minister for Defence Materiel, the Hon. Jason Clare MP, Defence skills plan to meet the challenges ahead, 5 September 2011.

181 Skills Australia, Building Australia’s Defence Supply Capabilities – Main Report for the Defence Industry Workforce Strategy, September 2012, p. 23.

182 ibid., p. 24.

183 Australian Government, Productivity Commission, Trade & Assistance Review 2012–13, June 2014, p. 35.

184 ANAO, Audit Work Program, July 2015, p. 42.

185 On 1 July 2015, the Defence Materiel Organisation (DMO) transitioned to the newly established Capability Acquisition and Sustainment Group (CASG) within the Department of Defence as recommended by the Defence First Principles Review. CASG continues to have the former DMO’s objective to provide the materiel equipment and sustainment elements of capability for the ADF in an effective, efficient, economical and safe manner.

186 On 1 July 2015, the Defence Materiel Organisation (DMO) transitioned to the newly established Capability Acquisition and Sustainment Group (CASG) within the Department of Defence as recommended by the Defence First Principles Review. CASG continues to have the former DMO’s objective to provide the materiel equipment and sustainment elements of capability for the ADF in an effective, efficient, economical and safe manner.

187 Department of Defence, Defence Annual Report 2014-15, Volume One, Appropriations and other resources, p 94.

188 Where possible, ‘Defence’ is used for consistency throughout the MPR, ‘DMO’ is still retained where the context is DMO specific.

189 Some of these projects are early in the acquisition phase.

190 When reference is made to Defence capability it shall mean materiel systems only.

191 The then Minister for Defence, Senator the Hon. David Johnston, Australia’s first F-35A takes to the skies, media release, 1 October 2014.

192 The then Minister for Defence, Senator the Hon. David Johnston, ‘Rollout’ of first two Australian F-35A Joint Strike Fighters in the USA, media release, 25 July 2014.

193 First Air Warfare Destroyer launched at ASC, Osborne, Adelaide Advertiser, Adelaide, 23 May 2015, http://www.adelaidenow.com.au/news/south-australia/first-air-warfare-destroyer-launched-at-asc-osborne/story-fni6uo1m-1227366174513.

194 The then Minister for Defence, the Hon. Kevin Andrews MP, Launch of the first Air Warfare Destroyer, media release, 26 May 2015.

195 McPhedran, Ian, No end in sight to terror war, Sunday Tasmanian, Hobart, 7 June 2015, p 23.

196 The then Minister for Defence, the Hon. Kevin Andrews MP, New flight simulator for Navy helicopter crews, media release, 27 March 2015.

197 HMAS Canberra enters into commission into Australian Navy at Sydney ceremony, ABC NEWS, 28 November 2014, www.abc.net.au/news/2014-11-28/hmas-canberra-commissioning-ceremony-in-sydney/5924340.

198 McPhedran, Ian, No end in sight to terror war, Sunday Tasmanian, Hobart, 7 June 2015, p 23.

199 Boom milestone for our Air Force, Queensland Times, Ipswich QLD, 10 June 2014, p 8.

200 This is a sustainment product but is managed as part of the Projects of Concern list.

201 The five projects are: P-8A Poseidon, MRH90 Helicopters, ANZAC ASMD 2A, ANZAC ASMD 2B; and, Maritime Comms.

202 The convention used in this report is to list projects in order of their total approved budget to deliver the project, from highest to lowest. Where the analysis requires a different order, an explanation is provided.

203 New Project for 2014-15 MPR.

204 ibid.

205 The Two Pass Government Approval process was mandated following the 2003 Kinnaird review.

206 Approval granted after project scope and budget were approved for transition to the in-service sustainment support system in 2010-11.

207 Approval granted in 2014 based on a risk assessment performed by DMO and endorsed by the Capability Manager, which concluded the overall risk rating for remaining work was low.

208 ibid.

209 ibid.

210 Where FMR was not included in the original project approval documentation, originally estimated FMR is taken from the latest version of the project’s Materiel Acquisition Agreement.

211 A schedule variance factor of 1 = on time; >1 = late; and <1 = early.

212 ‘New’ - a capability that has not previously existed in the ADF; ‘Replacement’ – a current capability that is being replaced by more up to date technology or to respond to a changing threat; ‘Upgrade’ – an upgrade to existing capabilities.

213 Defence’s categorisation of projects that represent the complexity of the project on a sliding scale of I to IV, with ACAT I representing the most complex projects.

214 Provides an indication of whether the projects were initially developed under pre or post-Kinnaird reforms.

215 While FMR is generally expected to occur ahead of FOC, there will be instances where FMR is due to occur after FOC. For example the Collins R&S project anticipates the Capability Manager declaring FOC ahead of FMR having being realised at the completion of a docking cycle one month later. It is likely that this may happen for other new platforms or multi-platform upgrade projects where finalisation of some aspects of materiel acquisitions (e.g. spares or future warranty resolution) may extend beyond FOC.

216 Minister for Defence, Minister for Defence Materiel and Minister for Finance and Deregulation - Joint Media Release – Air Warfare Destroyer update, 6 September 2012.

217 Where multiple elements of a mission system are involved (e.g. three surface combatants) this date represents Initial Operational Capability (IOC) of the initial Subset, including its associated operational support, i.e. when the IOC is achieved.