Audit snapshot

Why did we do this audit?

  • Comprising four discrete projects, and with an estimated combined program cost of $14–$20 billion for acquisition alone, Defence has described Land 400 as the biggest and most expensive acquisition program in the Australian Army’s history.
  • Land 400 Phase 2 was expected to address the risk of a capability gap emerging by 2020 in the Army’s ability to conduct its core business — sustained close combat.
  • This audit has been undertaken to provide the Parliament with independent assurance of Defence’s acquisition to date of combat reconnaissance vehicles (CRVs) under Land 400 Phase 2.

Key facts

  • In August 2018, Defence signed a contract, valued at $4.28 billion (including GST), with Rheinmetall Defence Australia.
  • At second pass government approval in March 2018, Initial Operating Capability was scheduled for December 2022 and Final Operating Capability was scheduled for June 2027.

What did we find?

  • Defence’s acquisition to date of CRVs under project Land 400 Phase 2 is largely effective, and Defence has the potential to ultimately achieve value for money outcomes if the identified risks and issues are managed appropriately and the contracted quality and quantity of goods and services are delivered according to the agreed schedule and cost.
  • Defence’s request for tender process was partly effective.
  • Defence conducted a largely effective tender evaluation.
  • Defence has established largely effective project governance and contracting arrangements.

What did we recommend?

  • The Auditor-General recommended that Defence review the process for providing assurance to its senior leadership that agreed internal review recommendations have been implemented in a timely manner.
  • Defence agreed to the recommendation.

$15.7 bn

Gross life cycle cost of Land 400 Phase 2 — $5.9 bn acquisition cost and $9.8 bn operating costs over 30 year life of vehicle fleet.

211

‘Boxer’ CRVs, 12 mission modules and associated support systems to be acquired under the contract.

$1.68 bn

Total expected Australian industry capability expenditure.

Summary and recommendations

Background

1. Comprising four discrete projects, and with an estimated combined program cost of between $14 billion and $20 billion for acquisition alone, Defence has described Land 400 as ‘the biggest and most expensive acquisition in Army’s history’.1 Through Land 400 Phase 2, Defence aims to provide ‘armoured survivability, mobility and lethality’ to the Australian Army’s (Army) mounted combat reconnaissance capability, by replacing and enhancing ‘those mounted combat reconnaissance and counter-reconnaissance capabilities currently enabled by the ASLAV [Army’s Australian Light Armoured Vehicle]’.2

2. On 9 August 2018, Defence established a contract, valued at $4.28 billion (including GST), with Rheinmetall Defence Australia (Rheinmetall) for the acquisition of 211 Boxer Combat Reconnaissance Vehicles (CRVs), 12 mission modules and associated support systems. The vehicles are to be delivered in two blocks (tranches):

  • Block I — comprising 25 vehicles intended to provide an early deployable capability for the Australian Defence Force (ADF) by December 20203; and
  • Block II — comprising 186 vehicles to be delivered between February 2022 and January 2027.

Rationale for undertaking the audit

3. This audit has been undertaken to provide the Parliament with independent assurance of the effectiveness of Defence’s acquisition to date of combat reconnaissance vehicles (CRVs) under Land 400 Phase 2.

4. Defence has described the Land 400 program as the biggest and most expensive acquisition in the Australian Army’s history. Land 400 Phase 2 was intended to enhance Army’s ability to conduct its core business — sustained close combat — and address the risk of a capability gap emerging by 2020, with Army publicly stating that a life-of-type extension of the ASLAVs was not possible due to obsolescence factors.

5. During the course of the tender process for Land 400 Phase 2, the Australian Government’s industry policy changed to focus on building Australian industrial capability to better support Defence. Previous policy focused on reducing risks through the consideration of military off the shelf (MOTS) equipment4, following reviews such as Going to the Next Level, The report of the Defence Procurement and Sustainment Review, 2008 (the ‘Mortimer Review’).

Audit objective and criteria

6. The objective of the audit was to assess the effectiveness and value for money to date of Defence’s acquisition of combat reconnaissance vehicles under project Land 400 Phase 2.

7. To form a conclusion against the audit objective, the following high-level criteria were adopted:

  • Did Defence conduct an effective tender process that contributed to the achievement of value for money?
  • Did Defence conduct an effective evaluation process that contributed to the achievement of value for money?
  • Did Defence establish effective project governance and contracting arrangements that contribute to the achievement of value for money?

8. This audit focuses on the procurement of the 211 CRVs under Land 400 Phase 2. The audit scope did not include assessments of:

  • the sustainment arrangements for the vehicles or the retirement of the existing ASLAV capability from service;
  • the development of the Military Vehicle Centre of Excellence (MILVEHCOE), the purpose built vehicle production facility in Queensland; or
  • Land 400 phases 3 and 4.5

Conclusion

9. Defence’s acquisition to date of combat reconnaissance vehicles under project Land 400 Phase 2 is largely effective, and Defence has the potential to ultimately achieve value for money outcomes if the identified risks and issues are managed appropriately and the contracted quality and quantity of goods and services are delivered according to the agreed schedule and cost. While Defence’s tender process was competitive and resulted in a procurement of the desired capability, the outcome was the result of a higher risk procurement approach (MOTS Plus) than had been approved at first pass (MOTS-based).

10. Defence’s request for tender process was partly effective. Effectiveness was reduced because Defence did not operate within the bounds of the first pass government approval and did not advise the Minister that it would issue a request for tender for both MOTS and ‘MOTS Plus’ options rather than the government-approved ‘MOTS-based’ approach. Defence did not define the meaning of ‘MOTS-based’ in its advice to the Minister, to inform the government’s decision-making at first pass — one of a number of shortcomings in Defence advice to ministers. Further, three Defence reviews identified that its approach to market, including the cost of tendering and schedule constraints, might have caused some potential bidders to not participate in the tender — potentially reducing competition and limiting the options available to the Australian Government.

11. Defence conducted a largely effective tender evaluation. The process was conducted effectively with the preferred tenderer, Rheinmetall, selected on the basis that its offer provided a higher level of assurance that it could meet Army’s capability and protection requirements in the scheduled timeframe, albeit at a higher cost than the alternative BAE Systems offer. Defence assessed value for money at each stage of the tender evaluation process and documented its assessments. The overall effectiveness of the process was undermined by deficiencies in Defence’s implementation of its risk mitigation activity and shortcomings in its management of conflicts of interest and probity requirements.

12. While Defence has established largely effective project governance and contracting arrangements, required project governance documentation has not been maintained. Defence has been establishing governance structures to help integrate activity within Defence and build its relationship with the prime contractor, but has been less effective in settling project-level governance arrangements and maintaining project documentation, including an up to date project risk register. There is evidence of high level and active management of known risks by the parties, including risks to schedule.

Supporting findings

The tender process

13. Defence was effective in defining its requirements. Through its reviews and industry engagement, Defence identified that there were MOTS options available that would meet its requirements for replacing its ASLAV fleet through Land 400 Phase 2. Defence also identified that there would be a need for developmental work to integrate government furnished equipment. A Defence Gate Review in June 2013 reported that the requirements for the project had been adequately defined but the schedule was high risk and that the project was unaffordable.

14. Defence did not always provide accurate advice regarding Land 400 Phase 2. Defence’s October 2014 Gate Review identified that advice provided in the proposed first pass submission to government was not accurate in its representation of the capability being acquired as MOTS. While available evidence indicates that Defence provided additional context and presented the acquisition as ‘MOTS-based’, it did not define the meaning of ‘MOTS-based’ in its advice to the Minister to inform ministerial and governmental decision-making.

15. Defence’s request for tender process was partly effective. Despite first pass government approval being for a ‘MOTS-based’ solution, Defence incorporated both a MOTS and a ‘MOTS Plus’ option into the tender process and did not bring this change to the Minister’s attention. Eight tenders (four MOTS and four MOTS Plus) were submitted as part of a competitive tender process, but two subsequent Defence reviews and an internal audit identified that Defence’s approach to the market, and the cost of tendering and schedule constraints, might have caused some potential bidders to not participate in the tender — potentially reducing competition and limiting the options available to the Australian Government.

Tender evaluation

16. Defence developed a tender evaluation plan and established a fit for purpose tender evaluation organisation.

17. Defence conducted a largely effective initial tender evaluation process. Tenders were assessed against the ten evaluation criteria documented in the approved Tender Evaluation Plan and value for money was considered in assessing which tenders proceeded to the planned risk mitigation activity. While the probity advisor undertook a probity review of the initial tender evaluation process, the assessment had a limited sampling approach and no record of the final probity report was retained. The findings from the review noted that a separate probity register was not maintained for the project and there were inconsistencies in the treatment of risk in the tender evaluation working group reports. Defence was unable to demonstrate that the recommendations from the 2016 review of stage one of the tender process had been implemented.

18. Defence’s risk mitigation activity was partly effective. While Defence’s decision to conduct a risk mitigation activity as part of the tender process was commensurate with the risk and significance of the procurement, there were issues in the activity’s implementation. These issues included: the vehicles tested did not match the tendered vehicle; minimal evidence was obtained regarding the performance of mandated Australian Government furnished equipment in the vehicles; and claims made by tenderers that were not adequately tested during the risk mitigation activity. Two Defence reviews on its implementation reported mixed findings.

19. Defence conducted an effective final selection process to identify a preferred tenderer (Rheinmetall) for the supply of mounted combat reconnaissance vehicles for Land 400 Phase 2. Application of the process set out in the Tender Evaluation Plan did not result in an outcome and further steps — involving parallel negotiations and additional criteria — were undertaken to identify a successful tender. Vehicle occupant protection was the primary technical and capability discriminator and heavily informed Defence’s value for money assessment. Defence sought advice from the Australian Government Solicitor on conducting parallel negotiations, and developed internal protocols for conducting these additional steps in the selection process.

20. The advice provided to government to inform its March 2018 second pass approval reflected the outcome of the evaluation process and appropriately informed government of judgements made to select the recommended tenderer. The advice to government regarding technical, schedule and cost risks was not, on its face, fully aligned with the documentation of risk in Defence’s January 2018 Post Negotiation Evaluation Report. Defence advised the ANAO that the advice to government on these risks took into account decisions made by Defence to mitigate risks identified in the Post Negotiation Evaluation Report.

21. Defence’s management of probity during the tender process was partially effective. While Defence had a plan, policies and procedures in place to manage probity, and sought advice from a probity adviser throughout the tender and evaluation process, there were gaps in relevant documentation and shortcomings in the project’s management of conflict of interest and probity against requirements.

Project governance and contracting arrangements

22. Defence has been partially effective in maintaining the required project governance documentation. In particular, there is a misalignment between the initial operational capability (IOC) date documented in Defence’s 2019 Material Acquisition Agreement and the IOC date agreed by government at second pass. These dates are currently out of alignment by four and a half years. Defence advised the ANAO that it considers the achievement of Interim IOC by December 2022 is consistent with Government approval at second pass for IOC by 2021–22.

23. While Defence has established a fit for purpose governance framework to support its monitoring, management and reporting on contractor and project performance, implementation of the governance framework has been inconsistent. There have been delays in establishing project-level governance arrangements and the project risk register has not been kept up to date. Defence’s management in these respects has not been fully effective for a project that is developmental in nature and is to replace a capability that is expected to reach its life of type in 2020. Further, key executive reporting mechanisms for the project do not accurately reflect the project status including the significant technical and schedule challenges identified through the project’s monitoring mechanisms, which impacts the timely identification of emerging risks by Defence senior stakeholders. There is, however, evidence of high level and active management of the relationship between Defence and the prime contractor, Rheinmetall.

Recommendation

Recommendation no.1

Paragraph 2.30

Defence review the process in place to provide assurance to its senior leadership that agreed independent assurance review recommendations have been implemented appropriately and in a timely manner.

Department of Defence response: Agreed.

Summary of Department of Defence’s response

24. Defence’s summary response is provided below. The department’s full response can be found at Appendix 1.

Defence welcomes the ANAO Audit Report into Defence’s procurement of combat reconnaissance vehicles (Land 400 Phase 2) and agrees with the recommendation.

The Rheinmetall Boxer vehicles being acquired to replace Army’s aging Australian Light Armoured Vehicle (ASLA V) fleet will, from the middle of 2022, provide Government with the option to deploy Australian soldiers to military operations in low, medium and high threat environments with the highest levels of protection currently available for an 8x8 vehicle.

Defence acknowledges the findings in relation to the tender process, and that schedule constraints, coupled with the costs of participating in the 12-month risk mitigation activity, resulted in at least two companies declining to bid. These are important lessons that have already been applied to other major acquisition programs, including Land 400 Phase 3.

Defence would like to note that the tender process resulted in consideration of 12 of the 14 Combat Reconnaissance Vehicle (CRV) contender vehicle platforms known to exist during the solicitation activity. This level of consideration and competition supports a value for money outcome. Further, an independent review of the tender process undertaken in 2018 found that CASG [Capability Acquisition and Sustainment Group] had completed the market solicitation, source selection and contract negotiation process to achieve a shortened schedule, and delivered a value for money outcome for Army.6

Through the Land 400 Phase 2 procurement process, Defence has ensured that a sovereign Australian capability is established to maintain a lethal, relevant and effective land combat vehicle capability into the future. Over the 30 years that the CRV is expected to be in service, this approach will direct more than $10 billion toward local defence industry, employ significant numbers of Australian workers and support potential export opportunities.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Procurement

Governance and risk management

Probity management

1. Background

Introduction

1.1 Comprising four discrete projects, and with an estimated combined program cost of between $14 billion and $20 billion for acquisition alone, the Department of Defence (Defence) has described the Land 400 program as ‘the biggest and most expensive acquisition in Army’s history’.7

1.2 Through its implementation of the Land 400 program, Defence also seeks to deliver a sovereign industrial capability priority identified in the Defence Industrial Capability Plan, relating to ‘Land combat vehicle and technology upgrade’.8 In regards to this priority, the plan states that:

Broad Australian industry involvement in the delivery of the new Combat Reconnaissance Vehicle program for example will ensure Australia develops the sovereign capability to maintain a lethal, relevant and effective capability into the future.9

1.3 The Australian Army (Army) has identified Land 400 as a high priority and necessary to address obsolescence in its armoured fleets and a capability gap. In 2016, Army’s website included the following statement:

Army’s core business is the conduct of sustained close combat. Only Army conducts this task in the ADF on behalf of Government. LAND 400 delivers part of that capability and is replacing some of the capability we already have — ASLAV and M113AS4 — in accordance with good military practice.

LAND 400 will allow a staged retirement of the in-service Australian Light Armoured Vehicle (ASLAV) and the M113AS4 Armoured Personnel Carrier fleets in line with their technical Life of Type and reducing tactical utility in the contemporary operational environment that involves increasing levels of lethality and complexity.

The highest priority for Army is to replace the ASLAV fleet with a CRV [combat reconnaissance vehicle] due to obsolescence factors that constrain tactical employment and increase the cost of ownership. These obsolescence factors cannot be mitigated through upgrade and without replacement starting in 2020, a capability gap will result.10

1.4 Land 400 comprises four phases, summarised in Table 1.1. Land 400 Phase 2, highlighted in grey, is the focus of this audit.

Table 1.1: Land 400 Phases

Land 400 Phase

Description

Phase 1 — Project Definition Study

  • Defence refers to early scoping work undertaken prior to 2009 as Phase 1.

Phase 2 — Mounted Combat Reconnaissance Capability

  • Acquisition of 211 Rheinmetall ‘Boxer’ Combat Reconnaissance Vehicles (CRVs) at a contracted cost of $4.28 billion (including GST) to replace Army’s ASLAV fleet.a

Phase 3 — Mounted Close Combat Capability

  • Acquisition of up to 450 Infantry Fighting Vehicles to replace Army’s fleet of M113 Armoured Personnel Carriers and 17 Manoeuvre Support Vehicles.

Phase 4 — Integrated Training System

  • Acquisition of the training systems necessary to enable training with the new mounted close combat capabilities. As at August 2019, Defence advised the ANAO that the indicative budget for this phase was $954.3 million.
   

Note a: Defence introduced the Australian Light Armoured Vehicle (ASLAV) into service in the mid 1990s. This acquisition was examined in Auditor-General Report No.9 2006–07 Management of the Acquisition of the Australian Light Armoured Vehicle Capability, available from https://www.anao.gov.au/work/performance-audit/management-acquisition-australian-light-armoured-vehicle-capability.

Source: Defence documentation.

1.5 Through Land 400 Phase 2, Defence aims to provide ‘armoured survivability, mobility and lethality’ to Army’s mounted combat reconnaissance capability, by replacing and enhancing ‘those mounted combat reconnaissance and counter-reconnaissance capabilities currently enabled by the ASLAV’.11 Figure 1.1 is an image of the base ‘Military Off The Shelf Plus’ configuration vehicle being procured by Defence.12

Figure 1.1: Rheinmetall Boxer Combat Reconnaissance Vehicle

Image of a Rheinmetall Boxer Combat Reconnaissance Vehicle.

Source: Department of Defence, https://www.defence.gov.au/CASG/EquippingDefence/Land%20400.asp [accessed 12 March 2020].

1.6 The Armoured Vehicle Division within Defence’s Capability Acquisition and Sustainment Group (CASG) is responsible for managing Project Land 400 on behalf of Army (the capability owner).13 A Materiel Acquisition Agreement has been signed which sets out Army’s requirements (in terms of capability, performance and budget) and CASG’s agreement to deliver those requirements.14

Land 400 Phase 2 — mounted combat reconnaissance capability

1.7 The schedule set out in the Defence Capability Plan 2004–2014 stated that in-service delivery of the ASLAV replacement would be 2015 to 2017. The Australian Government’s commitment to replace Army’s protected vehicles was also noted in the 2009, 2013 and 2016 Defence white papers.15

First pass government approval

1.8 On 9 December 2014, the government gave first pass approval16 to replace Defence’s ASLAVs with ‘military-off-the-shelf based options’, including the acquisition of around 225 CRVs and capability development funding to conduct risk reduction activities prior to the second pass approval stage.17 The government also agreed that the Minister for Defence would provide options differentiated by capability levels and price points to government at second pass.

Approach to market and tender evaluation process

1.9 On 19 February 2015, Defence released an open Request for Tender to procure combat reconnaissance vehicles under Land 400 Phase 2. In response, four tenderers each submitted a military off the shelf (MOTS) bid and a ‘MOTS Plus’ bid (eight tenders in total). Defence identified the successful tenderer through a three stage process outlined in Table 1.2 below.18

Table 1.2: Defence’s three stage tender evaluation for Land 400 Phase 2

Stage

Duration

Description

1

September 2015 to July 2016

Initial evaluation and shortlisting of tenders

Defence assessed the eight tenders received and shortlisted two potential suppliers — Rheinmetall Landsysteme Gmbh (Rheinmetall)a and BAE Systems Australia (BAE Systems) — to participate in the second stage of the tender evaluation.

2

August 2016 to August 2017

Risk mitigation activity with invited shortlisted tenderers

Defence conducted a 12-month risk mitigation activity with the two shortlisted tenderers, aimed at reducing the risks to Defence and maximising Australian industry participation.

3

September 2017 to January 2018

Final evaluation and selection

Defence assessed the results from its risk mitigation activity, evaluated the revised tenders from the two participants, undertook parallel negotiations with both tenderers and conducted an additional evaluation step to identify Rheinmetall as the successful tenderer.

     

Note a: During contract negotiations between Defence and Rheinmetall in June 2018, Defence agreed to Rheinmetall’s proposal to change the contracting entity from the entity that was the successful tenderer (Rheinmetall LandSysteme GmbH Germany) to Rheinmetall Defence Australia.

Source: ANAO analysis of Defence documents.

Second pass government approval

1.10 To inform second pass approval, the government was provided with advice and a recommendation that Rheinmetall be selected to supply the Boxer CRV. On 13 March 2018, the government approved Rheinmetall as the preferred supplier for Land 400 Phase 2.

1.11 In terms of contract delivery timeframes, the government approved 2021–22 for Initial Operating Capability (IOC)19 for the Boxer CRV. The government was informed, as part of the advice provided by Defence at second pass approval, that Final Operating Capability (FOC)20 would be achieved by 2026–27.21

Contracting arrangements

1.12 On 9 August 2018, Defence signed a contract, valued at $4.28 billion (including GST), with Rheinmetall Defence Australia (Rheinmetall) for the acquisition of 211 Boxer CRVs, 12 mission modules and associated support systems. The vehicles are to be delivered in two blocks (tranches):

  • Block I — comprising 25 vehicles intended to provide an early deployable capability by December 202022; and
  • Block II — comprising 186 vehicles to be delivered between February 2022 and January 2027.

1.13 Table 1.3 below outlines the various vehicle configurations to be delivered in each block and some key features of the different vehicle types.

Table 1.3: Vehicle configurations acquired under Land 400 Phase 2

Vehicle type

Number of vehicles to be acquired

Number of personnel carried

Key additional vehicle features

Block I Multi-purpose Vehicle (MPV)

13

Nine

 

Block I Reconnaissance Vehicle

12

Seven

Includes a turret and 30mm main cannon.

Block II Reconnaissance and Counter-Reconnaissance Vehicle

121

Seven

Includes a turret and 30mm main cannon.

Block II Command and Control (C2) Vehicle

15

Seven

 

Block II Joint Fires Surveillance (JFS) Vehicle

29

Six

Includes an enhanced target acquisition system.

Block II Recovery Vehicle

11

Three

Includes a winch system.

Block II Repair Vehicle

10

Three

Includes a crane.

Total

211

 

       

Source: Department of Defence, Land 400 Phase 2 Combat Reconnaissance Vehicle Acquisition Project Management Plan, Version 1.0, August 2018, pp. 15–16.

1.14 The manufacture and assembly of the Boxer CRVs will progressively transfer from Germany to Australia. The first 25 CRVs are to be manufactured and assembled in Germany, with final integration and acceptance testing undertaken in Australia. During production of the 186 Block II vehicles, manufacture and assembly will transition from Germany to a purpose built facility known as the Military Vehicle Centre of Excellence (MILVEHCOE) in Queensland.23 Defence anticipates full Australian manufacture24 at the MILVEHCOE will be achieved with the 31st Block II vehicle, which is currently scheduled to start production in May 2023.

Project expenditure

1.15 Government approved funding for Land 400 Phase 2 is set out in Table 1.4 below.

Table 1.4: Funding approvals for Project Land 400 Phase 2

Date of funding approval

Amount approved

Funding approval

First Pass (9 December 2014)

$168 million

For Defence to conduct risk reduction activities (Stage two).

Second Pass (13 March 2018)

$15.7 billion

For gross life cycle costs of $15.722 billion, comprising:

  • acquisition costs of $5.899 billion (including contingency); and
  • operating costs of $9.823 billion over the expected 30 year life of the vehicle fleet.
     

Source: Department of Defence.

1.16 As at 30 June 2020 Defence had spent $458.5 million of the approved acquisition budget.

1.17 In addition to the acquisition contract, Defence signed a seven year support contract, valued at $245.1 million (including GST), with Rheinmetall Defence Australia on 20 December 2018. As at 27 May 2020, the amount paid to Rheinmetall under the support contract was $3.27 million.

Engagement with Australian Industry

1.18 In March 2018 the Prime Minister, Minister for Defence Industry and Minister for Defence announced in a joint media release that:

Over the 30-year life of the vehicles, Australian industry will secure two thirds, or $10.2 billion, of the total investment in acquiring and maintaining the fleet, creating up to 1450 jobs right across Australia.25

1.19 Defence has stated that it expects:

The Australian workforce involved in the construction of the vehicles will be used to support sustainment, progressive development and upgrades to the Boxer. They will also be used to support potential export opportunities.26

1.20 Rheinmetall has released an Australian Industry Capability Plan which lists the businesses it has identified as being capable of delivering into the Land 400 Phase 2 acquisition program.27 Reporting on the delivery of the Land 400 Phase 2 Australian Industry Capability Plan is discussed further in Chapter 4.

Rationale for undertaking the audit

1.21 This audit has been undertaken to provide the Parliament with independent assurance of the effectiveness of Defence’s acquisition to date of CRVs under Land 400 Phase 2.

1.22 Defence has described Land 400 as the biggest and most expensive acquisition in the Australian Army’s history. Land 400 Phase 2 was intended to enhance Army’s ability to conduct its core business — sustained close combat — and address the risk of a capability gap emerging by 2020, with Army publicly stating that a life-of-type extension of the ASLAVs was not possible due to obsolescence factors.

1.23 During the course of the tender process for Land 400 Phase 2, the Australian Government’s industry policy changed to focus on building Australian industrial capability to better support Defence. Previous policy focused on reducing risks through the consideration of MOTS equipment, following reviews such as Going to the Next Level, The report of the Defence Procurement and Sustainment Review, 2008 (the ‘Mortimer Review’).

Audit approach

Audit objective, criteria and scope

1.24 The objective of the audit was to assess the effectiveness and value for money to date of Defence’s acquisition of combat reconnaissance vehicles under project Land 400 Phase 2.

1.25 To form a conclusion against the audit objective, the following high-level criteria were adopted:

  • Did Defence conduct an effective tender process that contributed to the achievement of value for money?
  • Did Defence conduct an effective evaluation process that contributed to the achievement of value for money?
  • Did Defence establish effective project governance and contracting arrangements that contribute to the achievement of value for money?

1.26 This audit focuses on the procurement of the 211 Boxer CRVs under Land 400 Phase 2 from Rhienmetall Defence Australia. The audit scope did not include assessments of:

  • the sustainment arrangements for the vehicles or the retirement of the existing ASLAV capability from service;
  • the development of MILVEHCOE, the purpose built vehicle production facility in Queensland; or
  • Land 400 phases 3 and 4 (described in Table 1.1 above).

Audit methodology

1.27 The audit procedures included:

  • the review of Defence records, including advice and procurement documentation; and
  • discussions with relevant Defence personnel and contractors.

1.28 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of approximately $664,000. Audit team members were: Natalie Whiteley, Kim Murray, Clyde Muthukumaraswamy and Sally Ramsey.

2. The tender process

Areas examined

This chapter examines whether Defence conducted an effective tender process that contributed to the achievement of value for money.

Conclusion

Defence’s request for tender process was partly effective. Effectiveness was reduced because Defence did not operate within the bounds of the first pass government approval and did not advise the Minister that it would issue a request for tender for both MOTS and ‘MOTS Plus’ options rather than the government-approved ‘MOTS-based’ approach. Defence did not define the meaning of ‘MOTS-based’ in its advice to the Minister, to inform the government’s decision-making at first pass — one of a number of shortcomings in Defence advice to ministers. Further, three Defence reviews identified that its approach to market, including the cost of tendering and schedule constraints, might have caused some potential bidders to not participate in the tender — potentially reducing competition and limiting the options available to the Australian Government.

Area for improvement

The ANAO has recommended that Defence review the process in place to provide assurance that agreed internal review recommendations have been implemented appropriately and in a timely manner.

2.1 In February 2015, Defence approached the market to acquire combat reconnaissance vehicles (CRVs) through an open request for tender.28 To assess whether Defence conducted an effective request for tender process that contributed to the achievement of value for money, the ANAO examined whether:

  • Defence had defined its requirements effectively prior to undertaking the tender;
  • the advice Defence provided to government was accurate and timely; and
  • Defence’s request for tender (RFT) process was effective.

Did Defence define its requirements effectively?

Defence was effective in defining its requirements. Through its reviews and industry engagement, Defence identified that there were military off the-shelf (MOTS) options available that would meet its requirements for replacing its ASLAV fleet through Land 400 Phase 2. Defence also identified that there would be a need for developmental work to integrate government furnished equipment. A Defence Gate Review in June 2013 reported that the requirements for the project had been adequately defined but the schedule was high risk and that the project was unaffordable.

Identification of vehicle options for Land 400 Phase 2 before first pass

2.2 Defence commenced planning activities, which included the specification of the requirements for Land 400, in the mid-2000s. Defence informed itself about possible options to deliver Land 400 through consultation with industry and engaging consultants to conduct studies, including:

  • Two requests to industry for information (2006 and 2010). From the responses to the 2010 request to industry, Defence determined that its proposed schedule for the project was optimistic.29
  • Two reviews (2009) which examined similar international combat vehicle systems programs to Land 400, and the test and evaluation strategy for the Land 400 program.
  • An Initial Acquisition Strategy Options Study report (June 2010) to identify potential strategies for acquisition and support for Land 400.
  • A Rapid Prototyping, Development and Evaluation (RPDE) ‘Quicklook’30 report (December 2011) to form a view on Australian Industry’s ability to support Land 400.

2.3 A key assumption underpinning the June 2010 Initial Acquisition Strategy Options Study and the December 2011 ‘Quicklook’ report was that the Land 400 vehicles would be a military off the shelf (MOTS) acquisition31 (see Box 1).

Box 1: History of the concept of ‘military off the shelf’ (MOTS)

The Australian Strategic Policy Institute has defined ‘military off the shelf’ (MOTS) equipment as equipment that:

  • is already established in service with the armed force of another country or Australia;
  • is sourced from an established production facility (not just a MOTS design); and
  • has at most minor modifications to deliver interoperability with existing ADF [Australian Defence Force] and/or allied assets.a

This definition was reproduced in Going to the Next Level, The report of the Defence Procurement and Sustainment Review, 2008 (the ‘Mortimer Review’). The Mortimer Review stated that:

Experience shows that setting requirements beyond that of off-the-shelf equipment generates disproportionately large increases to the cost, schedule and risk of projects.b

The Mortimer Review also noted that off-the-shelf ‘does not include upgrade projects where a number of off-the-shelf systems are purchased and integrated together for the first time’.c

Relevant to the period of time when Defence was determining its requirements for Land 400 and seeking first pass approval, the Australian Government’s:

  • 2009 Defence White Paper identified off-the-shelf solutions as the ‘benchmark against which a rigorous cost-benefit analysis of the military effects and schedule aspects of all proposals will be undertaken’; and
  • 2013 Defence White Paper stated that ‘off-the-shelf solutions will continue to provide the basis against which the risks and benefits of more developmental or bespoke procurement proposals will be assessed’.d

Defence’s capability development handbooks in use at the time it was developing the requirements and request for tender for Land 400 Phase 2, reflected government policy of off-the-shelf solutions being the benchmark for Defence capability acquisitions. The relevant handbookse also noted the importance of recognising that the first-time integration of a number of separate off-the-shelf systems is no longer an off-the-shelf solution and is therefore a developmental solution.

Defence’s thinking at this time was summarised in its contribution to the 2010–11 Major Projects Report published in December 2011f:

  • Defence policy mandates that if an Off‐The‐Shelf (OTS) option exists for Defence’s capability requirements, it will be presented for Government consideration and will be the benchmark against which a rigorous cost‐benefit analysis of any additional capability is sought, taking into account the cost and risk of doing so. Likewise, any option that proposes ‘Australianisation’ or modification to an existing OTS capability solution must detail the rationale and associated costs and risks to undertaking such modifications.
  • Military‐Off‐The‐Shelf (MOTS) and Commercial‐Off‐The‐Shelf (COTS) are acquisitions for equipment, hardware or software that already exist, are in‐service with one or more international customers for an equivalent purpose and require minimal or no change.
  • Australianised MOTS are acquisitions where the product is modified to meet unique Australian requirements such as the Bushmaster vehicles.
  • Developmental acquisitions are where an Off‐The‐Shelf product or solution does not currently exist. A solution needs to be delivered through: developing a new product; integrating existing Off‐The‐Shelf components to deliver a new product; or participating in another nation’s development program.
  • It can be expected that cost, schedule and risk parameters increase as the requirement for Australianisation or development work increases.

Note a: Australian Strategic Policy Institute, The Cost of Defence ASPI Defence Budget Brief 2008–09, pp. 161–162.

Note b: Defence, Going to the Next Level, The report of the Defence Procurement and Sustainment Review, 2008 (the ‘Mortimer Review’), p. 17.

Note c: ibid., p. 18.

Note d: The 2016 Defence White Paper and 2020 Defence Strategic Update do not refer to off the shelf acquisitions.

Note e: The 2012 and 2014 versions of Defence’s Capability Development Handbook.

Note f: Auditor-General Report No.20 2011–12, 2010–2011 Major Projects Report, pp. 118–20, available from https://www.anao.gov.au/work/major-projects-report/2010-11-major-projects-report.

Source: ANAO analysis.

June 2010 Initial Acquisition Strategy Options Study report

2.4 The 2010 Initial Acquisition Strategy Options Study, undertaken by a consulting firm on behalf of Defence, examined the issues, decision drivers and potential acquisition and support strategies for the Land 400 program. The findings of the review included that:

  • In the context of warfighting capability, complexity and cost, Land 400 was Army’s equivalent of Air Force’s Joint Strike Fighter and Navy’s future submarine projects, but the project had not attracted the degree of priority or attention that would be expected of a project with a budget well in excess of $10 billion.
  • Defence had not appreciated the complexity of the program, and had not allowed sufficient time for the Requirements Phase of the project.
  • In 2008, at the direction of Defence’s Chief of Capability Development Group, the Group ceased work on the project until Army invested further effort in identifying and documenting its requirements for the capability.32
  • As at June 2010, key capability requirements documents remained incomplete.
  • Defence’s schedule for the program was ‘ambitious’, and platforms needed ‘to be in existence now if they are to meet the schedule requirements for the program’.
December 2011 ‘Quicklook’ report

2.5 Defence’s ‘Quicklook’ process examined Australian industry’s capability to deliver and sustain the proposed Land Combat Vehicle System (LCVS).33 Among other things, the December 2011 ‘Quicklook’ report noted that:

  • The definition of a true MOTS acquisition requires that the integration work necessary to achieve ADF interoperability to be minor, but that this was ‘simply not accurate with respect to Land 400 program’.
  • Defence’s indicative schedule, based on achieving Initial Operating Capability in 2025, did not address the complexity of Defence’s required systems integration activity, and was therefore optimistic.
  • MOTS technology acquired through a 2013 request for tender (the scheduled date at that time) would be obsolete by 2025.
  • Existing and anticipated shortages of engineers were expected to impact repair and maintenance schedules and future upgrades of the vehicles.

Documentation of Land 400 Phase 2 requirements at first pass

2.6 The format and specifics of Defence’s guidance for documenting the requirements for Defence capability projects has varied over the years.34 Nevertheless, three principal capability definition documents that underpin future Defence capabilities have remained constant. These capability definition documents are the:

  • Operational concept document — intended to inform system acquirers and developers of the ADF’s intended military use of the new capability, and provide a context around which the other capability definition documentation can be related and used to support the verification of these requirements.
  • Function and performance specification document — the key requirements document that defines the functions and performance that will be required, in engineering terms.
  • Test concept document — outlines the approach and strategy to verify and validate that the design and operational requirements of the new or upgraded capability have been met.

2.7 Defence’s 2014 guidance on the capability definition documents stated that:

These three documents, if produced thoughtfully and in consultation with key stakeholders, will result in the development of sound requirements that support the cost schedule and risk profiles presented to Government at First and Second Pass. The prime objective of the CDD [Capability Definition Document] suite is to provide a traceable set of requirements…

2.8 Table 2.1 below outlines the status of the three capability definition documents for Land 400 Phase 2 at the time of the government’s first pass consideration in December 2014.

Table 2.1: Status of principal capability definition documents at first pass (December 2014)

Capability definition document

ANAO comments

Operational concept document

Version 4.0 was approved on 8 December 2014 by the Director-General Combined Armed Fighting Systems (for the whole of the Land 400 project).

Function and performance specification (FPS) document (for two vehicle configurations)

Defence developed two FPS documents, one each for the turreted and non-turreted variants of the proposed CRV.

Defence informed the ANAO that the draft FPS document (dated 29 July 2013) was available at first pass. It was not included in the attachments to the Minister’s submission to government.

Defence advised the ANAO in July 2020 that there is no requirement to have an FPS finalised prior to seeking first pass and that only a preliminary FPS is required.

Test concept document

Version 2.0 (August 2013) was endorsed/approved by three Defence Directors-General (for the whole of the Land 400 project).

   

Source: ANAO analysis of Defence documents.

2.9 In summary, Defence had developed the three capability definition documents at first pass, with one in a draft state. In the lead-up to government first pass consideration for Land 400 Phase 2 (and preceding review activity by senior Defence committees), Defence had received input from the Defence Science and Technology Organisation (DSTO) and a Gate Review in mid-2013.

2.10 The DSTO’s July 2013 technical risk assessment noted that the:

Lack of essential requirements in the LAND 400 Function and Performance Specification (FPS) which makes it hard for the Project to demonstrate that LAND 400 will close the identified capability gap, poses challenges for risk assessment including TRA [technical risk assessment], and increases the demand for post-First Pass trade analyses, especially when decisions will need to be made which of the many important FPS clauses should be forfeited.35

2.11 The Gate Review report of June 201336 stated that the project was intended to acquire and support vehicles to replace the ASLAV ‘with a MOTS design’, and the reviewers found that:

… the current project scope was adequately defined for first pass in a series of draft documents which specify, in considerable detail, the requirements and system boundaries.37

2.12 The review also acknowledged that the CRV schedule was high risk and that the project was unaffordable. The Gate Review board recommended that Defence delay the project until clear, affordable options could be developed that would meet stakeholders’ requirements, within a schedule that was not tighter than ‘medium’ risk, and able to be implemented by a project team with a planned level of staffing that was sustainable. Defence determined that this recommendation was complete in November 2013, following a discussion at the July 2013 Project Management Steering Group (PMSG). The steering group minutes record that the program schedule was revised to reduce the schedule risk to ‘medium’:

The revised program schedule, required to reduce schedule risk to ‘MEDIUM, was discussed. It just maintains the requested CRV IMR [Initial Materiel Release] date of 2020 but any further addition of time in the schedule will result in a CRV IMR not before 2021. CA [Chief of Army] acknowledged that this revised IMR timing was still within the guidance that he provided at the 29 Aug 12 PMSG, while highlighting the Army need to replace the in-service ASLAV family of vehicles in the shortest practical time.

2.13 The Gate Review board also recommended that Defence review the governance and program management arrangements for the project.38 Defence informed the ANAO that it reviewed the governance arrangements for the project and decided that the existing arrangements remained appropriate.

Development of first pass Acquisition and Support Implementation Strategy

2.14 The purpose of the Acquisition and Support Implementation Strategy (ASIS) was to document: ‘the recommended strategy for acquiring and implementing the materiel elements of the LAND 400 Phase 2 Mounted Combat Reconnaissance Capability in the manner most likely to achieve project success, i.e. to deliver a Materiel System that is fit for purpose, safe, consistent with Government approval, and achieved on time and within budget.’

2.15 The first pass ASIS was approved in May 2014 and included the following paragraphs in relation to MOTS:

48. A comparison of the available MOTS products against the CRV FPS suite suggests that whilst each of the FPS requirements can be achieved by one or more of the available solutions, there may not be a single solution that is compliant with all of the FPS requirements.

51. Many of the available materiel solutions that could achieve a high level of compliance against the FPS have continued to be further developed by the Original Equipment Manufacturer (OEM). This may allow for MOTS vehicles to be offered that already have a number of mature capability options available for inclusion to achieve an even greater level of compliance with the FPS, but with substantially less technical risk than would be associated with a new development.

2.16 Defence advised the ANAO in July 2020 that:

Para 51 in particular that gave rise to the development of the MOTS Plus definition. For example, it was known that a prototype Boxer with a Lance turret existed — the Boxer was MOTS, the Lance turret was arguably MOTS — but a Boxer with a Lance turret would not meet the definition of MOTS. Without some flexibility around the definition of MOTS there would likely be no more than one or two companies able to tender and they could only offer vehicles systems that were at least 10 or more years old.

2.17 For further discussion of MOTS Plus, see Box 2 later in this chapter.

Did Defence provide accurate advice to government and for first pass approval?

Defence did not always provide accurate advice regarding Land 400 Phase 2. Defence’s October 2014 Gate Review identified that advice provided in the proposed first pass submission to government was not accurate in its representation of the capability being acquired as military off the shelf (MOTS). While available evidence indicates that Defence provided additional context and presented the acquisition as ‘MOTS-based’, it did not define the meaning of ‘MOTS-based’ in its advice to the Minister to inform ministerial and governmental decision-making.

Ministerial briefings

2.18 In October 2012, the Minister had expressed concerns regarding the regularity and fullness of Defence briefings regarding the project. Following receipt of these comments, Defence provided 10 briefings to the Minister on Land 400 through to July 2019.39

2.19 In October 2011, Defence advice to the Minister focused on the delivery of a MOTS platform, with other requirements to be addressed with MOTS products where possible. For example, Defence informed the Minister that:

The platforms are anticipated to be military-off-the-shelf (MOTS) products, and the ancillaries and sub-systems a mix of commercial and military-off-the-shelf products (COTS/MOTS) wherever possible.

2.20 Defence further advised the Minister that:

Australianisation of the platforms will only occur where absolutely necessary, and is primarily expected to encompass those integration requirements necessary to enable the Combined Arms Fighting System.

2.21 In October 2012 and May 2014, Defence again advised the Minister that Defence’s preference was for a MOTS vehicle. The May 2014 advice indicated that Defence had:

… confirmed that suitable armoured fighting vehicle types exist whose designs are proven and in operation with other armies and could be acquired as MOTS.

2.22 Following receipt of the May 2014 advice, the Minister:

  • directed Defence to submit, for first pass approval, a proposal that Land 400 Phase 2 be restricted to the replacement of ASLAV vehicles only, with subsequent phases of the project to be considered later as individual projects40;
  • stipulated that the option to be put to government for the replacement of ASLAV vehicles should be a tender for a MOTS vehicle only;
  • directed that it was to be clearly stated [in the RFT] that there was no requirement for either manufacture or assembly in Australia (although companies were free to provide such offers) and the only Australian industry requirement would be for appropriate in-service support as determined by the Defence Materiel Organisation41 and Army analysis; and
  • directed Defence to put a process in place, or at least an alternative option, to allow government to consider Land 400 Phase 2 for second pass approval by August 2016 (involving a decision on the preferred tender) at the latest.

2.23 In July 2014, Defence again advised the Minister that:

…there are current Military Off the Shelf (MOTS) solutions capable of achieving each of the requirements contained in the KRM [Key Requirements Matrix].42

2.24 After providing a proposed first pass approval submission to the Minister for consideration, Defence undertook a Gate Review of Land 400 Phase 2 which reported in October 2014.

Gate Review October 2014

2.25 The Gate Review reported that the extent of the potential departure from ‘true MOTS’ was greater than had previously been disclosed to Defence senior leadership and the government. The reviewers indicated that this view was based on:

  • difficulties reconciling the project team’s internal advice that it was ‘quite likely that none of the available vehicles will meet all of the specifications’ set by Defence with the description of Land 400 Phase 2 provided to the Minister as a MOTS acquisition with limited modifications; and
  • Defence’s very detailed and specific requirements, that were at odds with the project’s stated procurement strategy based on acquiring MOTS vehicles and largely limiting modifications to the Command, Control, Communications, Computers and Intelligence (C4I) suite.

2.26 The Gate Review Board also indicated that it was concerned that draft advice to government that was provided to the Minister for clearance:

… might not provide an adequate description of the potential departure from true MOTS envisaged and suggested action to be taken to address this. The Board was of the view that a clear and simple description of the envisaged approach to MOTS needed to be developed to ensure common understanding across Defence and with Industry, and for incorporation in the solicitation [request for tender] documentation.

Defence response to the October 2014 Gate Review

2.27 On 17 November 2014, the Chief Executive Officer of Defence Materiel Organisation (DMO) noted, in margin notes to the Gate Review Board outcomes document, that the project team’s statement to the Board — that ‘no prospective tenderer currently has a MOTS vehicle or vehicles that can meet the full ADF’s requirements in all roles’ — was ‘not previously disclosed in any briefing’.43,44 The Chief Executive Officer agreed with the Board’s recommendation that ‘steps be taken to ensure the First Pass [advice to government] … better explains the full extent of potential departure from true MOTS that may be required to meet all CRV roles and to ensure currency of the vehicles acquired.’ The Chief Executive Officer made the following additional margin notes on the Board’s outcomes document:

Thank you for this review and particularly for uncovering the issues surrounding MOTS and the vehicle variants that might need development. I have ensured that the [advice] has been updated to fully describe the MOTS issues.

2.28 While Defence had a process for acquitting the implementation of gate review recommendations, in this case the process was not conducted in a timely manner and did not provide accurate advice about the actions undertaken to address the Board’s recommendation. A minute dated 3 December 2014 from the Head, Land Systems (discussed further at paragraph 2.32) advised the General Manager, Land Materiel that the recommendation had been addressed. The minute was sighted, but not endorsed, by the General Manager, Land Materiel, on 10 December 2015, the day after the government had provided first pass approval.45

2.29 Gate reviews (now known as Independent Assurance Reviews) conducted across acquisition projects help to draw risks and issues to the attention of the head of Defence capability acquisitions.46 Providing assurance that agreed recommendations have been implemented in a timely manner is an important means of confirming that identified risks and issues are being addressed.47 In this instance, advice to Defence management seeking endorsement of the implementation of recommendations was not acted on until after the government had acted on the relevant advice. Defence should review the process in place to provide assurance to its senior leadership that agreed independent assurance review recommendations have been implemented appropriately and in a timely manner.48

Recommendation no.1

2.30 Defence review the process in place to provide assurance to its senior leadership that agreed independent assurance review recommendations have been implemented appropriately and in a timely manner.

Department of Defence response: Agreed.

First pass approval

2.31 In November 2014 the Minister for Defence provided a submission to government seeking first pass approval for Land 400 Phase 2, which was granted on 9 December 2014. Approval was given for Defence to replace the ASLAVs with around 225 ‘MOTS-based’ CRVs of various configurations.

2.32 The 3 December 2014 minute from Head, Land Systems discussed in paragraph 2.28 above had advised the General Manager, Land Materiel, that MOTS and ‘MOTS Plus’ definitions had been agreed and included in the draft of the first pass submission to government prepared by Defence for the Minister’s consideration. On 10 November 2014, Defence provided a revised draft submission to the Minister which included changes to how ‘MOTS’ was described in the supporting advice. Defence’s advice covering the revised draft submission drew this change to the attention of the Minister. The additional wording included in the revised draft submission to the Minister noted:

  • that the project would aim to acquire a MOTS-based solution ‘as far as practicable’ and ‘that is established in-service with another armed force and can be sourced from an established production facility’;
  • that the integration of government furnished equipment would introduce higher risks than that associated with a genuine MOTS procurement;
  • that it was unlikely any of the suppliers will have MOTS vehicles for all the roles required and that it was anticipated tenderers will put forward solutions that require modifications to existing MOTS vehicle types to satisfy these roles; and
  • that during the risk mitigation activities, Defence would assess the level of developmental work required to meet all role variants, the tenderer’s ability to undertake the modifications, and the risk attached to these.

2.33 However, the term ‘MOTS Plus’ was not included in the revised submission to government drafted by Defence or in Defence’s advice to the Minister. The Minister’s first pass submission to government did not include a definition of the terms ‘MOTS-based’, ‘MOTS Plus’ or ‘MOTS’.

2.34 While Ministers involved in Defence matters might be expected to have a reasonable understanding of the MOTS concept, the terms ‘MOTS-based’ and ‘MOTS Plus’ were not in common use and were not defined in relevant Defence guidance such as the 2014 Interim Capability Life Cycle manual.49 It is essential that terms of art used in advice to government are well explained, to support ministerial decision-making and avoid the risk that such terms are not clearly understood by decision-makers.

2.35 On the issue of acquiring a ‘MOTS-based’ solution and developmental risk, first pass government approval was based on advice in the submission that:

  • The project aimed to acquire, as far as practicable, a MOTS-based CRV solution established in-service with another armed force and which could be sourced from an established production facility.
  • The majority of vehicles to be procured would undertake the reconnaissance role, for which an in service MOTS solution was expected or for which proven enhancements were available. However, it was considered unlikely that any of the suppliers would have MOTS vehicles for all the roles required, and it was anticipated that tenderers would put forward solutions that required modifications to existing MOTS vehicle types to satisfy these roles.
  • During the planned Risk Mitigation Activity, Defence would assess the level of developmental work required to meet all role variants, the tenderer’s ability to undertake the modifications, and the risks attached to these.
  • The vehicles would require the integration of government furnished equipment, which was considered to introduce higher risks than that associated with a genuine MOTS procurement.50

2.36 At first pass the government was advised that the project’s overall risk was ‘medium’. This reflected a 2013 technical risk rating51 for Land 400 of ‘medium’ based on Defence’s ‘MOTS acquisition assumption’ — that it was planning only minimal technology development required for integrating current and future government furnished equipment.

Did Defence conduct an effective request for tender process?

Defence’s request for tender process was partly effective. Despite first pass government approval being for a ‘MOTS-based’ solution, Defence incorporated both a MOTS and a ‘MOTS Plus’ option into the tender process and did not bring this change to the Minister’s attention. Eight tenders (four MOTS and four MOTS Plus) were submitted as part of a competitive tender process, but two subsequent Defence reviews and an internal audit identified that Defence’s approach to the market, and the cost of tendering and schedule constraints, might have caused some potential bidders to not participate in the tender — potentially reducing competition and limiting the options available to the Australian Government.

2.37 Defence established an Integrated Project Team (IPT) that included personnel from Army, the Capability Development Group and the Defence Materiel Organisation. The team was responsible for developing, coordinating and managing the tender documentation for Land 400 Phase 2.

2.38 This section examines:

  • Defence’s preparedness to conduct a tender process, including the release of draft documentation to the market; and
  • Defence’s release of the request for tender.

Addressing Defence procurement policy requirements

2.39 The Defence Procurement Policy Manual (the Manual) is the primary reference document for all Defence officials involved in the procurement process and builds on the Commonwealth Procurement Rules.5253 The Manual sets out policy and guidance.54

2.40 For a complex procurement, the Manual required that the following key documents be prepared and approved prior to any request documentation being released to the market:

  • an Endorsement to Proceed for procurements at or above $200,000;
  • a Procurement Strategy that clearly stated the goals and purpose of the procurement; and
  • an Acquisition and Support Implementation Strategy documenting the strategy for acquiring the procured capability.

2.41 In addition, a Tender Evaluation Plan must be prepared for a complex procurement. The Plan is the principal document for the management and conduct of the tender evaluation.

2.42 The ANAO’s assessment of whether key documents for the procurement were prepared and approved as required by Defence’s procurement policies is set out in Table 2.2 below.

Table 2.2: ANAO’s assessment of whether key documents were approved as required by Defence’s procurement policies

Key document and approval

ANAO assessment

Procurement Strategy and Endorsement to Proceed

Defence’s procurement strategy was documented in an Endorsement to Proceed approved in December 2014.

The requirement was met.

 

Acquisition Support and Implementation Strategy

The First Pass Acquisition and Support Implementation Strategy was approved in May 2014.

The requirement was met.

 

The Tender Evaluation Plan

A Tender Evaluation Plan was approved on 2 September 2015.

 

The requirement was met.

The Plan was approved one day before the close of the Request for Tender. The October 2014 Gate Review stated that the project ‘failed to follow better practice by early completion of a Tender Evaluation Plan to ensure the information sought in the tender will allow for effective evaluation’.

   

Source: ANAO analysis of Defence documentation.

2.43 As shown in Table 2.2, the key documents were in place as required by Defence’s procurement policies. Other mandatory requirements of the Commonwealth Procurement Rules and the Manual were also addressed. For example:

  • a Legal Process and Probity Plan was established for the Land 400 program that set standards of practice and behaviours, as well as assigning responsibilities to ensure probity standards were met;
  • a risk assessment was conducted and a risk register established that informed the Land 400 Risk Management Plan, Program Management Plan and Acquisition and Support Implementation Strategy55; and
  • the request for tender and all addendums were released through AusTender.56

September 2014 release of draft requirements to the market

2.44 The Minister agreed to the early release of Land 400 Phase 2 draft requirements documentation to industry in September 2014, prior to government first pass approval in December 2014.57

2.45 The Minister had been advised that release of a draft would assist to manage schedule risks, provide context to the capability need, and assist tenderers to make early commercial decisions on which platforms were suitable to tender. The advice to the Minister included the draft Key Requirements Matrix (KRM).58 Defence’s advice to the Minister stated that ‘there are current Military-off-the-Shelf (MOTS) solutions capable of achieving each of the requirements contained in the KRM’.59

2.46 Defence advised the ANAO that:

… the data released [in 2014] was intended to be sufficient to allow a potential tenderer to:

  • determine if it had a technically competitive solution
  • identify mature technologies that could contribute to a MOTS-Plus offer, and
  • determine likely teaming arrangements.

2.47 Defence further advised the ANAO that the documentation released to industry in 2014 was:

… expressly not ‘request’ documentation as evidenced by the disclaimer on each of the documents and that it was not issued through AusTender: The release of this draft documentation is expressly not intended to be the commencement of a solicitation activity, nor should it be construed as any form of commitment by the Commonwealth that any future solicitation activity will result.

2.48 There were differences between the draft request documentation released in September 2014 and the final request for tender documentation released in February 2015. Defence advised the ANAO that the risk of differences between draft request documentation and the final request for tender were understood and documents were caveated accordingly. The documents released to industry in 2014 included a caveat noting that the information ‘remains subject to change until the publication of any formal industry solicitation documentation’.60

Release of the Request for Tender — solicitation of MOTS and ‘MOTS Plus’ options

2.49 In February 2015, Defence issued an open request for tender for the acquisition and support of vehicles for Land 400 Phase 2. The Request for Tender set out that Defence sought a single contract for acquisition and sustainment, with all tenderers required to submit a MOTS offer. The MOTS offer was to present a vehicle in the primary reconnaissance role that would meet the initial screening requirements. Tenderers were to offer solutions to achieve the technical requirements of all seven combat roles.

2.50 In addition, the request for tender documentation indicated that a ‘MOTS Plus’ offer could be tendered, if a MOTS offer was also tendered. It is not evident from Defence’s documentation how Defence came to incorporate a ‘MOTS Plus’ option when government first pass approval was for a ‘MOTS-based’ solution, or whether this approach was brought to ministerial attention before release of the tender documentation.

2.51 ‘MOTS Plus’ was defined inconsistently across Defence’s Land 400 Phase 2 tender documentation, as set out in Box 2 below.61

Box 2: Definition of ‘MOTS Plus’

In November 2014 (after the draft Land 400 Phase 2 documentation was released to industry) the following definition of a ‘MOTS Plus’ option was approved by Defence’s General Manager Land Materiel:

A single package of pre-existing (non-developmental) upgrades that can be applied to a MOTS baseline (including MOTS baseline variants) in order to deliver an increased level of technical compliance with the Request for Tender’s Statement of Requirement.

The Land 400 Phase 2 project team was directed by Defence’s Head Land Systems Division to update the definition of ‘MOTS Plus’ contained in the request for tender documentation to reflect the approved definition. However, the approved definition of ‘MOTS Plus’ was not incorporated, and the Request for Tender documentation defined a ‘MOTS Plus’ vehicle as a MOTS vehicle that is:

… reconfigured with a package of options or upgrades that enhance or improve the compliance of the vehicle with the technical, functional and performance requirements of this RFT [Request for Tender].

A further definition of ‘MOTS Plus’ was used elsewhere in the procurement process (such as in the stage three evaluation report, discussed further in Chapter 3 of this audit report:

… a MOTS [Military Off the Shelf] baseline vehicle (including MOTS variants) reconfigured with a single package of upgrades in order to deliver an increased level of compliance with the technical, functional and performance requirements of this RFT.

2.52 Tenders were due by 25 June 2015. The due date was extended twice, as follows:

  • Defence agreed to a six week extension to 6 August 2015, following requests from two potential tenderers in March 2015. Defence documentation states that the tenderers required additional time to, among other things, develop high quality Australian industry content bids and review the mission profiles which were late additions.62
  • Following a further request by industry, the tender closure date was extended by a further four weeks to 3 September 2015. Related Defence documentation indicates this was done at the direction of the Minister.

2.53 In response to a brief from Defence, on 3 July 2015 the Minister expressed doubts as to whether Defence’s requirements would result in a ‘true MOTS solution’.

I am not convinced that the current requirements will result in a true MOTS solution — I would like to be briefed further on this issue and the project more broadly.

2.54 The Minister also requested a meeting with the Secretary, Chief of the Defence Force and relevant Defence personnel. In March 2020, Defence advised the ANAO that the requested meeting ‘did not proceed as the topic was addressed to the MINDEF’s [Minister for Defence’s] satisfaction as part of the regular weekly SEC/CDF [Secretary/Chief of the Defence Force] meetings’.63

Defence reviews of the request for tender process

2.55 In April 2016, a Defence Independent Assurance Review of Land 400 Phase 2 concluded that Defence’s requirement for tenderers to bid on both a MOTS and MOTS Plus solution was ‘flawed’ as Defence knew that no MOTS system fully met Defence’s requirements.64 The review noted that this resulted in ‘nugatory cost and effort’ for tenderers and Defence staff. The Board believed that the requirement to submit a MOTS solution in this case was inconsistent with the Smart Buyer approach — a key reform of the First Principles review of Defence.65

2.56 Defence subsequently conducted a review in July 2016.66 The review concluded that Defence’s requirement for tenderers to bid on both a MOTS and a MOTS Plus proposal ‘caused confusion for Industry and diminished the overall effectiveness of the evaluation as the process became more complex and lengthy.’ Further, the review noted that:

Industry identified the cost of tendering as their most significant criticism of the tender process. The high cost of tendering has several implications for both LAND 400 and for Defence in the longer term.

Industry suggested that the cost of tendering led to a number of potential bidders withdrawing from the process. As a result, the cost of tendering may have reduced the level of competition for Land 400 and potentially excluded highly desirable respondents from consideration.

2.57 In respect to these observations, Defence advised the ANAO in July 2020 that:

The decision to participate in a tender is always a commercial decision for a company. There are a number of factors to be considered by them in their decision to bid. No companies advised Defence they would not bid on the basis of MOTS or MOTS Plus considerations. Three companies advised Defence they would not bid. One of them did not bid as they felt their vehicle would not be competitive against the other candidates, one of these did not bid as they did not have a vehicle with a turret with a 30 or 35 mm cannon, and the third company because they did not have a vehicle that could be provided to Defence in the time frame required for RMA [the planned Risk Mitigation Activity].

2.58 The 2016 review identified that industry participants had raised concerns about the high cost of tendering and suggested that Defence’s ‘unrealistic schedule’ led to a number of potential bidders withdrawing from the tender process67, which may have limited the options available to the Australian Government. Further, the review noted concerns expressed by industry and Defence staff about the delays caused by decisions to extend the tender response period on two occasions, and the delay in announcing the outcome of the stage one tender evaluation.

2.59 In March 2017, Defence approved a payment of $2.2 million (incl. GST) to each of the two companies that were not shortlisted to proceed to stage two of the evaluation process, to offset some of their ‘unavoidable costs’ associated with the delay to the tender process.

2.60 An internal audit conducted by Defence in July 2017 observed that:

Despite the overall effectiveness of the RMA [Risk Mitigation Activity] Contract, the strategy to reduce the Commonwealth’s risk by allocating short timeframes (11 weeks) between contract signatures and the requirement to deliver RMA ready vehicles did impact the level of industry participation. Although the RFT sought proven solutions for LAND 400 Phase 2 (both ‘Military Off The Shelf’ and ‘Military Off The Shelf Plus’ options), significant customisation of proposed solutions (for the Military Off The Shelf Plus vehicles) was still required, with the final participant’s vehicle configurations submitted for RMA not being in-service with any nation. The short timeframe had the effect of requiring participants to begin building their vehicles prior to knowing if they had been selected to participate in the RMA. This was done to reduce the level of risk borne by the Commonwealth, but substantially increased cost and risk for industry, decreased industry incentive to participate, and contributed to at least two potential suppliers declining to bid on LAND 400 Phase 2. This ultimately limited the options available to the Commonwealth at Gate 1, but ensured that project timelines were able to be achieved.

2.61 Late in the audit (September 2020) Defence provided the ANAO with a copy of its ‘After Action Review of the Land 400 Phase 2 Source Selection Process’, completed in February 2019.68 The review scope included: examine all aspects of the LAND 400 Phase 2 market solicitation and source selection process; and analyse and identify areas in which improvements are possible in the planning and execution of both the market solicitation and source selection activities, and the management of risks and issues associated with these activities. The review found that the requirement for a non-developmental MOTS solution, together with the inclusion of the opportunity for tenderers to submit MOTS Plus offers, ‘was not explained adequately to industry. It added significantly to the complexity of the subsequent initial evaluation’. The review also found that ‘Industry did not respond as expected to the combination of the MOTS and MOTS Plus requirements, impacting on the vehicles available for evaluation’.

3. Tender evaluation

Areas examined

This chapter examines whether Defence conducted an effective tender evaluation process that contributed to the achievement of value for money.

Conclusion

Defence conducted a largely effective tender evaluation. The process was conducted effectively with the preferred tenderer, Rheinmetall, selected on the basis that its offer provided a higher level of assurance that it could meet Army’s capability and protection requirements in the scheduled timeframe, albeit at a higher cost than the alternative BAE Systems offer. Defence assessed value for money at each stage of the tender evaluation process and documented its assessments. The overall effectiveness of the process was undermined by deficiencies in Defence’s implementation of its risk mitigation activity and shortcomings in its management of conflicts of interest and probity requirements.

Area for Improvement

There is scope for improvement around the maintenance of probity during the procurement process. During a procurement, Defence must adhere to requirements regarding the acceptance of gifts or hospitality to ensure there is no actual or perceived conflict of interest with potential tenderers or tenderers.

3.1 In response to Defence’s Request for Tender for Land 400 Phase 2, four tenderers each submitted both a MOTS and a MOTS Plus bid (eight tenders in total). The objective of the tender evaluation was to determine the tender that best met Defence’s requirements and provided the best overall value for money for the supply and support of the mounted combat reconnaissance vehicle (CRV) capability.

3.2 To form a view on the effectiveness of Defence’s tender evaluation process for the supply of CRVs, the ANAO examined governance arrangements, the initial tender evaluation process (stage one), the subsequent risk mitigation activity (stage two), the final selection process (stage three), Defence’s advice to government regarding the preferred tenderer, and the management of probity.

Did Defence develop an evaluation plan and fit for purpose organisational arrangements to support the tender evaluation?

Defence developed a tender evaluation plan and established a fit for purpose tender evaluation organisation.

Tender evaluation plan

3.3 Defence developed a Tender Evaluation Plan outlining the framework, methodology and processes for the evaluation of responses to the request for tender. The document was approved by Head of Land Systems Division on 2 September 2015.

3.4 The plan set out the following dates for the evaluation of tenders:

  • Stage 1: February–December 2015.
  • Stage 2: October 2015–January 2017.
  • Stage 3: November 2016–August 2017.

3.5 As discussed in Chapter 2, in May 2014 Defence was directed by the Minister to put a process in place, or at least an alternative option, to allow government to consider Land 400 Phase 2 for second pass approval by August 2016 at the latest. In December 2014, the government provided first pass approval for second pass consideration in December 2017, and contract signature by March 2018. Subsequent changes approved by Ministers to the timeframe included:

  • an extension of the Request for Tender by six weeks that shifted consideration of second pass to mid calendar year 2018 (discussed in paragraph 2.52); and
  • a delay in the announcement of the shortlisted tenderers following the completion of stage 1 of the evaluation process, until the review of the AIC involvement was completed in July 2016 (discussed in paragraph 3.27).69

3.6 Defence’s evaluation of tenders took place between September 2015 and January 2018, with government consideration of second pass approval in March 2018.

Organisational arrangements for evaluating the tenders

3.7 The Tender Evaluation Plan set out the organisational arrangements to be established to support the evaluation of tenders. Figure 3.1 below illustrates the three-tier Tender Evaluation Organisation structure. The roles and responsibilities of the elements within the organisation are described in Table 3.1.

Figure 3.1: Tender Evaluation Organisation structure

A figure that shows the three tiered Tender Evaluation Organisation. Tier 1 comprises the Tender Evaluation Steering Group. Tier 2 comprises the Tender Evaluation Board supported by Tender Evaluation Board Secretariat and Tender Evaluation Board Advisory Group. Tier 3 comprises the seven Tender Evaluation Working Groups (Cost and Capability Options Development, Project Management, Systems Engineering, Integrated Logistics, Support and Training, Finance, Commercial and Contracting and Australian Industry Capability)

Source: Department of Defence documentation.

Table 3.1: Roles and responsibilities in the Tender Evaluation Organisation

Tier

Description

Tender Evaluation Steering Group (TESG)—Tier 1

 

The TESG consisted of two-star/SES Band 2 representatives and was chaired by the Head Land Systems Division. It was responsible for approval and oversight of the tender evaluation process for all stages up to pre-Second Pass contract negotiations. The Chair provided final approval of: the Initial Screening Report; incomplete or non-competitive tender reports; the Initial Source Evaluation Report; and the Source Evaluation Report.

Tender Evaluation Board (TEB)—Tier 2

The TEB oversaw the tender evaluation in accordance with the Tender Evaluation Plan. The Board consisted of one-star/SES Band 1 representatives and was chaired by the Director General Combined Arms Fighting Systems. It was supported by a secretariat and a Tender Evaluation Board Advisory Group.

TEB responsibilities included:

  • providing an initial screening report to the TESG, including reports on incomplete or non-competitive tenders;
  • conducting capability, affordability and value for money assessments;
  • developing an overall order of ranking;
  • directing the development and drafting of the source evaluation report; and
  • providing a source evaluation and selection recommendation to the TESG.

Tender Evaluation Working Groups (TEWGs)—Tiers 2 and 3

Seven TEWGs were responsible for conducting the detailed evaluation of tenders against assigned evaluation criteria and sub-criteria. The TEWGs are identified in Figure 3.1 above as blue shaded boxes.

   

Source: Defence documentation.

3.8 The organisational arrangements were fit for purpose and Defence documentation indicates that they were implemented as planned. Roles and responsibilities were well defined, specialist tender evaluation working groups were established to focus on specific evaluation criteria, and a clear hierarchy of advice was established to support decision-making. A secretariat was established to coordinate and support the process as a whole.

3.9 The Australian Government Solicitor was engaged to provide probity advice. The management of probity is discussed throughout this chapter and specifically from paragraph 3.79 onwards.

Did Defence conduct an effective initial tender evaluation process?

Defence conducted a largely effective initial tender evaluation process. Tenders were assessed against the ten evaluation criteria documented in the approved Tender Evaluation Plan and value for money was considered in assessing which tenders proceeded to the planned risk mitigation activity. While the probity advisor undertook a probity review of the initial tender evaluation process, the assessment had a limited sampling approach and no record of the final probity report was retained. The findings from the review noted that a separate probity register was not maintained for the project and there were inconsistencies in the treatment of risk in the tender evaluation working group reports. Defence was unable to demonstrate that the recommendations from the 2016 review of stage one of the tender process had been implemented.

Initial screening of tenders

3.10 The decision-maker was advised, in an Initial Screening Report, that tenders had been subject to an initial screening to identify any tenders that were significantly incomplete, did not meet the minimum content and format requirements, or did not satisfy the conditions for participation identified in the tender documentation. Defence advised the decision-maker that each of the eight tenders it received was substantially complete and fully compliant with the initial screening criteria.70

3.11 Defence assessed that all tenderers offered a vehicle that satisfied the definition of MOTS in the Conditions of Tender.

Tender evaluation stage one — initial evaluation and shortlisting

3.12 The Tender Evaluation Plan provided that each tender would be evaluated as a separate offer.71 During stage one of the evaluation, each valid tender would be subject to: a comparative assessment against ten evaluation criteria; and then a value for money assessment.

3.13 The ten tender evaluation criteria (set out in Box 3 below) included Defence’s technical, functional and performance requirements, the government’s schedule requirements, and the tenderer’s proposed strategy to mitigate the risk of technical obsolescence over the expected 30 year life of the vehicle fleet. Defence decided not to prioritise or assign weightings to the ten evaluation criteria.72

Box 3: Defence’s ten tender evaluation criteria for Land 400 Phase 2

  1. The past performance of contractual obligations (including Australian industry involvement) by the tenderer, including involvement in any contract that is, or has been, listed as a Project of Concern.
  2. The tenderer’s degree of overall compliance with the request for tender.
  3. The extent to which the proposed solution meets the technical, functional and performance requirements stated in the draft contracts, including the Statements of Work and Description of Requirement.
  4. The nature and extent of risks to the Commonwealth or to the tenderer’s capacity to meet the Commonwealth’s requirements.
  5. The technical, corporate and managerial capability of the tenderer to provide the supplies and services in accordance with the Commonwealth’s requirements.
  6. The tenderer’s prices and pricing structure.
  7. The financial viability of the tenderer, its related bodies corporate and proposed subcontractors.
  8. The tenderer’s proposed compliance with Australian industry content requirements, including the proposed value of work performed in Australia.
  9. The ability of the tenderer to meet the Commonwealth’s proposed schedule.
  10. The tenderer’s proposed growth strategy for the mission and support systems.a

Note a: Defence’s intention was that evaluation criterion 10 would enable it to assess each tenderer’s proposed strategy to mitigate the risk of technical obsolescence (for example, upgrades) over the expected 30-year life of the vehicle fleet.

Source: ANAO presentation of Defence documentation.

Assessment of the tenders against the ten tender evaluation criteria

3.14 The Tender Evaluation Working Groups were tasked with assessing the tenders against the ten evaluation criteria.73 The groups assigned a rating for compliance, confidence and risk for each criterion.74 Based on the working group’s reports, the Tender Evaluation Board concluded that not all of the evaluation criteria were useful for discriminating between tenders, and it was not possible to differentiate at all between the tenders against Defence’s criteria 1, 2, 6, 7 and 10. In respect to these criteria:

  • all tenders were non-compliant with criterion 1075;
  • all tenders were ranked equally against criteria 1, 2 and 7; and
  • the information provided by the tenderers in response to criterion 6 was not comparable.

3.15 Based on the remaining criteria (3, 4, 5, 8 and 9), the Tender Evaluation Working Groups assessed the value for money offered for Defence. The following aspects of each tender were assessed:

  • overall capability rating — the extent to which the tender met Defence’s capability requirements;
  • cost per unit of capability;
  • total cost of ownership — total estimated cost of acquisition and support over the 30 year expected life of the vehicle fleet;
  • affordability — the extent to which the tender cost estimates aligned with Defence’s budget provisions for the project at that time; and
  • schedule.
Overall capability rating

3.16 Defence assessed that none of the tenders was fully compliant against all of Defence’s capability requirements for Land 400 Phase 2.

3.17 Defence assessed that, at a 70 per cent level of confidence, the Rheinmetall MOTS Plus tender achieved 79 per cent of the capability sought by Army, and the BAE Systems MOTS Plus tender achieved 68 per cent. Defence’s overall capability ratings for the six unsuccessful tenders ranged between 25 and 52 per cent (at a 70 per cent level of confidence). Defence concluded that the remaining six tenderers offered levels of capability that represented ‘an unacceptable risk to the Capability Manager, regardless of any cost/capability trade-offs that could be undertaken’.

Probity review of stage one assessment

3.18 Defence requested that the Australian Government Solicitor undertake a probity review of the conduct of the tender evaluation process in the Land 400 Phase 2 Request for Tender process. Defence advised that the review was completed in January 2016, prior to completion of stage one of the tender evaluation and the down selection of BAE Systems and Rheinmetall Defence Australia. Neither Defence nor the Australian Government Solicitor retained a final version of the review report. The probity advisers noted that they had undertaken a limited assessment:

… the review was not intended to encompass an exhaustive analysis of the available evaluation documentation … the probity advisers adopted a sampling approach by requesting access to a range of material identified as representative of key aspects of the evaluation methodology, record keeping and probity requirements.

3.19 The purpose of the review was to examine the conduct of the evaluation to date and advise the Program on the extent to which the evaluation activities appeared to be consistent with the following approved suite of documentation governing the process:

  • Land 400 Program Legal Process and Probity Plan (5 March 2015 version);
  • Tender Evaluation Plan; and
  • Tender Evaluation Instruction.

3.20 The draft review findings included:

  • a separate probity register was not maintained and the project team should record probity issues in the conflicts and confidentiality register;
  • other key document registers were maintained for the procurement; and
  • an examination of the draft Tender Evaluation Working Group reports noted that there were some inconsistencies in the treatment of risk, including examples where risk was assessed differently to the guidance set out in the Tender Evaluation Plan.

3.21 While no records were available, Defence advised the ANAO in July 2020 that the findings were discussed and considered by the Tender Evaluation Board Chair and disseminated to the Tender Evaluation Working Group leads.

Outcomes of tender evaluation

3.22 On the basis of its assessment of each tender against the evaluation criteria (discussed in paragraphs 3.14 to 3.17 above) and the value for money assessment, the Tender Evaluation Board concluded that:

  • two of the tenders were — from a technical, functional, performance and value for money perspective — suitable for progression to stage two of the tender evaluation process; and
  • the remaining six tenders offered levels of capability that represented an unacceptable risk to the Capability Manager, regardless of any possible cost or capability trade-offs.

3.23 In February 2016, the Tender Evaluation Board:

  • endorsed the findings and outcomes of stage one of the detailed evaluations of the tenders by the seven Tender Evaluation Working Groups; and
  • recommended to the Tender Evaluation Steering Group that BAE Systems’ MOTS Plus tender (the Patria Armoured Modular Vehicle with various modifications) and Rheinmetall’s MOTS Plus tender (the Boxer CRV with various modifications) should be invited to participate in stage two of the tender evaluation process — the risk mitigation activity.

3.24 On 15 March 2016, the Chair of the Tender Evaluation Steering Group approved the Initial Source Evaluation Report for Land 400 Phase 2. The report set out the Tender Evaluation Board’s findings and outcomes, and its recommendation that two tenders progress to stage two of the tender evaluation process. The Tender Evaluation Steering Group members also endorsed the report.

3.25 The Tender Evaluation Board’s evaluation report also stated that there were a large number of lessons and issues from the initial evaluation process that needed investigation before continuing the evaluation. Specifically, the report stated that:

This Stage-1 evaluation activity has raised and revealed a large number of lessons and issues regarding the construct and conduct of the tender evaluation. This need[s] to be investigated carefully and quickly, in order to inform the revision of the Tender Evaluation documentation for Stages 2 and 3 and to inform the planning and conduct of those Stages.

3.26 The board’s report did not document what lessons and issues needed investigation. Defence advised the ANAO in March 2020 that in April 2016, in response to this finding, it initiated a review of stage one to inform the latter stages of the Land 400 Phase 2 evaluation and the future Land 400 Phase 3 solicitation process. That review is discussed in paragraphs 3.28–3.29 below.

3.27 On 30 March 2016 Defence paused the project with the Minister’s approval, delaying announcement of the successful tenderers. The pause was initiated to allow Defence to review its planned approach to the upcoming risk mitigation activity, to determine whether it aligned with the 2016 Defence Industry Policy Statement released on 25 February 2016.76 On 28 July 2016, Defence announced the March 2016 decision — that it had shortlisted the BAE Systems and Rheinmetall MOTS Plus tenders.

Defence review of the conduct of the stage one tender process

3.28 Defence initiated a review of stage one of the tender process in April 2016. The review report was dated 30 July 2016 and concluded that, notwithstanding some shortcomings and opportunities for improvement, the tender evaluation process was ‘valid, followed relevant policy and the outcome of the evaluation was clear and unambiguous’. The review made 18 recommendations — 14 aimed at improving the performance of the Land 400 Phase 2 project team and four aimed at improving the performance of the wider LAND 400 Program.

3.29 In May 2017, some 10 months after the July 2016 review report was finalised, Defence established an implementation plan, recommendations tracker and user guide to assist with implementation of the recommendations. Defence was unable to demonstrate that the recommendations from this review had been implemented.

3.30 As noted in paragraph 2.61, in July 2018 Defence undertook a further review of the Land 400 Phase 2 source selection and market solicitation process, including the stage one tender process. There were 82 lessons identified in the February 2019 review report.

3.31 In February 2020, the Head of Armoured Vehicle Division provided a minute to the Deputy Secretary, Capability Acquisition and Sustainment Group on the implementation status of recommendations from the 2019 review. The minute noted that all but three recommendations had been implemented by December 2019. Of the remaining three recommendations:

  • two (relating to project administration and managing tenderer engagements) would be reviewed in light of any commentary in this ANAO performance audit of Land 400 Phase 2; and
  • the final recommendation (relating to succession planning within Armoured Vehicle Division projects) was to be addressed and confirmed via the next round of deep dive activities into each business area in the first quarter of 2020.

Did Defence conduct an effective risk mitigation activity?

Defence’s risk mitigation activity was partly effective. While Defence’s decision to conduct a risk mitigation activity as part of the tender process was commensurate with the risk and significance of the procurement, there were issues in the activity’s implementation. These issues included: the vehicles tested did not match the tendered vehicle; minimal evidence was obtained regarding the performance of mandated Australian Government furnished equipment in the vehicles; and claims made by tenderers that were not adequately tested during the risk mitigation activity. Two Defence reviews on its implementation reported mixed findings.

Risk mitigation activity

3.32 The Tender Evaluation Plan documented that shortlisted tenders would be subject to a risk mitigation activity, which was stage two of the tender process. The objective of this activity was to reduce risk to the Commonwealth and further inform the tender evaluation process by increasing Defence’s understanding of the shortlisted tenderers and the claimed capability of the proposed CRVs.

3.33 To participate in the risk mitigation activity, the two shortlisted tenderers were each required to provide vehicles for selective testing and evaluation. Defence’s Tender Evaluation Plan required tenderers to provide Defence with two vehicles ‘in the configuration that was offered in their shortlisted tender response’.

3.34 The Tender Evaluation Plan outlined the following key components of the risk mitigation activity:

  • The vehicles were subjected to a test and evaluation program to confirm the capability claims made in the initial response. The program assessed only the most critical aspects of the tendered solution, focusing on the highest priority activities such as survivability testing against blast and ballistic threats.77
    • The shortlisted tenderers were also required to provide a system level demonstration of their proposal for integrating the mandated Government Furnished Materiel (GFM) on a third vehicle.
  • In parallel, work was undertaken to analyse and assess: cost and capability trade-off options; cost, schedule and technical risks associated with any vehicle configurations; comparisons of the tenderer’s product specification for each capability role against the function and performance specifications; further development of the metrics to be used during the performance of the support contract; opportunities to further evolve a tenderer’s baseline support system approach toward a more availability focussed model; and opportunities to further maximise the proposed level of Australian industry content.
  • Risk mitigation activity participants were required to provide draft contract deliverables to inform any future acquisition and support contracts.

3.35 As noted in paragraph 3.27, Defence reviewed its planned approach to the upcoming risk mitigation activity to determine whether it aligned with the 2016 Defence Industry Policy Statement prior to commencing the activity. In June 2016, the review identified some additional activities for stage two, which Defence considered had the potential to increase the opportunities for Australian industry involvement in the project. Defence advised the Minister for Defence Industry in July 2016 that the activities identified through the review were included in the contracts with the two shortlisted tenderers for the risk mitigation activity.

3.36 The risk mitigation activity commenced in August 2016 and concluded in 2017. The outcomes of the activity were considered during the next (third) stage of Defence’s tender assessment process.

3.37 The tender evaluation report78 advised the decision-maker that the main elements of the risk mitigation activity were:

  • a technical test and evaluation activity which included, but was not limited to, assessing the vehicles’ mobility, useability, and the degree of protection for occupants;
  • a series of parallel workshops where cost and capability trade-off options were assessed;
  • the development of supplementary documentation to support the preparation of draft contracts; and
  • the investigation of opportunities to maximise the level of Australian industry involvement.

3.38 The tender evaluation report indicated that the vehicle configuration provided by the tenderers and tested by Defence during the risk mitigation activity did not match the tendered vehicles. The report also indicated that the tenderers were only able to integrate ‘very little’ mandated Australian Government furnished equipment during the risk mitigation activity, resulting in minimal evidence of the performance of this equipment in the tendered vehicles.

3.39 Defence spent approximately $58.55 million (GST inclusive) on the risk mitigation activity.79 This included $27.5 million (GST inclusive) paid to each tenderer — a total of $55 million — to cover some of the tenderers’ costs of participating in the activity.80

Defence reviews of the risk mitigation activity

3.40 Defence undertook three reviews of the risk mitigation activity: an internal audit (July 2017) and two ‘After Action Reviews’ (December 2017 and February 2019).

Internal audit

3.41 Defence conducted an internal audit of its planning for, and implementation of, the risk mitigation activity for Land 400 Phase 2 during the activity. The July 2017 audit concluded that:

The RMA [risk mitigation activity] has been executed largely to schedule, with planned activities delivering, or expected to deliver, valuable data within the 12 month envelope allocated. The structure of the RMA has provided Industry Participants with the opportunity to fully participate as partners in the process. This, along with the design of the RMA has been effective in allowing Industry Participants to better understand how the ADF operates and supports armoured vehicles and refine their final offers to the Commonwealth. This approach is in line with Smart Buyer principles, and with treating defence industry as a Fundamental Input to Capability.81

3.42 The internal audit report noted that the vehicles submitted for testing:

  • required ‘significant customisation’;
  • were not in service with any nation; and
  • were only representative of a vehicle in the reconnaissance role. The sub-systems specific to the remaining six roles that Defence required were not demonstrated in the risk mitigation activity.

3.43 The internal audit made six recommendations for ‘minor’ process improvements for Land 400 Phase 2 governance and risk management, and identified lessons for Defence to capture for Land 400 Phase 3. By 12 January 2018, Defence had determined that it had implemented all the recommendations. The ANAO’s review of implementation indicated that two recommendations were implemented, two were partially implemented, one was not implemented and for one there is insufficient evidence to determine whether it had been implemented.82

‘After Action Reviews’

3.44 In December 2017, Army conducted an ‘After Action Review’ of the risk mitigation activity to capture lessons that could inform planning for the Land 400 Phase 3 risk mitigation activity and to inform the ‘holistic’ test and evaluation process.83

3.45 The review identified a number of limitations of the risk mitigation activity, including:

  • The approach taken specifically required no comparative assessment between the down-selected vehicles and allowed some stages of the risk mitigation activity to use different test and evaluation methodologies, such as testing against the manufacturer’s claims instead of the functional and performance specifications. This led to difficulties reconciling the results for direct comparison at other stages in the process.
  • Additional specialist stakeholders (with expertise relating to survivability, government furnished equipment supply and integration, explosive ordnance and export controls) should have been involved earlier in the process to clarify requirements for the risk mitigation activity, and assist in establishing the scope and cost of the activity.84
  • The Project Office did not have adequate time to review the Initial Source Evaluation Report to ensure adequate transfer of information from stage one to stage two of the tender evaluation process. As a result, risks identified in stage one did not adequately inform the risk mitigation activity.85

3.46 Defence’s 2019 After Action Review commented that the risk mitigation activity:

… was a good, well-conceived step, allowing the Commonwealth to test tenderer claims and to pursue additional information. Its conduct however was impacted by the schedule slippage caused by the delay in the down-select announcement and industry consultation period — the time available constrained the breadth and depth of the evaluation that could be done.

… The RMA stage met many of its objectives. It allowed the Commonwealth to gain a significantly greater understanding of the tendered vehicles and to test many of the tenderers’ claims about their performance. It allowed the discussion, and resolution of a large number, of program management, commercial and industry matters. It provided CASG and Army with a significantly better baseline of information against which to undertake the final source evaluation.

… On the other hand, there are areas for improvement, in both the design and the conduct of the RMA stage. While the RMA construct, structure and processes were well-conceived and operated reasonably well, there was insufficient attention and vigilance to identifying and resolving issues that were becoming apparent at this stage.

3.47 The 2019 review found that:

… there is evidence of risk having transferred into the contract negotiation phase, due to the Risk Mitigation Activity [RMA] process not being utilised fully to identify and resolve issues …

3.48 The ANAO sought advice from Defence regarding how it obtained assurance that the risks identified in stage one — which the reviews considered had not adequately informed stage two — had been tested. Defence advised that the claims made by tenderers that were not adequately tested during the risk mitigation activity were further examined through parallel negotiations with each tenderer in December 2017. The parallel negotiation process is discussed in paragraphs 3.58–3.66 below.

Did Defence conduct an effective final selection process?

Defence conducted an effective final selection process to identify a preferred tenderer (Rheinmetall) for the supply of mounted combat reconnaissance vehicles for Land 400 Phase 2. Application of the process set out in the Tender Evaluation Plan did not result in an outcome and further steps — involving parallel negotiations and additional criteria — were undertaken to identify a successful tender. Vehicle occupant protection was the primary technical and capability discriminator and heavily informed Defence’s value for money assessment. Defence sought advice from the Australian Government Solicitor on conducting parallel negotiations, and developed internal protocols for conducting these additional steps in the selection process.

3.49 Defence conducted the third stage of its Land 400 Phase 2 tender evaluation between August 2017 and January 2018. Stage three comprised individual and comparative assessments of the tenders with the objective of identifying ‘which shortlisted tender had the greatest likelihood of delivering the best value for money solution to meeting the Commonwealth’s requirements, consistent with Commonwealth procurement policies’.

Stage three tender assessment

3.50 Following the risk mitigation activity, the two shortlisted tenderers submitted ‘optimised’ tenders. Defence advised the ANAO in July 2020 that the optimised tenders were the final version of the tender that was delivered at the end of the Risk Mitigation Activity, and that:

This version of the tender was inclusive of all of the technical test data and user evaluation feedback gathered during the RMA [Risk Mitigation Activity] process; the draft contract data items that had been developed during RMA; revised AIC [Australian Industry Capability] plans that had been informed by the L400-2 [Land 400 Phase 2] AIC Roadshows; and amended pricing reflecting acceptance of the capability options analysis undertaken during RMA.86

3.51 Both tenderers submitted revised tenders which proposed changes to the vehicles submitted for, and tested in, the risk mitigation activity. Defence considered that this process provided:

… an opportunity for both BAESA [BAE Systems Australia] and RLS [Rheinmetall] to deliver final tender submissions that better meet the Commonwealth’s technical, functional and performance requirements. The associated drawback is that this has resulted in the Stage 3 evaluation having to assess a number of significant vehicle sub-systems, and their integration into the vehicle, solely on the basis of documentary evidence (as was the case in Stage 1) because they were not demonstrated or otherwise evaluated during RMA [risk mitigation activity].

3.52 In assessing the final tenders, Defence considered the vehicles ‘as tested’ but ‘made allowances for changes implemented or proposed in the final offer’. Table 3.2 below summarises Defence’s evaluation of the final tenders against its ten evaluation criteria.

Table 3.2: Summary of Defence’s Land 400 Phase 2 tender evaluation — stage three

Evaluation criteria

Ranking: Rheinmetall

Ranking: BAE Systems

  1. Past performance

Equal

  1. Overall compliance with the request for tender

Equal

  1. Technical, functional and performance compliance against six high-level Army operational requirements: protection; lethality; mobility; C4ISR (command, control, communications, computers, intelligence, surveillance and reconnaissance); sustainability; and suitabilitya

First

Second

  1. Risk

Equal

  1. Technical, managerial and corporate capability

Equal

  1. Price and pricing structureb

Second

First

  1. Financial viability

Equal

  1. Australian industry capability

First

Second

  1. Schedule

Second

First

  1. Growth, evolution and obsolescence strategy

Equal

     

Note a: Defence’s evaluation report notes the paramount importance that the Capability Manager has attributed to the protection and survivability of the soldiers that will operate the CRV, meaning that protection was the most influential single contributor to Defence’s overall assessment of evaluation criterion three.

Note b: Defence estimated a total cost of ownership for the BAE Systems and Rheinmetall offers were around double the estimates from stage one of the tender evaluation. Defence’s Source Evaluation Report notes that Defence’s ability to assess the cost of the Rheinmetall tender was constrained by Rheinmetall limiting Defence’s access to its financial data. In contrast, Defence noted a high level of transparency in the BAE Systems Australia tender.

Source: ANAO from Defence documents.

3.53 After assessing outcomes from the risk mitigation activity and the two participants’ revised tenders, Defence could not differentiate between the two tenders against evaluation criteria 1, 2, 4, 5, 7 and 10 (as shown in Table 3.2 above). Defence’s assessment of value for money therefore turned on the remaining four evaluation criteria (3, 6, 8 and 9).87 The value for money assessment did not establish a clear choice between the two tenders.

3.54 At the conclusion of stage three of the tender evaluation process, Defence considered that:

  • the estimated total cost of ADF ownership of Rheinmetall’s Boxer CRV was more than BAE Systems’ Patria;
  • BAE Systems’ offer provided a more transparent and acceptable price and pricing structure and a significantly lower total cost of ownership; and
  • while neither tender met Defence’s specific protection requirements, Rheinmetall’s tender offered a higher level of more assured protection for the Army personnel who would operate it.

3.55 Defence concluded that it could not execute an acceptable contract with either tenderer as there was uncertainty about the capability, engineering, financial and commercial aspects of the final offers submitted by both tenderers.

3.56 In November 2017, the Tender Evaluation Board:

  • endorsed the findings and outcomes for stage three of the detailed evaluation of tenders by Defence’s six tender evaluation working groups and one advisory group;
  • recommended that the Tender Evaluation Steering Group note and endorse the findings and outcomes of the tender evaluation; and
  • agreed that Defence commence parallel negotiations with both tenderers to seek to reduce Defence’s financial and commercial uncertainty about the tenders, and provide an opportunity for the tenderers to improve their offers.

3.57 In December 2017, the Chair of the Tender Evaluation Steering Group endorsed the Tender Evaluation Board’s findings, outcomes and recommendation. Subsequently, Defence conducted parallel negotiations with each tenderer from 5–14 December 2017, with the aim of reducing ‘the areas of uncertainty and ambiguity related to financial and commercial issues and to provide an opportunity for the tenderers to improve their offers’.88

Parallel negotiations with tenderers

3.58 The objectives of the parallel negotiations were to:

  • resolve issues identified during stages two and three of the tender evaluation;
  • improve the base of information on which to make the source selection, including the refinement of plans and programs forming part of each tender;
  • negotiate positions on key issues with each tenderer which could be accepted by the Commonwealth; and
  • provide an opportunity for each tenderer to improve its offer in terms of capability and value for money.

3.59 During the negotiation process, Defence identified options that it viewed as improving the schedule, risk and price of the tendered systems, including reducing the total cost of ownership to Defence by:

  • reducing the number of vehicle variants, by configuring variants to perform multiple roles (for example, using the one variant to conduct Joint Fires and Surveillance roles, combining the repair and recovery variants, and using a proposed variant configured in a Casualty Evacuation role in place of a full service ambulance variant);
  • introducing a ‘raise, train, sustain’ configuration with separate deployment kits in order to reduce both acquisition and sustainment costs;
  • introducing a block approach to vehicle delivery, with a small number of initial vehicles to be delivered early in the project, as a risk mitigation activity for both the capability gap and the project engineering, design and testing activity;
  • improving the approach to the mandated systems engineering reviews; and
  • improving logistic support systems, including an approach more integrated with current land environment simulation planning.

3.60 Defence’s January 2018 Post Negotiation Evaluation Report states that the process:

… provided greater fidelity of information to the Commonwealth and allowed the Tenderers to refine their offers. In relation to many of the ECs [Evaluation Criteria], the additional information did not lead to a change in the assessments of compliance and risk. However, the information led either to adjustments to the assessments for other ECs [Evaluation Criteria] or to a firmer basis for the judgments about, and discrimination between, the tendered offers. This has provided a more informed and assured basis upon which to make the final source selection recommendations.

3.61 Defence’s assessment of the information received during the parallel negotiation process resulted in both tenders being ranked equal against evaluation criterion eight (Australian industry capability) and criterion nine (schedule) and a reduction in Defence’s estimates for the total cost of ADF ownership for both tenderers. The estimated reduction was 8.1 per cent (for Boxer) and 0.8 per cent (for Patria) in the estimated total cost of ownership after the parallel negotiations.

3.62 Defence’s January 2018 Post Negotiation Evaluation Report attributes the significant reduction in the total cost of ADF ownership estimates for Rheinmetall’s offer primarily to a negotiated ‘decrease in the number of vehicles … and expensive sub-systems’ sought by Defence. That is, lower initial acquisition costs were expected due to a reduction in the scope of the project (fewer vehicles and sub-systems) and the removal of the associated 30 years of support for those systems. The report does not document whether a similar approach was adopted in respect to the BAE Systems tender. Defence advised the ANAO in February 2020 that:

All of the cost saving options discussed with Rheinmetall were also raised with BAESA [BAE Systems Australia] including reductions in variant numbers, minimising deployable mission systems and the use of lower spec/lower cost vehicles for raise, train, sustain activities … As the AMV-35 [the BAE offering] does not have the same level of modularity as the Boxer [the Rheinmetall offering] there were significantly fewer opportunities to achieve similar cost savings.

3.63 Following the parallel negotiations, and consistent with its two previous value for money assessments, Defence considered capability, economic benefit to the Australian economy, schedule, pricing, and cost per unit of capability. As in its earlier value for money assessments, Defence also considered the affordability of both revised proposals, but did not include this as a factor in its selection recommendation.89

3.64 In January 2018, Defence concluded that:

  • both tenderers would provide a similar level of capability against Defence’s requirements, with varying relative strengths and differing levels of capability, schedule and cost risk; and
  • the total cost of ADF ownership of both offers continued to be unaffordable against Defence’s budget provisions for the capability, but affordability would not be a consideration in Defence’s selection of a preferred tenderer.90
Assessment of unresolved issues from the Risk Mitigation Activity

3.65 Defence’s parallel negotiation directives (for the separate negotiations with BAE Systems Australia and Rheinmetall) set out the unresolved issues from stage two of the evaluation process (the risk mitigation activity), that were to be examined during the negotiation process.91 These issues related to intellectual property, blast and ballistics remediation, integration of anti-tank guided weapons, delivery schedule, gross vehicle mass and payload, and inaccuracies and omissions in the contract master schedule.

3.66 Of the 44 unresolved issues for Rheinmetall, 16 (36 per cent) were resolved during the parallel negotiation process. Of the 46 unresolved issues for BAE Systems Australia, three (seven per cent) were resolved. Defence’s Land 400 Phase 2 Source Evaluation Report for stage three stated that:

… We do not expect these negotiations to resolve all issues — rather, they are intended to reduce the areas of uncertainty and ambiguity related to financial and commercial issues and to provide an opportunity for the tenderers to improve their offers. This will provide a better informed basis for the final source selection and reduce risk associated with the selection of the preferred tenderer.

3.67 Defence’s 2019 After Action Review considered that the parallel negotiation process:

… was used to identify ways and means of improving the schedule, risk and price of the tendered systems, through consideration of both Commonwealth suggestions and tenderer offers … [and]

… achieved its aim of providing greater fidelity of information and a more informed and assured basis upon which to make the final source selection recommendation; it provide[d] a firmer basis for judgments about, and discrimination between, the tenderers. Both tenderers refined their offers and, significantly, provided improved price and value for money offers.

3.68 That said, the 2019 review concluded that there was:

… clear evidence of complexity, uncertainty and risk that was transferred into the contract negotiation phase due to the Risk Mitigation Activity [RMA] and Parallel Negotiation processes not being utilised fully to identify and resolve issues.

Selecting the successful tender

3.69 In January 2018, Defence determined during the parallel negotiation process that:

The final quantitative assessment, applying the analytical tools selected for this tender evaluation, indicates that, overall, the two Tenderers provide a similar level of capability against Army’s MCRC [Mounted Combat Reconnaissance Capability] requirements, with varying relative strengths. Necessarily, therefore, judgment and more subtle discriminators must be applied to differentiate the two offers and determine a preferred tenderer.

3.70 Defence applied the following additional discriminators to the tenders:

  • undertakings versus evidence;
  • developmental versus proven solutions; and
  • commercial structures.

3.71 In this assessment, Defence judged that protection was the ‘primary technical or capability discriminator’.92

3.72 Defence concluded that overall, the offer from BAE Systems Australia provided a ‘more transparent and acceptable price and pricing structure’ and ‘a significantly lower’ total cost of ownership for Australia (12 per cent less than the Rheinmetall offer). However, Defence considered that Rheinmetall’s tender offered a higher level of more assured protection for the Army personnel who would operate it. At the conclusion of the parallel negotiation process on 7 January 2018, the Chair of the Tender Evaluation Steering Group endorsed the recommendation that the preferred tenderer was Rheinmetall. The Parallel Negotiation Evaluation Report stated that:

… on the basis of the confidence and assessed level of risk in the tenderers’ capacity to deliver the required level of protection in an acceptable timeframe that Rheinmetall Landsysteme offers better VFM [value for money].

Did Defence provide appropriate advice to Government to inform second pass approval?

The advice provided to government to inform its March 2018 second pass approval reflected the outcome of the evaluation process and appropriately informed government of judgements made to select the recommended tenderer. The advice to government regarding technical, schedule and cost risks was not, on its face, fully aligned with the documentation of risk in Defence’s January 2018 Post Negotiation Evaluation Report. Defence advised the ANAO that the advice to government on these risks took into account decisions made by Defence to mitigate risks identified in the Post Negotiation Evaluation Report.

3.73 In March 2018, the Australian Government provided second pass approval for Land 400 Phase 2.

3.74 The government had agreed at first pass that the Minister for Defence would provide options differentiated by capability levels and price points to government at second pass. The advice to government recommended that it approve the acquisition of 211 Rheinmetall Boxer CRVs (211 drive chassis and 223 mission modules) and associated support systems at a cost of $15.722 billion, comprising acquisition costs of $5.899 billion and future operating costs of $9.823 billion. Two options were presented to government: one as recommended (Rheinmetall Boxer); and one as acceptable to Defence should the government decide the recommended option was unaffordable (BAE Patria). The government approved the acquisition as recommended.

3.75 In support of the recommendation to acquire the Rheinmetall Boxer CRVs, the government was advised that while neither vehicle satisfied all elements of Defence’s desired protection requirements, the Boxer CRV was considered the ‘lower risk, more proven option’, and that Rheinmetall could ‘remediate its protection shortfalls with comparatively lower additional engineering effort’ and was assessed as having ‘low developmental risk’.

3.76 The advice to government was that Defence had assessed the overall risk for Rheinmetall’s Boxer CRV as ‘medium’, based on Defence’s assessment of: technical risk (medium)93; schedule risk (medium); and cost risk (low).

3.77 Defence’s risk assessment documented in the January 2018 Post Negotiation Evaluation Report had described these risks as follows:

  • Technical — ‘Deficient Significant’, with a ‘High’ level of risk against EC3 (technical, functional and performance requirements).
  • Schedule — ‘Deficient-Significant’ with a ‘High’ level of risk against EC9 (the ability of the tenderer to meet the Commonwealth’s proposed schedule).
  • Cost — ‘Deficient-Significant’ with a ‘High’ level of risk against EC6 (prices and pricing structure).

3.78 The March 2018 advice to government regarding technical, schedule and cost risks was not, on its face, fully aligned with the documentation of risk in Defence’s January 2018 Post Negotiation Evaluation Report. Defence advised the ANAO that ‘differences in risk methodologies make direct comparison or reconciliation difficult’ of the outcomes of the risk assessments. Defence further advised the ANAO that:

  • The difference in schedule risk is largely attributable to the introduction of the Block I interim capability (recorded in the post negotiation evaluation report) which was sufficient under the risk assessment method to reduce schedule risk to ‘medium’ for the purposes of reporting the risk to government.
  • The cost risk presented to government was based on a comparison of the estimated total cost of ownership against Defence’s internal budget provisions (essentially an affordability risk) which Defence mitigated by supplementing the Land 400 Phase 2 project with offsets from other Army programs. In contrast, the cost risk assessed in the evaluation process focussed on Defence’s confidence of the tenderers’ supporting cost models.

Did Defence effectively manage probity during the tender process?

Defence’s management of probity during the tender processes was partially effective. While Defence had a plan, policies and procedures in place to manage probity, and sought advice from a probity adviser throughout the tender and evaluation process, there were gaps in relevant documentation and shortcomings in the project’s management of conflict of interest and probity against requirements.

Probity advice

3.79 The Australian Government Solicitor (AGS) provided probity advice during the tender and the three-stage evaluation process, including the parallel negotiations. Under the Land 400 Legal Process and Probity Plan, the role of the probity advisor was to:

  • Assist the project in meeting the legal process and probity requirements outlined in the plan and provide advice to the project in the event that any legal process or probity issues arose during the conduct of the project.
  • Provide advice on the conduct of the project, including involvement in the development and review of project documentation to ensure that:
    • applicable rules and procedures were followed; and
    • processes for managing communication with parties external to the project, including potential tenderers and tenderers to the project, were established and complied with.

3.80 Defence documentation indicates that AGS:

  • Conducted a probity review of the conduct of stage one of the tender evaluation process in the LAND 400 Phase 2 Request for Tender (RFT) process.
  • Provided advice to Defence in September 2016 on the probity risks involved in Rheinmetall’s proposal to Defence to change the contracting entity from Rheinmetall Landsysteme GMBH to Rheinmetall Defence Australia.
  • Provided advice to Defence in March 2017 on the probity risks involved in changing one of the evaluation criteria (criterion 9, relating to schedule) in the RFT and the approved tender evaluation plan. In April 2017, AGS indicated that the proposed amendment raised a number of complex issues for the evaluation and potentially significant probity risks.
  • Provided advice to Defence in November 2017 on potential probity issues with the draft Source Evaluation Report for stage three of the tender evaluation process.
  • Provided advice to Defence In November 2017 on the probity issues associated with the proposed parallel negotiations as part of the Land 400 Phase 2 tender evaluation.
  • Provided probity advice to Defence in February 2018 on the (already endorsed) Post Negotiation Evaluation Report.
  • Provided probity advice to Defence in December 2017, and again in August 2018, on Defence’s Source Evaluation Report for stage three of the tender evaluation process.

3.81 AGS advice to Defence indicated that its probity advice did not include a review of:

  • the conduct of the detailed evaluation by the Tender Evaluation Working Groups during stage three of the tender evaluation, nor the tenders received;
  • the risk mitigation activity deliverables or documentation; or
  • documentation supporting the stage three Source Evaluation Report.94

Management of conflicts of interest

3.82 Defence requires its personnel to act in the public interest and avoid or effectively manage actual, potential or perceived conflicts of interest between their private interests and official duties.95 Defence documented its requirements for managing conflicts of interest for Land 400 Phase 2 in the Land 400 Legal Process and Probity Plan.96

3.83 The ANAO examined Defence’s management of conflicts of interest for Land 400 Phase 2, in particular whether the following requirements documented in the Legal Process and Probity Plan were adhered to:

  • maintenance of a conflicts register for the duration of the project;
  • appropriate escalation of any disclosed actual, perceived or potential conflict of interest, including that all such conflicts were recorded in the conflicts register and reported in accordance with the requirements of the plan;
  • review of completed conflict of interest declarations by the probity advisor; and
  • review of the conflicts register by the probity advisor to ascertain whether all identified conflicts and their treatment were recorded.

3.84 The ANAO’s review indicated that the project has a conflicts register and has sought probity advice on potential conflict of interest issues. However there were some shortcomings in the project’s management of its conflict of interest requirements, including:

  • the conflicts register does not include information of the conflicts that had been declared nor the potential probity issues identified;
  • it is unclear on what basis the Commercial and Contracting Team determine which conflict declarations are escalated for probity advice and how many such declarations have been made; and
  • since 2016, Defence did not ensure that the conflicts of interest register was reviewed by the probity advisor.

3.85 In August 2020, Defence advised the ANAO that:

The Project has noted ANAO’s request for maintaining a registry of conflicts declared regardless of the outcome and has instigated an immediate change to the Conflict of Interest register to record all declarations of interest regardless of the outcome of the assessment.

3.86 Between November 2016 and March 2018, during stage three of the tender evaluation, Defence identified two instances of a perceived conflict of interest and two instances of actual breaches of the Legal Process and Probity Plan (discussed below). Defence determined at the time that none of these cases affected the integrity of the Land 400 Phase 2 evaluation process.

Perceived conflicts of interest
November 2016

3.87 In November 2016, Defence identified a perceived conflict of interest of a member of the Land 400 Phase 2 Tender Evaluation Board.97 This incident involved comments made by a member of the Tender Evaluation Board which may have led to a perception of bias. The member subsequently withdrew from the Tender Evaluation Board. In response to a Defence request for advice on the matter, in February 2017 AGS confirmed that Defence actions to date should remove any future risk that the individual’s conflict of interest could be perceived to affect the evaluation process, and suggested an additional measure to further address that risk.98

3.88 AGS also indicated that in the context of managing any potential perceived conflict of interest in the project to date, Defence should consider whether a sufficient potential risk existed to warrant an examination of the evaluation process to date (that is, the initial selection and risk mitigation activity).

3.89 In July 2020, Defence advised the ANAO that having assessed the potential risk, Defence accepted the member’s resignation from the Tender Evaluation Board and redirected team members who were directly involved in the evaluation to an alternate supervisor to remove any probity issues or concerns.

Sponsorship

3.90 In March 2018 Defence sought and obtained advice from AGS, in its capacity as probity advisor for Land 400 Phase 2, on the probity risks associated with Rheinmetall Simulation Australia’s sponsorship of an Army sporting team and an Army support association. AGS indicated that the nature of the arrangement could potentially be perceived as a conflict of interest within the definition of the Legal Process and Probity Plan, but did not appear to represent a substantive probity risk to the integrity of the Land 400 Phase 2 procurement. No changes were made following the provision of advice by AGS.

Breaches of the Legal Process and Probity Plan
Dinner to discuss RMA contract negotiations

3.91 In relation to the acceptance of gifts and hospitality, the 2015 Land 400 Legal Process and Probity Plan states that:

No person involved in the Project may accept or solicit gifts or hospitality from any party that has a likely or potential interest or association with the Project, including from a tenderer or potential tenderer to the Project. Any offer of a gift or hospitality from such a party should be referred to DPM L400 and recorded in accordance with DMH (FIN) 0-0-006 — Accepting and providing gifts, hospitality, working meals and sponsorship.

Should personnel involved in the Project consider that exceptional circumstances exist that warrant a variation to this blanket policy, they are to seek the written approval of DGCAFS, or if appropriate the Head of Land Systems Division (HLS).99

3.92 Prior to the conclusion of the risk mitigation activity, Rheinmetall paid for a dinner for two senior Defence personnel.100 Defence’s register of gifts received in 2017–18 states the following: ‘Dinner — LAND400 Ph2 RMA [risk mitigation activity] contract discussions’.

3.93 Defence’s overarching policy (dated May 2015 and in place at the time of the dinner) on Defence employees and ADF members accepting gifts and hospitality stated that it was inappropriate for a Defence official to accept gifts or benefits from a person or company involved in a tender process with Defence, and/or is the subject of a decision within the discretionary power or substantial influence of the official concerned. Further, the policy required that officials ensure that the acceptance of any hospitality was for the purposes of achieving Defence outcomes and did not give rise to a conflict of interest or compromise Defence’s reputation. The policy noted that the main risk of accepting a gift or benefit is that it may result in an actual or perceived conflict of interest, and that at the extreme, it could be perceived as a bribe.

3.94 Defence’s Head Land Systems Division approved the acceptance of this hospitality and described the dinner as a ‘working meal’ which involved discussion of other contracted work undertaken by Rheinmetall for Defence. The declared amount for the dinner was $150.

3.95 Defence advised the ANAO in July 2020 that:

Defence acknowledges that this activity was incorrectly described in the Gifts and Hospitality Register. The exceptional circumstances were the very limited time available for both principals to meet.

Solicitation of employment from shortlisted tenderer

3.96 In July 2017 a member of the evaluation team solicited employment from one of the shortlisted tenderers for their partner.

3.97 Defence advised the ANAO in August 2020 that the team member was an external contractor, and was immediately removed from the tender evaluation team and not permitted any further engagement with the project in any capacity. Defence further advised the ANAO that it did not deem it necessary to engage the probity advisers on this matter.

4. Project governance and contracting arrangements

Areas examined

This chapter examines whether Defence established effective project governance and contracting arrangements that contributed to the achievement of value for money.

Conclusion

While Defence has established largely effective project governance and contracting arrangements, required project governance documentation has not been maintained. Defence has been establishing governance structures to help integrate activity within Defence and build its relationship with the prime contractor, but has been less effective in settling project-level governance arrangements and maintaining project documentation, including an up to date project risk register. There is evidence of high level and active management of known risks by the parties, including risks to schedule.

Area for improvement

There would be merit in Defence aligning the Initial Operational Capability (IOC) date documented in its 2019 Material Acquisition Agreement and the IOC date agreed by government at second pass.

4.1 The Defence Capability Life Cycle Manual sets out the frameworks for project governance and management and the roles and responsibilities in the delivery of a project.101 Against the frameworks set out in the manual, the ANAO examined whether Defence has established fit for purpose governance arrangements for the delivery of Land 400 Phase 2, including relevant project governance documentation and structures to support its monitoring, management and reporting of contractor and project performance.

Has Defence established a fit for purpose project governance framework — project governance documentation?

Defence has been partially effective in maintaining the required project governance documentation. In particular, there is a misalignment between the Initial Operational Capability (IOC) date documented in Defence’s 2019 Material Acquisition Agreement and the IOC date agreed by government at second pass. These dates are currently out of alignment by four and a half years. Defence advised the ANAO that it considers the achievement of Interim IOC by December 2022 is consistent with Government approval at second pass for IOC by 2021–22.

Project management arrangements

4.2 Project management responsibility rests with the Armoured Vehicle Division in Defence’s Capability Acquisition and Sustainment Group (CASG). The project is managed day-to-day by the Land 400 Phase 2 project team. Defence has also established specific governance arrangements to support its monitoring and management of project delivery against contracted requirements. These governance arrangements include a Strategic Relationship Board and Project Management Stakeholder Group which are examined in the next section (from paragraph 4.20). This section focuses on Defence’s development of project governance documentation.

Project governance documentation

4.3 The August 2017 Updated Interim Capability Life Cycle Manual outlined the key governance documents for projects. The three key documents and the date these documents were signed for Land 400 Phase 2 are summarised in Table 4.1 below.

Table 4.1: Key project governance plans listed in the Capability Life Cycle Manual

Document name and description in the Capability Life Cycle Manual

ANAO comment

Project Directive

The formal approval issued by the Vice Chief of the Defence Force after an approval has been issued by government. The Project Directive reflects the approved scope and timeframes for the next sequence of activities, as well as the responsibilities and resources allocated and the key risks and issues.

The most recent project directive for the project is dated 23 June 2015.

It was not updated to reflect second pass government approval in March 2018.

Integrated Project Management Plan

This describes the detailed conduct of activities to deliver and sustain a product.

An Acquisition Project Management Plan was signed on 24 August 2018, shortly after Defence entered into the acquisition contract with Rheinmetall Defence Australia (9 August 2018).

Product Delivery Agreement

Also referred to as the Materiel Acquisition Agreement. The agreement provides the metric-focused baselines for performance reporting, in order to support the consolidation and aggregation of performance data at a Program and Portfolio level.

The current version of the Materiel Acquisition Agreement was signed by the Capability Manager on 15 May 2019.

   

Source: August 2017 Updated Interim Capability Life Cycle Manual.

4.4 The three documents are discussed further below.

Project Directive

4.5 Defence’s Capability Life Cycle Manual lists Project Directives as a key governance document. Previously known as Joint Project Directives, they state the terms of government approval, reflecting the approved scope and timeframes for activities, responsibilities and resources allocated, and key risks and issues.102 Project Directives are used by Defence to circulate an agreed summary of key project information to relevant personnel within the Department without risk of breaching rules for handling decisions of Cabinet.

4.6 The most recent project directive for Land 400 Phase 2 is dated 23 June 2015. In December 2019, Defence informed the ANAO that it has not updated the project directive to reflect the government’s March 2018 second pass decision for Land 400 Phase 2 because project directives are no longer required in Defence.103 This is contrary to the requirements of Defence’s instructional material applying at the relevant time — the 2017 Interim Capability Life Cycle Manual.104

Project Management Plan

4.7 The Project Director for Land 400 Phase 2 approved an Acquisition Project Management Plan on 24 August 2018. This occurred shortly after Defence entered into the acquisition contract with Rheinmetall Defence Australia (9 August 2018). The plan states that it is the primary project management reference for direction and guidance related to the conduct of the Land 400 Phase 2 project by the Capability Acquisition and Sustainment Group (CASG). The plan describes how the project is to be managed. The project management framework is set out in the plan and related documents, including the Material Acquisition Agreement, Project Directive and Acquisition and Support Implementation Strategy.

4.8 The 2017 Defence Internal Audit of the Land 400 Phase 2 Risk Mitigation Activity (discussed in paragraphs 3.41–3.43) recommended that Army put in place planning to establish and chair appropriate ongoing program and project governance frameworks, and that the project management plan should capture the ongoing governance structure.105 As of January 2020, the agreed recommendation remained incomplete.106

4.9 The Acquisition Project Management Plan indicates that assurance regarding the advice provided to Defence and government for Land 400 Phase 2 is to be obtained through Independent Assurance Reviews conducted prior to key project milestones. These reviews allow early identification of problem projects and products, facilitating their timely recovery.107 The Review Board is to be chaired by the Deputy Secretary Capability Acquisition and Sustainment Group (CASG) or a nominated representative, and will comprise CASG senior executives and an external member. To date, there have been five independent assurance reviews of Land 400 Phase 2: June 2013 (discussed in paragraphs 2.11–2.13), October 2014 (discussed in paragraphs 2.25–2.28), April 2016 (discussed in paragraph 2.55), October 2017 and March 2019.

Materiel Acquisition Agreements

4.10 Defence uses Materiel Acquisition Agreements to establish an overarching agreement between the Capability Manager (the customer, in this case Chief of Army) and CASG (the supplier). The agreement for Land 400 Phase 2 sets out the quantities and levels of military equipment and supplies that CASG is to acquire on behalf of the capability manager. The agreement also includes: key project milestones; 12 measures of effectiveness; criteria for milestone completion including initial operating capability (IOC)108 and final operating capability (FOC)109; customer furnished supplies; the sustainment arrangements; and the agreed budget. The agreement for Land 400 was originally executed in January 2015 and was updated most recently in May 2019.

March 2019 Independent Assurance Review

4.11 A March 2019 Independent Assurance Review noted that Defence had agreed internally to delay IOC from 2021–22 to 2025 and that this did not align with the 2015 Materiel Acquisition Agreement (MAA) or Defence’s original advice to government. The Review’s summary assessment (dated 9 April 2019) stated that:

I note the agreement between CASG and Army for a delay of IOC to 2025 and an Interim IOC in the 2022 timeframe, noting the precise date cannot be finalised until the Block I schedule has stabilised. The current planned IOC and dates do not align with the current MAA or the original advice to government. The plan for an Interim IOC in 2022 and IOC in 2025 is sound, however, there is a need to update Government with these developments, along with a general update on the project status.

4.12 The updated (May 2019) MAA includes the following milestones:

  • December 2022 — interim initial operating capability (IIOC) for Block I vehicles. The definition of IIOC is: ‘IIOC will occur when the first operationally deployable CRV element (likely to be the first Mounted Combat Squadron, including mission, support and training systems, and facilities, if required, has been delivered to the first Combat Brigade and support organisations and accepted into operational service.’ The Agreement also states that: ‘Specialist CRV variants such as Ambulance, Repair and Recovery may, depending upon development timelines, not be available to support IIOC in which case these functions would be represented by suitable in-service assets (e.g. Protected Mobility Vehicle Ambulance, L1213B Heavy Recovery Vehicle)’;
  • December 2026 — initial operating capability (IOC). IOC is documented in the Agreement as occurring more than four years later than the second pass approval date of 2021–22 agreed by government110; and
  • June 2027 — final operating capability (FOC).

4.13 Defence has not provided the ANAO with a clear explanation as to why it adopted the unusual concept of ‘interim initial operating capability’ (IIOC) in the MAA, or why it chose to document in the MAA that IOC would occur later than approved by government. In practical terms, however, the MAA indicates that the capability approved by government for delivery at the IOC stage will instead be delivered at the IIOC stage. In July 2020 Defence advised the ANAO that it:

… considers the achievement of Interim IOC by December 2022 is consistent with Government approval at Second Pass for IOC by 2021–22.

4.14 In October 2020, Defence further advised that:

Defence maintains its assessment that, subject to the impact of COVID-19, it will achieve Initial Operating Capability on schedule in Financial Year 2021/2022, as approved by Government at second pass.

4.15 As originally observed by the IAR reviewers, there is merit in Defence providing advice to government on the project’s status and schedule. There is also merit in Defence aligning the IOC date documented in its MAA with the government second pass approval date for IOC. In October 2020 Defence advised the ANAO that:

Defence agrees there is misalignment between the CABSUB (approved by Government at Second Pass) and Defence’s Material Acquisition Agreement, and will ensure this is updated to achieve alignment.111

4.16 In addition to considering the Material Acquisition Agreement, the March 2019 Independent Assurance Review considered schedule, budget, contractor performance, risk management, workforce, and project documentation. The Review Board noted that the Land 400 Phase 2 project was exhibiting early signs of performance issues. Based on its considerations, the Review Board:

  • found that while Defence’s acquisition strategy is to deliver a MOTS vehicle, the selected vehicle with its planned modifications involves significant elements of development;
  • noted that leading project indicators suggested that Rheinmetall was struggling and the outlook was a matter of some concern;
  • queried the value of certain activities being undertaken in addition to the contract;
  • noted that the project required a more deliberate strategy of savings as the project budget was under pressure, with all but $13 million of $658.3 million contingency allocated, and no management reserve;
  • noted that to avoid a potential two year capability gap, Defence had contracted for an accelerated acquisition of 25 vehicles (Block I vehicles), to provide an initial deployable capability by July 2021. These vehicles would have a different configuration to the balance of the vehicles to be acquired under the project (known as Block II vehicles); and
  • noted that some of the three key project documents were dated, not endorsed, or did not refer to the existence and implications of Block I and Block II vehicles.112

4.17 The Review Board also noted that Defence’s internal reporting did not reflect project slippages and schedule pressures to senior stakeholders. Defence’s internal reporting to senior stakeholders is discussed further in paragraphs 4.22–4.26 below.

4.18 The Review Board made nine recommendations to the Deputy Secretary Capability, Acquisition and Sustainment Group (CASG). These included: reviewing the Block I schedule; implementing a savings strategy; establishing governance across the relevant advisory boards; and confirmation of Project Maturity Scores,113 the project’s ACAT rating,114 and that the project is not a candidate for Project of Interest/Concern. The recommendations and actions outlined were endorsed by the Capability Manager’s delegate. CASG’s Head Armoured Vehicle Division responded to the Board, advising that many of the recommendations had been implemented as of 9 April 2019.

Has Defence established a fit for purpose project governance framework — arrangements to support the monitoring, management and reporting of contractor performance?

While Defence has established a fit for purpose governance framework to support its monitoring, management and reporting on contractor and project performance, implementation of the governance framework has been inconsistent. There have been delays in establishing project-level governance arrangements and the project risk register has not been kept up to date. Defence’s management in these respects has not been fully effective for a project that is developmental in nature and is to replace a capability that is expected to reach its life of type in 2020. Further, key executive reporting mechanisms for the project do not accurately reflect the project status including the significant technical and schedule challenges identified through the project’s monitoring mechanisms, which impacts the timely identification of emerging risks by Defence senior stakeholders. There is, however, evidence of high level and active management of the relationship between Defence and the prime contractor, Rheinmetall.

4.19 Defence is responsible for ensuring that the contracted scope of capability will be delivered in a timely and cost-effective manner and to establish a successful relationship with Rheinmetall for the acquisition and sustainment of the Boxer CRVs. The agreed contract forms the basis for the relationship and should include provisions to enable effective monitoring, management and reporting of contractor performance. Defence has also introduced specific governance structures to support its monitoring, management and reporting of contractor and project performance.

Project management and monitoring

4.20 Defence’s August 2018 Acquisition Project Management Plan establishes the mechanisms for Defence to monitor the project against schedule. These mechanisms include a series of meetings to share information and expertise about the project from various viewpoints (senior management, working level, risk and assurance, and the relationship between Defence and the prime contractor). These meetings are intended to routinely review the status of the project, risks and schedule.

4.21 The following groups have responsibilities for monitoring and managing project delivery:

  • Strategic Relationship Board;
  • Project Management Stakeholder Group (and previous iterations of this Defence senior leadership stakeholder group); and
  • Integrated Project Team.
Strategic Relationship Board

4.22 Defence established the Land 400 Phase 2 Capability Strategic Relationship Board in 2018. The Board’s terms of reference state that it is the ‘senior forum through which the strategic relationship between the Commonwealth of Australia and Rheinmetall Defence Australia is led, directed and overseen’.115 The specific scope of the Board includes:

  • strategic level management of the acquisition and support contracts between CoA [Commonwealth of Australia] and RDA [Rheinmetall Defence Australia];
  • resolution of strategic level issues and developments, and development of strategies to manage strategic level risks; and
  • exploration of opportunities to enhance the capability being delivered through Phase 2.

4.23 The Board is to meet every four months, or otherwise as agreed by the co-chairs. As at August 2020, the Board had met on five occasions: October 2018, March 2019, July 2019, December 2019, and May 2020.116 The minutes of these meetings indicate that Defence has been active in raising concerns, such as delivery against milestones and the project schedule, and agreeing outcomes with Rheinmetall. At its December 2019 meeting, the Board discussed a number of project issues. These included: delays in achieving key milestones under the contract (discussed further in paragraph 4.33 below); the impact of workforce issues on contracted deliverables; relationship principles and behaviours; risks associated with redesigning aspects of the vehicle; and the integration of developmental and unproven technology.

4.24 Board outcomes have included agreement to establish additional engagement mechanisms, including establishing weekly status meetings with senior stakeholders to discuss program performance. The status meetings were originally held weekly between April 2019 and March 2020. As of March 2020 the meetings were held fortnightly.

Project Management Stakeholder Group

4.25 Defence’s August 2018 Acquisition Project Management Plan states that the Project Management Stakeholder Group (PMSG) is to be established, comprising project stakeholders at the one-star/SES Band 1 level, and providing the formal mechanism for stakeholder input to the planning and execution of Land 400 Phase 2. The Program Management Steering Group has not met since the contract was signed in August 2018. Defence advised the ANAO in August 2020 that the first meeting is scheduled for 14 September 2020.

Integrated Project Team

4.26 Defence has established a Land 400 Phase 2 Integrated Project Team to assist the Land 400 Phase 2 Project Director to integrate technical, cost and schedule information and to undertake planning, control and management responsibilities. The Integrated Project Team is comprised of project staff, Army and CASG representatives, with other stakeholders participating as required according to their expertise and area of influence. The first meeting of the Integrated Project Team took place on 21 February 2018, with subsequent meetings on 5 April 2018, 2 May 2018 and 27 February 2020.

Risk management

4.27 The Capability Life Cycle Manual states that: ‘risk management in the Acquisition Phase focuses on risks associated with acquisition, procurement, capability realisation and sustainment, as well as monitoring risk treatment plans.’117

4.28 Defence manages the risks for the project through a project risk log. Defence advised the ANAO in April 2020 that the project risk log was not up to date.

4.29 Defence’s advice to government in March 2018 was that it had assessed the overall risk for Rheinmetall’s Boxer CRV as ‘medium’, including a schedule risk of ‘medium’. The risk assessment documented in the Post Negotiation Evaluation Report of January 2018 assessed the schedule risk as ‘high’. Defence advised the ANAO that the difference in schedule risk is largely attributable to the introduction of the Block I interim operating capability (recorded in the post negotiation evaluation report) which was sufficient under the risk assessment method to reduce schedule risk to ‘medium’ for the purposes of reporting the risk to government.

Contract management

4.30 On 9 August 2018, Defence established a contract, valued at $4.28 billion (including GST), with Rheinmetall Defence Australia for the acquisition of 211 Combat Reconnaissance Vehicles, 12 mission modules and associated support systems. Defence monitors whether contractual requirements are being met by monitoring against delivery milestones and through contract progress meetings.

Delivery milestones

4.31 Rheinmetall is to deliver the CRVs in two tranches, Block I and Block II. Figure 4.1 below provides an overview of the contracted delivery schedule for the various vehicle configurations in Block I and Block II, and the contracted delivery date for training relating to Block I and Block II vehicles.

Figure 4.1: Delivery schedule for Land 400 Phase 2

A figure that outlines the projected delivery schedule of Block I and Block II vehicles, and training from 2019 to 2027.

Source: Defence documentation.

4.32 The manufacture and assembly of the Boxer CRVs will progressively transfer from Germany to Australia.118 As of August 2019, three Block I vehicles had arrived in Australia with each undergoing engineering work.119 On 23 September 2019, Defence accepted one of these vehicles with a list of 69 ‘deviations’ from the conditions and requirements of contract.120 Defence advised the ANAO that it accepted the second and third vehicles on 1 November 2019. In March 2020, Defence advised the ANAO that the remaining Block I vehicles are due between May 2020 and January 2021.121 Defence also provided advice to the ANAO in September 2020 on the impacts of COVID-19 on the project. This information is included in Appendix 5.

4.33 Of the 49 milestones due by the end of June 2020, Rheinmetall had:

  • Delivered 27 milestones on time.
  • Delivered 13 milestones late. Of these 13 milestones, six were met within a month of the due date and seven were more than one month late. Late milestones have included delivery of the:
    • Approved Contract Work Breakdown Structure, which was due by January 2019.122 The contract work breakdown structure was approved by Defence in August 2019 (seven months late).
    • Contract Master Schedule, which was due to be approved by Defence by February 2019.123 In March 2020, Defence advised the ANAO that it approved the Contract Master Schedule on 5 March 2020.
  • Nine outstanding milestones, three of which were less than a month overdue and six of which were more than a month overdue.

4.34 The contract includes a range of provisions to assist Defence to manage delivery against contracted milestones, including ‘stop payment’ milestones.124 The acquisition contract lists 314 milestones against which Rheinmetall will be paid, with 16 of those being ‘stop payment’ milestones.125 Following the approval of Contract Change Proposal 8 in May 2020, there are now 317 milestones under which Rheinmetall will be paid. Three of the stop payment milestones (milestones 31, 41 and 47) were to be achieved as of May 2020. Milestone 31 was achieved late — on 3 September 2019. As a result the Commonwealth enforced the Stop Payment provisions in accordance with the contract. Following the approval of Contract Change Proposal 3 in November 2019, milestones 41 and 47 became due in July 2020126 and February 2021 respectively.127

Contract progress meetings

4.35 Working level Contract Progress Meetings are conducted every three months and are co-chaired by a Commonwealth representative and the Contractor’s Project Manager. Defence and Rheinmetall held six Contract Progress Meetings between November 2018 and June 2020.

4.36 Contract Progress Meetings have discussed issues with sub-contractors, delays in Rheinmetall achieving some milestones, achieving clarity around the quality assurance process for Block I vehicles, issues regarding vehicle weight, concerns with the Contract Master Schedule and Contract Work Breakdown Structure, the schedule for Block I vehicles, and the reliability of the turret design.

4.37 Action items recorded in the minutes include: Defence reviewing milestones; Rheinmetall organising workshops with Defence to assist with Integrated Baseline Review; and Rheinmetall checking that codification data will be available for Block I vehicle platforms. Defence advised the ANAO in July 2020 that:

Issues arising from the CPMs [Contract Progress Meeting] are incorporated into the consolidated Action Item Log (held on Rheinmetall’s JIRA system to which the CoA [Commonwealth of Australia] is being provided access). Each issue is assigned a person responsible for taking action, with a clear objective and due date. All action items are subject to regular review at the working level (primarily during working groups structured around SME [Subject Matter Expert] knowledge domains e.g. engineering, project management, ILS [Integrated Logistics Support], training, scheduling and EV [Earned Value] in order to track progress and record closure upon completion. These actions are also reported upon during the fortnightly management teleconferences held with Rheinmetall and, for those issues deemed significant enough, escalated to the SRB [Strategic Relationship Board].

Contract Change Proposals

4.38 As at 30 June 2020, Defence and Rheinmetall had executed ten Contract Change Proposals since signing the acquisition contract in August 2018. The Contract Change Proposals have effected 175 changes to the contract including one change to the project scope — to implement an integrated electronic architecture for the CRVs128, with an associated increase of $133.9 million in the contract value.129 Defence advised the ANAO in September 2020 that two Contract Change Proposals have effected changes to the payment terms and arrangements, resulting in a reduction of $71.8 million to the contract value.

Project reporting

4.39 The project management plan outlines the regular reporting mechanisms that are expected to support the monitoring and control of the project. These reports are to include:

  • weekly reports to the Head of Land Systems within Defence’s Capability Acquisition and Sustainment Group (CASG);
  • monthly project progress reports through the Monthly Reporting System130 and the Project Performance Review Information Platform to the Deputy Secretary CASG; and
  • reporting to the Project Management Stakeholder Group.

4.40 Defence also reports on project progress through the Quarterly Performance Report, which is distributed to senior stakeholders within government and Defence131 and through ‘deep dive’ reviews. The deep dive reviews are intended to provide senior management within CASG’s Armoured Vehicle Division with a detailed understanding of the status of a project by reviewing the immediate past, the present (next three months) and future (three to six months). The ANAO’s analysis of project reporting on the Land 400 Phase 2 project is set out in Table 4.2 below.

Table 4.2: ANAO analysis of project reporting for Land 400 Phase 2

Report type

Expected frequency

ANAO analysis

Reports to the division head

Weekly

Under the Acquisition Project Management Plan, CASG’s Assistant Secretary Armoured Fighting Vehicles (previously Director General Combined Armed Fighting Systems) must provide weekly reports on the project to the Head of Armoured Vehicles Division (previously Land Systems Division).

Project progress reports to the Deputy Secretary, CASG

Monthly

The ANAO reviewed monthly project reports (including from the Monthly Reporting System (MRS) and the Project Performance Review Information Platform (PPRIP)) for the period December 2019 to May 2020.

Both the MRS and PPRIP reports display most key performance indicators for Land 400 Phase 2 as ‘green’ (capability, Initial Operating Capability, Final Operating Capability, and cost). The reports focus on what has been done, with limited information provided in the report to inform the reader of significant risks and issues experienced by the project — such as those identified through Defence’s Independent Assurance Review process or by the project’s Strategic Relationship Board.

The May 2020 MRS and PPRIP reports both identify two milestones at high risk of delayed achievement — the Interim Initial Material Release for Block I, and the Initial Operational Test and Evaluation for Block Ia

Reports to the Project Management Stakeholder Group

Ad hoc

There are four Project Management Stakeholder Group (PMSG) meetings per calendar year, two of which are full PMSGs, and two are PMSGs (Interim).

The full PMSG includes complete detail in briefings and progress reports. A PMSG (Interim) briefs progress and risk updates, provides information to support decisions.

Quarterly Performance Report (QPR)

Quarterly

Defence first included Land 400 Phase 2 in its Capability Acquisition and Sustainment Quarterly Performance Report (QPR) in September 2019.b In the March 2020 QPR, the traffic light ratings for the project relating to cost, capability and schedule (including expected achievement against Initial Operating Capability and Final Operating Capability) were green.c The associated text noted a ‘slight delay in the delivery of Block I vehicles driven by issues in the global Boxer program’.

As at March 2020, Defence had determined that Land 400 Phase 2 was not a Project of Interest or Concern.d

Reporting on ’deep dive’ reviews

Every 3–6 months

As at December 2019, Defence had conducted three deep dives (March, May and September 2019). These reviews reported a number of key risks for the project, including that: the vehicle weight may exceed the mass limit of 38,500 kg; and changes to the electronic architecture may introduce significant issues in production of the turret.

Supplementary reporting

Ad hoc

In early 2020, Defence provided advice to Defence Ministers and the Deputy Secretary CASG on the ‘significant challenges in delivering the new Combat Reconnaissance Vehicle capability on time’. Advice had also been provided to the Minister for Defence regarding contracting challenges in July 2019.

In January 2020, Defence informed the Defence Minister of ‘ongoing challenges’ experienced by Rheinmetall in delivering the contracted requirements. They relate to Defence’s assessment that Rheinmetall was unlikely to achieve the contracted delivery dates for Boxer vehicles in 2020.

In February 2020, Defence informed the Deputy Secretary CASGe of ‘the challenges facing the Project in the near term with the initial Block I vehicles encountering a significant schedule challenge, and the Block II vehicles facing a number of complex technical challenges’. The February 2020 advice included Defence’s assessment of indicative project performance metrics provided by Rheinmetall, as at November 2019.

     

Note a: The Initial Interim Material Release for Block I will occur when 21 CRV mission systems have been delivered to 7th Brigade, Brisbane. Initial contractor provided logistics support arrangements are in place including: user documentation; technical data; maintenance support, logistics instruction, engineering support; spares; and training systems. The Initial Operational Test and Evaluation for Block I will occur when 21 CRV mission systems have been delivered to the Capability Manager to undertake test and evaluation of the mission system supported by initial Army training and development as an interim deployable capability.

Note b: The stated purpose of the QPR is to provide Government and Department of Defence stakeholders with insight into the delivery of capability to the Australian Defence Force. The QPR includes the Top 30 acquisition projects. ‘Appendix B: Top 30 Acquisition Projects by 2019-20 Forecast Expenditure,’ in Defence Portfolio Budget Statements 2019-20, p. 116, available from http://www.defence.gov.au/Budget/19-20/2019-20_Defence_PBS_00_Complete.pdf [accessed 20 September 2019].

Note c: In the QPR, the Key Acquisition Project Dashboard and Performance Summaries for Key Acquisition Projects use a traffic light system to rate performance. A green rating is acceptable performance. For capability, a green rating is on track to deliver approved scope. For cost, a green rating is on track to deliver within approved budget, and for schedule a green rating is delivery before, on, or up to no more than 14 days after the baseline date.

Note d: Defence established its Projects of Concern regime as a framework to manage the remediation of underperforming materiel acquisition projects. Projects of Concern receive targeted senior management attention and are subject to regular reporting to government. A Project of Interest is one that Defence management considers to be underperforming and in need of senior management attention and close monitoring to prevent it deteriorating to the point of becoming a Project of Concern. See Auditor-General Report No.31 2018–19, Defence’s Management of its Projects of Concern, available from https://www.anao.gov.au/work/performance-audit/defence-management-its-projects-concern [accessed 16 December 2019].

Note e: This advice was provided through a Minute from Defence’s Head Armoured Vehicles to the Deputy Secretary CASG.

Source: ANAO analysis of Defence documentation.

Rheinmetall’s Project Status Reports

4.41 Rheinmetall is required to prepare and deliver Progress Status Reports every three months. These are defined as: ‘the Contractor’s principal statement and explanation of the status of the project at the end of each reporting period’.

4.42 To date, there have been eight Progress Status Reports covering the period August 2018 to August 2020. The ANAO’s review of these reports indicated that they have brought a number of risks and issues to Defence’s attention, including:

  • ongoing vacancies in staff positions, including key specialist roles;
  • possible schedule delays, including for the Block I reconnaissance design and development schedule, due to modification of the turrets and protection requirements132;
  • possible schedule delays for Block II reconnaissance vehicles due to the flow-on impacts of Block I design activities;
  • vehicle gross weight could be exceeded for Block I and Block II vehicles due to protection requirements;
  • risks associated with integration of components, including government furnished equipment133 and Anti-Tank Guided Missiles; and
  • technology transfer and training of skilled personnel may not be finished prior to production starting in Australia.

4.43 As part of the Progress Status Reports, Rheinmetall included risk treatment strategies to address the identified risks. The strategies include: reviewing workforce planning; setting up integrated project teams with workgroups focussed on key risks; and outsourcing some work packages to a third party to accelerate development. As noted in paragraph 4.28, Defence manages risks for the project though a risk log, which was not up to date as of April 2020. In March 2020, Defence advised the ANAO that:

The most recent Rheinmetall risk board confirmed that 38% of current Issues were previously Risks where, using standard risk management terminology, an Issue is a Risk that has been realised …

This is a higher percentage that would normally be expected if the risk mitigation strategies were fully effective. Rheinmetall acknowledge this is an area needing further improvement but it should be acknowledged that the company’s risk management processes have matured significantly since contract signature and continue to improve.

Progress reporting on Australian Industry Capability

4.44 The Australian Government’s 2016 Defence Industry Policy requires companies looking to supply and support capability to Defence, in projects valued at over $20 million, to submit an Australian Industry Capability (AIC) Plan to Defence. The Australian Government expects Defence to enforce any contracted AIC Plan to ensure the benefits are realised.

4.45 The total project expenditure for Land 400 Phase 2 AIC across the acquisition contract is $1.68 billion. The expected AIC expenditure by area as outlined in Rheinmetall’s AIC plan is shown in Table 4.3 below.

Table 4.3: Total projected expenditure for Land 400 Phase 2 Australian Industry Content across the acquisition contract by area

AIC area

Description

Projected expenditure ($ million)

Percentage of projected AIC expenditurea

Material supply

Supply of materials for Boxer CRV through Australian companies.

638

38%

Production and Integration

Creation of full serial Boxer CRV production in Australia

159

9%

Engineering

Design authority and new Intellectual Property for Boxer CRV established and maintained in Australia

136

8%

Establishment of Australian industry and principal business

Transfer of knowledge to establish an Australian sovereign capability

395

23%

Local overheads

Establish a regional hub for manufacturing, support and engineering centred in MILVEHCOE

357

21%

Total 

1685

100%

       

Note a: Percentages of projected AIC expenditure do not add up to 100 due to rounding.

Source: Rheinmetall Australian Industry Capability Plan.

4.46 Figure 4.2 below, supplied by Defence, represents the activities that are expected by Defence to contribute to AIC through the life of the Land 400 Phase 2 acquisition contract.

Figure 4.2: AIC schedule over time

 

Note: H1 is the first half of the year and H2 is the second half of the year.

Source: Rheinmetall, AIC Progress Report, August 2020. Supplied by Defence.

4.47 The Rheinmetall contracted AIC Plan states that:

The AIC Director will, in-concert with the Commonwealth’s representative, establish the AIC reporting process and parameters (KPI [Key Performance Indicator] and SHI [System Health Indicator]) at the commencement of the Program. The AIC Data and Reporting manager will work with Approved Subcontractors and develop the AIC reporting process.

4.48 Defence advised the ANAO in September 2020 that Rheinmetall’s AIC Key Performance Indicators and System Health Indicators are still in the process of being agreed and have not yet been established for reporting purposes.

4.49 The Rheinmetall contracted AIC Plan states that in terms of monitoring and reporting on progress:

The AIC Director will submit a status report to the Commonwealth every six months that details AIC conformance. These reports will highlight, performance, risk, remediation and any specific action plans that will impact both AIC or the Program, including Program level decisions that impact AIC. Any failure to meet AIC KPI [Key Performance Indicators] and SHI [System Health Indicators] will be considered in a quality framework to ensure issues are identified and remediated in a manner that sustains AIC success. AIC will be tracked and reported internally monthly across the entire program.

4.50 Rheinmetall’s Progress Status Reports to Defence (discussed above) include its reported progress against delivery of its AIC Plan. The status report for 1 May 2019 to 31 July 2019 reports total in-Australia expenditure of $72.2 million from the commencement of the contract to 31 July 2019, and states that this expenditure is in line with the agreed to-date estimates.134 The report indicates that $42.7 million (59 per cent) of this amount comprises expenditure on ‘overheads’, including Australian taxes, customs duties, insurances, bank fees, software and supplies, infrastructure, corporate overheads, goods and services.

4.51 Defence advised the ANAO in November 2019 that in addition to this status reporting on AIC, Defence has rights under the contract to:

  • conduct AIC progress reviews at Rheinmetall or sub-contractor premises to assess and verify the implementation of the AIC plans, and the adequacy of a contractor’s financial management information system and data collection methods; and
  • access any Rheinmetall records or accounts relevant to, or impacting on, the performance of work under the contract.

4.52 The ANAO sought advice as to whether Rheinmetall’s progress against the AIC Plan has been assessed and verified by Defence. Defence advised the ANAO in July 2020 that:

Rheinmetall’s progress against its AIC Plan has been reviewed by Defence and reported to senior leaders in both Defence and Government.

4.53 The ANAO also sought advice regarding how assurance is to be obtained on Rheinmetall’s delivery of AIC. Defence advised the ANAO in July 2020 that:

Rheinmetall provides AIC progress reports on a quarterly basis as part of the Project Status Report. Defence assures this data via the application of AASB [Australian Accounting Standards Board] standards (detailed at para. 4.3.2 of the AIC Plan):

  • AIC financial performance data will be subject to the audit and data validation standards of the Australian Accounting Standards Board (AASB).135 For AIC financial performance data that is derived from international origin (for example Germany or USA), the International Accounting Standards Board (IASB) standards will also be used;
  • The AIC Director will coordinate with the Business Administration group to ensure compliance of the AIC Audit and Data validation standards136; and
  • All Sub-contractors will be subject [to] Quality Audit for the AIC Data collection and report obligations of their sub-contracts.137

4.54 Defence further advised the ANAO that as at September 2020, an audit of AIC under the project had yet to be undertaken.

Appendices

Appendix 1 Department of Defence’s response

The response from the Secretary of the Department of Defence and the Chief of Defence Force. A summary of this response is contained in the Summary and Recommendations chapter of this report.

Appendix 2 Defence’s plans for full Australian manufacture

1. Defence provided the following advice to the ANAO in September 2020 regarding its plans for full Australian manufacture of the vehicles:

The first 25 vehicles (Block I) will be initially assembled in Germany as a part of skills and technology transfer activities. This will familiarise Australian companies and workers with the manufacturing techniques and provide experience with the assembly line for these highly complex vehicles. These vehicles will be shipped to Australia for final integration and acceptance testing prior to delivery to the Commonwealth.

After this initial phase, a gradual transition will occur for the assembly of the vehicles (Block II) from Germany to Australia. This will be via a coordinated ramp down in Germany and ramp up in Australia, thereby maximising the effect of technology transfer and reflecting the growing skill base in Australia.

There will remain some vehicle subsystems for which the transfer of manufacture or assembly from Europe to Australia would not be cost effective and they will continue to be supplied from Germany (e.g. welded drive module hulls, 30mm cannons, power packs, etc.). Final assembly, integration, set to work and testing of those elements would, however, still occur in Australia.

Selected low-volume variants will continue to be assembled in Germany.

Based upon what has occurred with the first three vehicles, it is quite possible that even greater levels of major subsystem rework will be transferred from Europe to Australia to help alleviate some of the capacity issues that have been experienced in the German Boxer production facilities due to recent contracts for Boxer vehicles with the United Kingdom, Germany and Lithuania.

Appendix 3 Timeline of key dates and milestones for Land 400 Phase 2

Timeline of key dates and milestones for Land 400 Phase 2

Timeline of key dates and milestones for Land 400 Phase 2

Note: In March 2018, Defence advised government that it would meet the following key milestones for the project: entering into contracts with the provider in May/June 2018; initial vehicle delivery in 2020 (including 13 non-turreted MPVs between Q3 2019 and Q1 2020 and 12 turreted CRVs between Q1 2020 and Q3 2020); Initial Operating Capability in 2022; and Final Operating Capability in 2026–27.

Source: ANAO analysis of Defence documentation.

Appendix 4 Status of Recommendations from July 2017 Defence internal audit

Table A.1: Status of Recommendations from July 2017 Defence internal audit on Land 200 Phase 2 risk management activity

Recommendation

Defence assessment

ANAO assessment and commenta

  1. Audit Branch recommends that the lessons learnt process being established for LAND 400 Phase 2 (by Helmsman) be integrated with the Capability Acquisition and Sustainment Group Lessons Learned Program to ensure that where applicable, lessons can be recorded and shared beyond LAND 400.

Implemented

At the time Defence closed this recommendation, Defence had partially implemented the recommendation.

The evidence provided to support the closure of the recommendation does not demonstrate that Defence had met the intent of the recommendation at that time.

There is no evidence that Defence had integrated the Land 400 Phase 2 lessons learnt process with the CASG Lessons Learned Program at that time.

  1. Audit Branch recommends that the following lessons be captured for LAND 400 Phase 3:
    1. The Risk Mitigation Activity Contract structure proved effective in providing transparency of planned test and evaluation activities to Industry Participants.
    2. Contractual timelines for a Risk Mitigation Activity should seek to reduce risk to the Commonwealth, while also encouraging competition, and minimising the capital cost to industry prior to contract signature.

Implemented

At the time Defence closed this recommendation, it is likely that Defence had implemented the recommendation. However, the evidence provided to support the closure of the recommendation does not demonstrate that Defence had implemented the recommendation at that time.

 

  1. Audit Branch recommends that:
    1. Army, as the responsible Capability Manager under the new Capability Life Cycle, put in place planning (no later than Gate 2) to establish and chair appropriate ongoing program/project governance frameworks.
    2. The LAND 400 Phase 2 Project Management Plan, when updated for Gate 2, fully captures the revised ongoing governance structure.

Not implemented as at October 2017.

Implemented as at 12 January 2018.

 

At the time Defence closed the recommendation in January 2018 it had not implemented the recommendation.

Defence extended the target date from 20 October 2017 to 2 November 2017.

On 30 October 2017, Defence’s Assistant Secretary Audit approved a request from Army to further extend the implementation target date to 1 December 2017.

Defence closed the recommendation on 12 January 2018.

In August 2019, Defence informed the ANAO that there was no closure pack/supporting documentation recorded in Defence’s Audit Recommendations Management System (ARMS) to support the closure of the recommendation.

In January 2020, Defence provided documents from Army to support the closure of the recommendation. Internal audit assessed that the documentation was insufficient evidence to support the closure of the recommendation.

  1. Audit Branch recommends that:
    1. A lesson learnt be captured for LAND 400 Phase 3 that risk management processes, tools, responsibilities, and vertical risk reporting triggers be agreed between the key stakeholders early in the project lifecycle and remain stable between project Gates.
    2. Prior to Gate 2 the processes outlined in the draft Risk Management Plan for triggering vertical risk reporting be reviewed to ensure that all senior risk owners and stakeholders are fully informed of their responsibilities and are updated of any changes to risk status in a timely manner.
    3. Post-Gate 2 an assessment of existing risk management processes / tools be conducted to ensure ongoing risk management arrangements are understood by all stakeholders, and the project is compliant with extant Defence and CASG risk management policy.

Implemented

At the time Defence closed the recommendation, Defence had partially implemented the recommendation.

At the time Defence closed the recommendation, Defence had implemented the minimum requirement of part (a) of the recommendation — Defence had recorded the lesson listed in the audit finding in a lessons learnt spreadsheet.

However, the evidence provided to support the closure of the recommendation does not demonstrate that Defence had implemented the intent of part (a) of the recommendation, or parts (b) and (c) of the recommendation at that time.

 

  1. Audit Branch recommends that the requirements for facilities and related equipment that allow for the comprehensive testing of advanced targeting systems against realistic threat sets be captured as a lesson learnt for LAND 400 Phase 3.

Implemented

At the time Defence closed the recommendation, Defence had implemented the minimum requirement of the recommendation — Defence had recorded the lesson listed in the audit finding in a lessons learnt spreadsheet.

However, the purpose of capturing the lessons was to ensure that they informed Land 400 Phase 3. The evidence provided to support the closure of the recommendation does not contain any evidence or assertion that this had occurred, or would occur.

  1. Audit Branch recommends that the following lessons be captured for LAND 400 Phase 3:
    1. That the construct of the Army Trials Troop has been effective in supporting the Risk Mitigation Activity.
    2. That the Gate 2 baseline capability and preferred vehicle sustainment requirements should be agreed as soon as practicable after the commencement of a Risk Mitigation Activity.
    3. That practical test serial arrangements be agreed with all stakeholders as early as practicable to identify and address any tactical level issues prior to activity commencement.
    4. That consistent and practical probity implementation guidance for the Risk Mitigation Activity should be included either as an annex to the probity plan, or in the trial directive.

Implemented

At the time Defence closed the recommendation, Defence had implemented the minimum requirement of the recommendation — Defence had recorded the four lessons listed in the audit finding in a lessons learnt spreadsheet.

However, the purpose of capturing the lessons was to ensure that they informed Land 400 Phase 3. The evidence provided to support the closure of the recommendation does not contain any evidence or assertion that this had occurred, or would occur.

     

Note a: Defence advised the ANAO in September 2020 that: ‘the Land 400 Phase 3 project is managed by the same AVD executive team as Land 400 Phase 2 and that a number of the senior SME’s (ILS, engineering etc.) have responsibilities across both Phases 2 and 3. Further, a core element of the Phase 3 team was intentionally built with former Phase 2 team members in order to ensure an effective transfer of corporate knowledge and lessons learned. This was a conscious strategy to leverage the experience gained and the lessons that had been learnt on Land 400 Phase 2.’

Source: ANAO review of Department of Defence information.

Appendix 5 Defence’s advice on the impact of COVID-19 on Land 400 Phase 2

1. In September 2020 Defence provided the ANAO with the following advice on the impact of COVID-19 on Land 400 Phase 2:

Defence advises that Rheinmetall submitted their draft Recovery Plan, in accordance with the requirements of the COVID-19 Recovery Deed, on 30 July 2020.

In summary, Rheinmetall has advised that the initial impact assessment of the overall program delay realised from the impacts of the COVID-19 Event is a minimum of 2 calendar months, noting not all delays are equal across the program. Delays have occurred across all Block I and Block II mission system variant delivery dates and milestone dates, including Support System delivery dates for Training, Training Equipment, and Support and Test Equipment and the related milestone dates, and dates for the conduct of Mandated System Reviews.

Rheinmetall asserts it has quantified the impacts resulting from the COVID-19 Event, and these delays include nine Block I Boxer CRV Reconnaissance variants being delivered in 2021, rather than in 2020 as planned.

Rheinmetall confirms it remains committed to delivering a deployable Block I Boxer CRV capability by December 2021. To mitigate the impacts of the COVID Event on Block I, Rheinmetall has reworked the delivery plan, so that by the end of 2020, it will have:

  • delivered all Block I Boxer MPV variants;
  • completed all Reconnaissance Block I Drive Modules and shipped at least ten to Australia (two will be retained in Germany until the completion of turret certification testing);
  • completed all Reconnaissance Block I Mission Modules and shipped at least ten to Australia (two will be retained in Germany until the completion of turret certification testing); and
  • completed all Block I Turrets in Germany.

Rheinmetall is forecasting similar delays across all Block II Boxer CRV variants and the Training Equipment. It is currently advising of a delay to the Block II Boxer CRV Reconnaissance Detailed Design Review from 30 November 2020 to 03 February 2021, with an associated delay of at least 2 months to all subsequent related Contract milestone dates and delivery dates, including the Support System.

Defence advises the delays identified by Rheinmetall are an initial assessment of the impact of the COVID Event, and further detailed analysis by Rheinmetall is on-going. Defence also notes the delays due to the COVID Event are in addition to the delays previously notified by Rheinmetall in February 2020.

Footnotes

1 Department of Defence, Land 400 Phase 2 Flyer, p. 2, available from http://sussanley.com/wp-content/uploads/2015/08/land_400_phase_2_flyer_-_web-9-7895.pdf [accessed 4 August 2020].

2 Land 400 Phase 2 Acquisition and Support Implementation Strategy version 2.0, p. 8.

3 Under the contract, the Statement of Work notes that ‘The Contractor and the Commonwealth agree to undertake a collaborative approach, with shared responsibilities, to achieving an early deployable capability for the ADF through the introduction of an interim ‘Block I’ capability’.

4 Defined by Defence as: ‘A system or equipment that is available for purchase, which is already established in-service with another military or government body or commercial enterprise and requires only minor, if any, modification to deliver interoperability with existing ADF assets.’ See Auditor-General Report No. 19 2019–20 2018–19 Major Projects Report, p. 121 and footnote 27.

5 These phases are described in Table 1.1 of this audit report.

6 ANAO comment: this was one of a number of management initiated reviews relating to aspects of LAND 400 Phase 2. As discussed in paragraphs 10 and 15 above, two other Defence reviews and an internal audit identified that Defence’s approach to the market, and the cost of tendering and schedule constraints, might have caused some potential bidders to not participate in the tender — potentially reducing competition and limiting the options available to the Australian Government.

7 Department of Defence, Land 400 Phase 2 Flyer, p. 2, available from http://sussanley.com/wp-content/uploads/2015/08/land_400_phase_2_flyer_-_web-9-7895.pdf [accessed 4 August 2020].

8 In the Defence Industrial Capability Plan, Defence defines sovereignty as the independent ability to employ Defence capability or force when and where required to produce the desired military effect. Australian defence industrial capability is the capability provided by Australian industry that contributes directly to the delivery of a defence capability. It becomes a sovereign industrial capability when Australia assesses it to be strategically critical and must therefore have access to, or control over, the essential skills, technology, intellectual property, financial resources and infrastructure as and when required. The Defence Industrial Capability Plan identifies 10 initial Sovereign Industrial Capability Priorities, including ‘Land combat vehicle and technology upgrade.‘ Defence Industrial Capability Plan 2018, p. 20 and p.37, available from http://www.defence.gov.au/SPI/Industry/CapabilityPlan/Docs/DefenceIndustrialCapabilityPlan-web.pdf [accessed 20 June 2019].

9 ibid., p. 37.

10 Australian Army website, Project LAND 400, available from https://www.army.gov.au/our-future/modernisation-projects/project-land-400 [accessed 5 August 2019]. The information on this webpage was last updated on 21 December 2016.

11 Land 400 Phase 2 Acquisition and Support Implementation Strategy version 2.0, p. 8.

12 The Request for Tender (RFT) documentation defined a ‘Military Off The Shelf (MOTS) Plus’ vehicle as a MOTS vehicle that is: ‘…reconfigured with a package of options or upgrades that enhance or improve the compliance of the vehicle with the technical, functional and performance requirements of this RFT.’

13 Defence advised the ANAO that the Armoured Vehicle Division was established on 21 January 2019. From 1 July 2010 until 21 January 2019, Land 400 was managed by the Combined Arms Fighting Systems Branch within Land Systems Division.

14 The Agreement established between Army and CASG is discussed in Chapter 4 of this audit report.

15 Defence, Defence White Paper 2009, Defence White Paper 2013 and Defence White Paper 2016, available from http://www.defence.gov.au/whitepaper [accessed 8 August 2019].

16 At ‘first pass’, options under consideration are narrowed and funding is approved for various activities, primarily cost and risk analysis.

17 At ‘second pass’, government endorses a specific capability solution and approves the funding required for the acquisition phase.

18 Chapter 2 of this report discusses stage 1. Stages 2 and 3 are discussed in Chapter 3.

19 IOC is the capability state relating to the in-service realisation of the first subset of a capability system that can be employed operationally. Declaration of initial operating capability is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group(s) that the fundamental inputs to capability have been delivered. See Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report, p.viii.

20 FOC is the capability state relating to the in-service realisation of the final subset of a capability system that can be employed operationally. Declaration of final operating capability is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group(s) that the fundamental inputs to capability have been delivered. See Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report, p.viii.

21 A timeline of key dates for the project from first pass approval in December 2014 to the expected date for the achievement of FOC in June 2027 is provided at Appendix 3 of this audit report.

22 Under the contract, the Statement of Work notes that ‘The Contractor and the Commonwealth agree to undertake a collaborative approach, with shared responsibilities, to achieving an early deployable capability for the ADF through the introduction of an interim ‘Block I’ capability’.

23 Construction of the facility commenced in November 2018 with completion expected in June 2020. Defence informed the ANAO in July 2020 that the facility is operational and in use for final assembly, integration and rework of the Block I vehicles.

24 Appendix 2 sets out Defence’s advice to the ANAO on its plans for full Australian manufacture of the vehicles.

25 M.Payne, [Minister for Defence], available from https://www.minister.defence.gov.au/minister/marise-payne/media-releases/joint-media-release-prime-minister-minister-defence-industry, ‘Joint media release — Prime Minister, Minister for Defence Industry and Minister for Defence - New Vehicles to protect our troops and create 1450 jobs’, media release, 14 March 2018.

26 Department of Defence, Land Combat Vehicle System Overview, available from https://www.defence.gov.au/CASG/EquippingDefence/Land%20400.asp [accessed 20 October 2019].

27 Australian industry content for Land 400 Phase 2 refers to the estimated investment expected to occur within Australia, as opposed to investment that occurs overseas and imported into Australia. Publicly available Australian Industrial Capability Plans are listed on Defence’s website, available from https://www.defence.gov.au/spi/Industry/PublicPlans/Public-AIC-Plan-LAND-400-Phase-2-Acquisition-Rheinmetall.pdf [accessed 25 February 2020].

28 The evaluation of the tenders is discussed in Chapter 3.

29 In 2006, Defence had not yet established specific requirements for Land 400 but the project scope encompassed replacing the capabilities provided by Defence’s ASLAV, M113 and Bushmaster vehicle fleets that directly participate in mounted close combat.

30 Defence’s RPDE Program ran from 2005 to 2016 as a collaborative arrangement between Defence, industry and academia. RPDE ‘Quicklooks’ were intended to provide guidance, advice and input on Defence issues by rapidly bringing together experts from industry and academia. The ‘Quicklook’ team engaged industry, conducted a workshop, approached state governments and relevant industry bodies, such as Engineers Australia, and reviewed the available literature to form a position on Australian industry’s ability to support Land 400.

31 Amongst the assumptions underpinning the Land 400 Acquisition Strategy Options Study was that the: ‘Land 400 vehicles will predominantly be fielded as a Commercial-off-the-Shelf (COTS)/Military-off-the-Shelf (MOTS) capability’. Land 400 Acquisition Options Study, June 2010, p. 11.

32 Defence advised the ANAO in October 2019 that challenges around project scope and cost — specifically balancing Army’s capability aspirations with affordability — limited progress on the project between 2006 and 2010.

33 The review reflected the April 2011 advice of the Defence Materiel Organisation, that upon first pass approval, the Land 400 project would release a request for tender inviting industry to tender for the entire Land 400 LCVS capability. That is, all vehicle types not just the combat reconnaissance vehicle.

34 An updated Capability Life Cycle Manual was issued in January 2020, replacing an interim manual first issued in 2016 and updated in 2017. The 2014 manual was the version applicable at the time of the tender process.

35 Defence Science and Technology Organisation, Project Land 400 Land Combat Vehicle System First Pass Technical Risk Assessment, Version 1.1, 8 July 2013.

36 The purpose of this Gate Review was to consider the readiness of the project for progression to two senior Defence Committees responsible for Defence capability acquisitions, and then to government for first pass approval. Gate Reviews were internal assurance reviews of Defence acquisition projects that included elements of independence such as the presence of external board members. Gate Reviews are now known as Independent Assurance Reviews.

37 In October 2014 a further Gate Review provided the following advice: ‘The Function and Performance Specification (FPS) for each role contains hundreds of requirements, mostly classified as ‘important’, and there are no essentials (which the Board supported) … The Board suggested that work be done in collaboration with CM [the Capability Manager] and CDG [Capability Development Group] to determine which requirements (e.g. blast survivability) were particularly important. These should be classified as ‘very important’ for clarity in the FPS and then rolled up into the KRM [Key Requirements Matrix]’. Defence advised the ANAO in September 2020 that ‘very important’ was a new criteria priority designation and was one with which industry was not familiar. Defence further advised that industry treated the ‘very important’ requirements as being effectively equivalent to ‘essential’ requirements, despite the term being defined in the Conditions of Tender. The October 2014 Gate Review is discussed further in paragraphs 2.25–2.28.

38 Governance of the project is discussed in Chapters 3 and 4 of this audit report.

39 Defence advised the ANAO in July 2020 that these 10 briefings were in addition to numerous verbal briefings provided by the Chief of Army, the Secretary of Defence and the Chief of the Defence Force to the Minister. Defence could not provide the ANAO with records of these verbal briefings.

40 It was at this point that Defence disaggregated Land 400 into individual projects (phases) — Phase 2 ASLAV replacement, Phase 3 M113 replacement, and Phase 4 Integrated Training System. Government approved this approach in December 2014 as part of the first pass approval for Land 400 Phase 2.

41 The First Principles Review: Creating One Defence recommended that the Defence Materiel Organisation be disbanded and its core responsibilities in relation to capability delivery be transferred to a new Capability Acquisition and Sustainment Group. The transfer occurred on 1 July 2015. Defence, First Principles Review, p. 9, available from https://www.defence.gov.au/Publications/Reviews/Firstprinciples/Docs/FirstPrinciplesReviewB.pdf [accessed 29 July 2020].

42 In an attachment to the submission Defence identifies 10 vehicles that Defence had assessed as suitable candidates for Land 400 Phase 2.

43 Underlining is as per the annotation made on the document by the DMO Chief Executive Officer.

Previous Auditor-General performance audits of Defence have also reported on shortcomings in advice to the Minister, including Auditor-General Report No.6 2013–14 Capability Development Reform, paragraphs 12.34–12.37.

The risks and consequences of identifying projects as MOTs when they have a high developmental component were discussed in Auditor-General Report No.36 2005–06 Management of the Tiger Armed Reconnaissance Helicopter Project — AIR 87, p. 16; and Auditor-General Report No.11 2016–17 Tiger — Army’s Armed Reconnaissance Helicopter, p. 2.

44 Defence advised the ANAO in July 2020 that the:

CEO DMO had been made aware of this issue as part of the ASIS [Acquisition and Support Implementation Strategy] development and clearance process. The version of the ASIS he had reviewed and commented on only two months prior (14 Sep 14) included the following paras:

  • 48. A comparison of the available MOTS products against the CRV FPS suite suggests that whilst each of the FPS requirements can be achieved by one or more of the available solutions, there may not be a single solution that is compliant with all of the FPS requirements.
  • 51. Many of the available materiel solutions that could achieve a high level of compliance against the FPS have continued to be further developed by the Original Equipment Manufacturer (OEM). This may allow for MOTS vehicles to be offered that already have a number of mature capability options available for inclusion to achieve an even greater level of compliance with the FPS, but with substantially less technical risk than would be associated with a new development.’

45 Of the 15 action items arising from the gate review, seven had been marked as complete, one was being finalised, six were ongoing and one was deemed no longer applicable.

46 Previously the CEO of the Defence Materiel Organisation, now the Deputy Secretary, Capability Acquisition and Sustainment Group (CASG).

47 A consequence of not addressing risks and issues identified through these reviews was identified in Auditor-General Report No. 40 2018–19 Modernising Army Command and Control — the Land 200 Program, which concluded that Defence management’s failure to implement the same recommendation from a series of independent assurance reviews represented a failure of governance.

48 Defence’s current independent assurance review instruction, approved on 2 August 2016 by the Deputy Secretary CASG, states that: ‘Independent Assurance Reviews are focused on planning and managing for success. They provide the DEPSEC assurance that projects and products will deliver approved objectives and are prepared to progress to the next stage of activity’ (‘Objective’, paragraph 6). The instruction further states that: ‘Over the life of each Defence capability, DEPSEC CASG is responsible to the capability managers, the Defence Executive and ultimately to Government to provide assurance of capability delivery and sustainment. Independent Assurance Reviews are an integral and effective corporate level process, conducted by senior management, to enable this responsibility to be fulfilled (‘Principle 1 - Purpose’, paragraph 9).

In September 2020 Defence advised the ANAO that ‘independent assurance review recommendations do not take effect until accepted or agreed by the responsible line manager, and that the DEPSEC [Deputy Secretary CASG] is the person who agrees or accepts the recommendations of the IARs for the major projects.’

49 The 2014 interim manual was the version applicable at the time.

50 During the Risk Mitigation Activity (stage 2 of the evaluation process), tenderers were only able to integrate very little mandated Australian government furnished equipment, resulting in minimal evidence of the performance of this equipment in the tendered vehicles. As a result, some of the high risk aspects of the procurement were unable to be tested during the Risk Mitigation Activity (see paragraph 3.38).

51 The risk rating was undertaken by DSTO. Defence defines technical risks as risks of the project not achieving its objectives due to uncertainties relating to technology immaturity and/or systems immaturity. In Defence’s technical risk assessments, technology immaturity describes the situation that a project may not achieve its objectives because of an underpinning technology not maturing in the required timeframe. Systems immaturity characterises the circumstance that the project may not achieve its objectives due to uncertainties that arise in the integration of critical technologies and/or sub-systems dependent on them, or technology-related uncertainties associated with the integration of the system into the ADF.

52 For the Request for Tender process, any reference to the Defence Procurement Policy Manual relates to the October 2014 version of the manual.

53 The additional requirements under Division 2 of the Commonwealth Procurement Rules (CPRs) do not apply to this procurement. In accordance with Rule 2.6 of the CPRs, the Secretary, as the Accountable Authority for Defence, has pre-determined that procurements for Ground Effect Vehicles, Motor Vehicles, Trailers and Cycles (United States Federal Supply Code 23) are exempt from Division 2 requirements.

54 Defence documents used in conjunction with the Defence Procurement Policy Manual were the Defence and Defence Materiel Organisation Accountable Authority Instructions and the ASDEFCON suite of tendering and contracting templates.

55 Additional requirements around risk management during procurement are included in the Defence Procurement Policy Manual 2014 and the Project Risk Management Manual.

56 Defence released 24 addendums covering additions and retractions of information in the original Request for Tender as well as responses to potential tenderers’ questions.

57 Draft documentation released included: declassified preliminary operational concept document, key requirements matrix, government furnished material list, indicative communications fit list, and draft solicitation key discriminators.

58 A copy of the proposed request for tender was not included in the advice. Defence informed the ANAO in September 2020 that prior to First Pass approval, the Request for Tender documentation had not yet been developed so could not have been provided to the Minister.

59 In October 2014, the gate review board noted that: ‘No prospective tenderer currently has a MOTS vehicle or vehicles that can meet the full ADF’s requirements in all roles’. The project office had informed the gate review that: ‘A comparison of the available MOTS products against the CRV FPS [Function and Performance Specification] suite suggests that whilst each of the FPS requirements can be achieved by one or more of the available solutions, there may not be a single solution that is compliant with all of the FPS requirements’. See paragraph 2.27 above.

60 The full disclaimer in the documents released to industry in 2014 was: ‘The Commonwealth is releasing this draft document in advance of a possible industry solicitation activity for the Acquisition and Support of Combat Reconnaissance Vehicles as part of the LAND 400 Phase 2 Project. The provision of this information is intended to enable potential tenderers to better prepare themselves should any such solicitation activity eventuate: Potential tenderers should note that this information is provided as advice only and remains subject to change until the publication of any formal industry solicitation documentation. The Commonwealth does not guarantee, and accepts no legal liability whatsoever arising from or connected to, the accuracy, reliability, currency or completeness of any information contained within this document, and this information does not form an offer capable of acceptance. The release of this draft documentation is expressly not intended to be the commencement of a solicitation activity, nor should it be construed as any form of commitment by the Commonwealth that any future solicitation activity will result.’

61 As discussed in paragraph 2.33, a definition of ‘MOTS Plus’ was not provided to government to inform ministers’ consideration of first pass approval.

62 Defence informed the ANAO that the mission profiles were an inadvertent omission from the original tender release.

63 Defence has no record of the discussions at this meeting.

64 As discussed in paragraph 2.27, the Land 400 project team had advised an October 2014 Gate Review Board that ‘no prospective tenderer currently has a MOTS vehicle or vehicles that can meet the full ADF’s requirements in all roles’. Defence advised the ANAO in September 2020 that: ‘there was no requirement to fully meet Defence’s requirements and, as is now known, none of the MOTS Plus offers “fully met Defence’s requirements” either.’

65 The First Principles Review: Creating One Defence, was commissioned by the Minister for Defence in August 2014, recommended that Defence establish a single end-to-end capability development function and that this would require, among other changes, ‘Moving to a leaner “smart buyer” model that better leverages industry, is more commercially oriented and delivers value for money’. Defence, First Principles Review, pp. 32–3, available from https://www.defence.gov.au/Publications/Reviews/Firstprinciples/Docs/FirstPrinciplesReviewB.pdf [accessed 29 July 2020]. Defence has stated that ‘Central to Smart Buyer is a decision-making framework which enables Capability Managers and project teams to identify and analyse key project risks and drivers.’ Secretary’s foreword to the 2016–17 Defence Major Projects Report, in Auditor-General Report No. 26 2017–18, 2016–17 Major Projects Report, p. 67.

66 The July 2016 review examined stage one of Defence’s evaluation process. It considered Defence’s management of the request for tender, initial evaluation and shortlisting for Land 400 Phase 2. It concluded that Defence had developed ‘an integrated suite of [Request for Tender] documents that was consistent with policy and guidance and included input from a diverse range of stakeholders’. It also found shortcomings in the quality, management and control of project documents, including accountability for their accuracy and timeliness.

67 ANAO comment: the review indicated that the high cost of tendering was largely driven by the cost of constructing test vehicles for the risk mitigation activity. Further, tenderers had 11 weeks from signing the risk mitigation activity contract to deliver vehicles suitable for the risk mitigation activity (some of which required significant customisation to the MOTS variants). The short timeframe meant that potential suppliers of ‘MOTS Plus’ offers needed to begin building their vehicles prior to knowing if they had been selected to participate in the risk mitigation activity.

68 During this audit the ANAO made several requests for reviews and specifically requested reviews to identify lessons learned. Defence’s responses to these requests did not include this particular review and advised that the 2016 review was the relevant lessons learned document. In July 2020 the ANAO was provided with a copy of the executive summary from a specially prepared summary of the full 2019 review report, embedded as an attachment to a minute from the Head of Armoured Vehicle Division to the CASG Lessons Learned Program. In response to the proposed audit report issued to Defence under section 19 of the Auditor-General Act 1997 towards the end of the audit process, the ANAO was provided with a specially prepared summary of the 2019 review on 7 September 2020. A full copy of the 2019 review was only provided to the ANAO (on 16 September 2020) after it was specifically requested.

69 Defence informed the ANAO in July 2020 that some of the delay was to increase the focus on AIC which increased the level of AIC in both tenderers offers.

70 The initial screening criteria were: (1) the tenderer complies with clause 2.2 of the Condition of Tender (language and measurements of tender); (2) the tenderer is a single legal entity; (3) the tenderer includes an executed tenderer’s deed of undertaking; and (4) the tenderer offers a vehicle that satisfies the definition of MOTS in the Conditions of Tender, and provides a copy of an executed contract or other documentary evidence for that vehicle in the configuration offered.

71 Defence advised the ANAO in September 2020 that: ‘The submission of a baseline MOTS offer allowed for a direct cost/risk benefit comparison to be undertaken with the more capable, but higher risk and higher cost, MOTS Plus offer (if submitted).’

72 Defence’s procurement guidance states that not assigning weightings to tender evaluation criteria allows Defence officials to qualitatively (rather than quantitatively) evaluate tenders on a balance of all the stated criteria.

73 Defence’s evaluation was against its specific criteria relating to ADF and Australian Government requirements. The outcome of tender processes conducted by other potential purchasers of the tendered solutions, against different criteria, may well differ from the outcomes of this Defence process.

74 The assessment matrix was based on the following definitions:

  • Compliance with the requirements of the RFT documentation (Exceeds, Compliant and Deficient, with Deficient being further classified as Deficient critical (Dc), Deficient significant (Ds), and Deficient minor (Dm)).
  • Confidence — level of certainty (or uncertainty) that can be applied to the evidence provided by the Tenderer to support claims of compliance and rated as High, Medium, or Low.
  • Risk — likelihood of an inappropriate outcome and its consequence or impact on — cost, schedule, performance, supportability, environment, reputation, compliance and security (Extreme, High, Medium, or Low).

75 The Initial Source Evaluation Report for Land 400 Phase 2, recommended by Defence’s Tender Evaluation Board (TEB), and approved by Defence’s Tender Evaluation Steering Group in March 2016, stated that: ‘The TEB was not satisfied with the individual tender and comparative analysis performed against EC10 [evaluation criteria 10]. In its view, the Tenderers misinterpreted the primary intent of this EC [evaluation criterion] which was to identify the proposed growth path to mitigate the risk of technical obsolescence over the vehicle’s 30 year [life]. Tenderers focussed instead on detailing their plan solely to achieve compliance with the current set of technical requirements. The TEB assesses that all tenders are non-compliant against this criterion’.

76 This review was completed on 30 June 2016. The announcement of the successful tenderer was further delayed by Defence until Ministers were sworn in following the federal election of 2 July 2016.

77 Where schedule or cost constraints limited the extent of testing undertaken during this stage, those residual risks were carried over to post second pass approval. The tender evaluation plan noted that delaying the mitigation of identified technical risks until after contract signature was likely to incur additional cost and schedule premiums.

78 The outcomes of the risk mitigation activity and the outcomes from the stage three assessment were reported to the Tender Evaluation Steering Group in a single report.

79 Defence advice to the ANAO, November 2019.

80 Defence advised the ANAO in September 2020 that: ‘An up-front investment to better define requirements and understand capabilities prior to signing a contract is an effective means of mitigating future cost and schedule risk. Kinnaird’s 2003 Defence Procurement Review suggested that, for complex projects, the proportion of investment to total project funds may be in the order of 10% to 15% (page 17). The Land 400 Phase 2 pre-contract expenditure falls well below this.’

81 Defence, Audit Task Number: 17-011, Final Report, LAND 400 Phase 2: Risk Mitigation Activity, July 2017.

82 See Appendix 4 for the recommendations and the ANAO’s assessment of their implementation.

83 After Action Reviews are an Australian Army term for lessons learned reviews. They are conducted on an as required basis. Defence informed the ANAO in July 2020 that ‘holistic’ refers to using any relevant observations or lessons identified from the activity to assist in the design, development, execution and reporting of future land trials within the Land Test and Evaluation Agency.

84 These included: the Proof and Experimental Establishment Graytown (use of their specialised facilities for survivability testing); Land 200 (supply of government furnished equipment for integration assessments); the Directorate of Operations and Training Area Management (use of training areas around the country); the Land Explosive Ordnance Systems Program Office (use of ammunition not in-service); and the Export Control Office (for Third Party Equipment Transfer within Integration).

85 Defence advised the ANAO in July 2020 that all action items resulting from the 2017 After Action Review have been implemented, and have been used to inform the Risk Mitigation Activity being conducted by LAND 400 Phase 3.

86 Defence also advised the ANAO that as per the Conditions of Tender (paragraphs 5.6.7 and 5.6.8) all tenderers were aware of the Commonwealth’s intention to allow for the optimisation of tenders at the conclusion of the Risk Mitigation Activity, and that revised pricing could be considered as part of the final detailed evaluation. Defence further advised that whilst the Australian Government Solicitor (AGS) was not explicitly asked to provide formal advice on paragraphs 5.6.7 and 5.6.8 of the Conditions of Tender, it was discussed as part of the development of the evaluation process and during the evaluation itself, and that AGS provided formal review and advice on the draft Conditions of Tender and Request for Tender documentation suites. Defence advised that during these reviews AGS did not raise concerns about the Commonwealth’s intention to allow tenderers to submit revised pricing following the Risk Mitigation Activity.

87 Defence informed the ANAO in July 2020 that all the evaluation criteria were fully assessed and formed part of the value for money assessment. However, where the results between the tenders were the same or very close, their contribution to differentiation between the tenders, for the purpose of determining the preferred tender, was minimal. In these cases, both tenderers were assessed as providing the same level of ‘value’ to the Commonwealth, and therefore those evaluation criteria were not considered to be discriminators on the ‘value’ side of the value for money equation.

88 Defence’s report on the outcomes of the parallel negotiations states that Defence developed this course of action in consultation with the Australian Government Solicitor (the probity advisor).

89 The Parallel Negotiation Evaluation Report states that ‘this report does not consider affordability aspects of the two tenders as this is the domain of the Capability Manager’.

90 The Tender Evaluation Instruction for stage three of the tender process noted that: ‘the Affordability Assessment will be completed after the Tender Ranking has been finalised because it is not to be considered a discriminator and does not influence the tender evaluation process.’

91 Defence informed the ANAO in April 2020 that its senior leadership was notified of these unresolved issues through the Source Evaluation Report outcomes.

92 In 2014, Army had endorsed an order of priority for Consolidated Operational Needs for the overall capability of the vehicle, which were incorporated into the assessment of evaluation criterion 3. Consolidated Operational Need 1 was ‘Protection’ and was to be given a weighting of 31 per cent.

93 The assessment of technical risk was drawn from the January 2018 Technical Risk Assessment conducted by the Defence Science and Technology Group.

94 The Land 400 Legal Process and Probity Plan did not require the probity advisor to review these things.

95 Defence’s framework for managing conflicts of interest during the procurement process included Defence Instruction (General) PERS 25-6 Conflicts of Interest and Acceptance of Offers and Gifts and Hospitality, which was replaced by Defence Instruction (General) PERS 25-6 Conflicts of Interest and Declarations of Interest. This policy was superseded by Defence Instruction Administrative Policy in August 2019.

96 Three versions of this plan for Land 400 were developed during the course of the tender and evaluation: 2012–2015, 2015–2016, 2016–2019, and 2019.

97 The individual concerned was also responsible for approving the functional performance specifications for the proposed combat reconnaissance vehicles.

98 This required a change to Army’s chain of command for the project, which Defence implemented in February 2017.

99 2015 Land 400 Legal Process and Probity Plan, p. 7. The plan also notes that Defence personnel must comply with the following Defence Instructions (General): DI(G) PERS 25-7 Gifts, Hospitality and Sponsorship and DI(G) PERS 25-6 Conflicts of Interest and Declarations of Interest.

100 The dinner took place on 12 July 2017.

101 The relevant part of the manual is section 1.47(b), p. 20. An updated Capability Life Cycle Manual was issued in January 2020, replacing an interim manual first issued in 2016 and updated in August 2017. This section focuses on the requirements of the 2017 manual, which applied at second pass in March 2018 and when Defence entered into the acquisition contract with Rheinmetall Defence Australia (August 2018).

102 Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report. See also Auditor-General Report No.6 2013–14 Capability Development Reform, which discussed Defence’s rationale for, and progress with, implementing Project Directives.

103 Defence informed the ANAO in January 2020 that the government’s March 2018 second pass approval is referenced in the following governance documents: the Project Management Plan, the Materiel Acquisition Agreement and the Integrated Investment Program Major Equipment Capital 2017–18 Funding Transfer Minute.

104 The Interim Capability Life Cycle Manual 2017 (the version in place during the procurement process and up until January 2020) stated that: ‘The Secretary of the Department of Defence (Secretary) and the Chief of the Defence Force (CDF) expect Defence personnel to comply with provisions in manuals unless the particular circumstances warrant departure from the provisions.’ Interim Capability Life Cycle Manual, August 2017, p. 5.

105 Recommendation 3(b) of the internal audit was that ‘the Land 400 Phase 2 Project Management Plan, when updated for Gate 2, fully captures the revised ongoing governance structure’.

106 Defence’s internal audit function assessed that the documentation provided by Army to support the closure of the recommendation was insufficient. See Appendix 4 of this audit report.

107 See Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report, p. 21.

108 IOC is the capability state relating to the in-service realisation of the first subset of a capability system that can be employed operationally. Declaration of initial operating capability is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group(s) that the fundamental inputs to capability have been delivered. See Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report, p. viii. IOC is also referred to as initial operational capability.

109 FOC is the capability state relating to the in-service realisation of the final subset of a capability system that can be employed operationally. Declaration of final operating capability is made by the Capability Manager, supported by the results of operational test and evaluation and declaration by the Delivery Group(s) that the fundamental inputs to capability have been delivered. See Auditor-General Report No.19 2019–20, 2018–19 Major Projects Report, p. viii. FOC is also referred to as final operational capability.

110 In the MAA, Defence has defined the achievement of IOC as when the initial scope of Land 400 Phase 2, including mission, support and training systems, and facilities, if required, have been delivered to one Combat Brigade and support organisations and accepted into operational service.

111 ANAO comment: following receipt of this Defence advice late in the audit, the Auditor-General decided to not include a proposed recommendation on this matter in this audit report.

112 Block I and II vehicles were discussed in paragraphs 1.12–1.13 of this audit report.

113 CASG’s project maturity score quantifies the maturity of a project. See Auditor-General Report No. 19 2019–20, 2018–19 Major Projects Report, pp. 114–16.

114 Defence projects are graded into one of four acquisition categories (ACATs), reflecting project complexity. See Auditor-General Report No. 19 2019–20, 2018–19 Major Projects Report, pp. 112–13.

115 The Board is co-chaired by the Head of Rheinmetall Vehicle Systems Division and the Senior Commonwealth Representative (currently Head Armoured Vehicles, Defence).

116 The next Strategic Relationship Board meeting was 9 September 2020.

117 August 2017 Updated Interim Capability Life Cycle Manual, p. 41.

118 As noted in paragraph 1.14, during production of the 186 Block II vehicles, manufacture and assembly will transition from Germany to a purpose built facility known as the Military Vehicle Centre of Excellence (MILVEHCOE) in Queensland. Defence anticipates that full Australian manufacture at the MILVEHCOE will be achieved with the 31st Block II vehicle, which is currently scheduled to start production in May 2023. Defence’s advice to the ANAO regarding full Australian manufacture is include in Appendix 2.

119 Defence informed the ANAO in September 2020 that the engineering work undertaken was minor manufacturing assembly (all pre-fabricated parts) and the integration of Government Furnished Equipment.

120 For example, the deviations listed in the acceptance certificate include missing brackets, housing and webbing; deviations with interior power outlets and rear stowage rack.

121 Defence informed the ANAO in October 2020 that it had taken delivery of six vehicles to date.

122 The Contract Work Breakdown Structure (CWBS) forms the framework for contract planning, management and status reporting and for estimating costs, schedule and technical achievements at completion.

123 The Contract Master Schedule describes the contractor’s planned sequence of activities, milestones and decision points necessary to meet the objectives of the contract.

124 There are also provisions relating to liquidated damages, withholding payments and contract termination.

125 Stop payment milestones allow the Commonwealth to withhold payments from the contractor for failure to achieve nominated milestones or delivery dates, or failure to meet other agreed contractual provisions.

126 Defence informed the ANAO in September 2020 that milestone 41 involved a trial to verify elements of the proposed design for the Boxer CRV Recovery variant. Defence further advised that during the initial set-up for the trial, an equipment failure resulted in a test vehicle being seriously damaged and consequently the trial has had to be further delayed. Defence advised that under the terms of the COVID-19 Recovery Deed it is unable to enforce any stop payment milestones whilst the Deed is in effect. The COVID-19 Recovery Deed is outlined in Appendix 5.

127 There are 11 Contract Change Proposals. Contract Change Proposals 1–8 and 11 have been approved; and Contract Change Proposals 9 and 10 are yet to be approved.

128 Based on Defence advice to the Deputy Secretary CASG, this change added almost 600 new requirements and took nearly 12 months to finalise.

129 Comprising an increase of $91.5 million in the contract value (base date dollars) plus an associated increase of $42.4 million increase in the estimated amount to be paid by Defence for price escalations due to associated changes to milestone payment dates. The price increase is associated with additional Defence requirements necessary for Rheinmetall to deliver ‘a fully networked reconnaissance vehicle capability’ for the Block II vehicles. These requirements introduce an enhanced situational awareness and Command and Control, Computers and Intelligence (C41) systems integration for the Block II vehicles from the tendered basic Risk Mitigation Activity baseline vehicle.

130 The Monthly Reporting System was Defence’s primary system for reporting on the performance of its acquisition projects, and provided much of the project-specific data for the Quarterly Performance Report. Defence advised the ANAO in September 2020 that the Monthly Reporting System and its reporting functionalities transitioned to the Monthly Reporting Module (MRM) within the Project Performance Review Information Platform (PPRIP) on 3 August 2020.

131 The report is approved by the Deputy Secretary CASG and provided to the Secretary and Chief of the Defence Force (CDF) for endorsement. The Secretary and CDF provide the report to the Ministers for Defence and Defence Industry. The report is also provided to the Defence Investment Committee for noting and feedback. The Defence Investment Committee includes representatives from the Department of the Prime Minister and Cabinet and the Department of Finance. The reporting process was reviewed in Auditor-General Report No.3 2019–20 Defence’s Quarterly Performance Report on Acquisition and Sustainment, available from https://www.anao.gov.au/work/performance-audit/defence-quarterly-performance-report-acquisition-and-sustainment [accessed 14 May 2020].

132 Defence advised the ANAO in September 2020 that the Commonwealth did not direct the changes to the modification of turrets and protection requirements. These are Rheinmetall proposed changes from the RMA vehicle configuration to the Block I vehicle configuration and are obligations contained within the contract.

133 Government furnished equipment includes command, control, communications, computers and intelligence systems, training equipment, and weapons systems.

134 Defence informed the ANAO in July 2020 that Rheinmetall’s Project Status Report appears to be an accurate representation of the AIC achievement up until that date.

135 ANAO comment: Defence advised the ANAO in September 2020 that Rheinmetall’s financial reporting for AIC is drawn directly from their general ledger module in SAP, which is managed in accordance with International Financial Reporting Standards (IFRS) and the Australian Accounting Standards Board (AASB). Defence further advised the ANAO that there is not any specific AASB standard(s) that are applicable to the AIC values.

136 ANAO comment: Defence advised the ANAO in September 2020 that the AIC data is generated directly from Rheinmetall’s financial system as part of the standard EVM reporting obligations and is validated through the application of the standard financial management practices that are applicable to all financial data.

137 ANAO comment: Defence advised the ANAO in September 2020 that all Approved Sub-Contractors are required to deliver AIC status reports which show AIC activity over the previous reporting period. As Rheinmetall has recently aligned its AIC reporting with the CMS (contract master schedule), CWBS (contract work breakdown structure) and the EVM (earned value management) reporting requirements, the processes applicable to the validation of the EVM data are the same applied to ensuring the quality and accuracy of AIC content, with the EVM data also used for the generation of the AIC value generated from labour hours.