Audit snapshot

Why did we do this audit?

  • Effective delivery of key enabling services, including ICT, supports the Department of Defence (Defence) in meeting its purpose: ‘Defend and protect Australia and advance its strategic interests’.
  • The Enterprise Resource Planning (ERP) program is a major ICT reform initiative of strategic importance to Defence.
  • This audit provides independent assurance to Parliament on Defence’s progress to date in implementing Tranche 1 of the ERP program, including governance, monitoring and reporting, and the procurement and contract management of a systems integrator.

Key facts

  • The ERP program will be delivered in three tranches, with Tranche 1 split into Release 1A and Release 1B.
  • Release 1A achieved Final Operating Capability in December 2020.
  • Release 1B is planned to achieve Initial Operating Capability in quarter 4 2022 and Final Operating Capability in mid-2023.

What did we find?

  • Defence’s administration to date of Tranche 1 of the ERP program has been largely effective. However, there is scope for Defence to improve program governance arrangements, in particular the management of probity and the management of conflicts of interest in decision-making arrangements for varying contracts.
  • Defence has established largely fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program.
  • Defence conducted a largely effective procurement process for Tranche 1 of the ERP program, which included consideration of value for money.
  • Defence established largely fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program.

What did we recommend?

  • The Auditor-General made five recommendations aimed at improving Defence’s: risk reporting; probity management; program decision-making arrangements; contract governance; and benefits realisation planning.
  • Defence agreed to implement all five recommendations.

$1–2b

Total value of the ERP program in the 2016 Integrated Investment Program.

$364m

Tranche 1 budget approved by Government at second pass in 2018.

$274.5m

Value of systems integrator official orders as at 3 May 2021.

Summary and recommendations

Background

1. The First Principles Review: Creating One Defence was released publicly by the Minister for Defence in April 2015. One of the First Principles Review’s six key recommendations was to ‘fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness.’

2. The Defence White Paper released on 25 February 2016 also identified issues related to Defence information and communications technology (ICT) systems. The 2016 Integrated Investment Program, released with the Defence White Paper, committed to enhancements to the Department of Defence’s (Defence) ICT and business processes to support the implementation of First Principles Review recommendations. The 2016 Integrated Investment Program documented an ‘approximate investment value’ of $1–2 billion (out-turned) for an Enterprise Resource Planning System/Service, now known as the Enterprise Resource Planning Program (ERP), and a program timeframe of 2016–2025.

3. The ERP program involves the streamlining of Defence business processes associated with hundreds of separate Defence ICT applications into one SAP S/4HANA system, with the intent of enabling better governance, faster processing and lower maintenance and support costs. Functional areas where applications and processes are in scope for the program are: finance, human resources, supply, maintenance, engineering, procurement and estate.

4. The ERP program has three tranches. This audit focussed on Tranche 1, with an approved value of $364 million. In December 2020 the Australian Government approved $250 million for Tranche 2.

Rationale for undertaking the audit

5. Effective delivery of key enabling services, including ICT, supports Defence in meeting its purpose: ‘Defend and protect Australia and advance its strategic interests’.1 The ERP program is a major ICT reform initiative of strategic importance to Defence, as it is intended to assist in the delivery of an integrated, enterprise approach to service delivery as set out in the First Principles Review: Creating One Defence.2

6. This audit provides independent assurance to Parliament on Defence’s progress to date in implementing Tranche 1 of the ERP program, including governance, monitoring and reporting and the procurement and contract management of a systems integrator.

Audit objective, criteria and scope

7. The audit objective was to examine the effectiveness to date of Defence’s administration of the Enterprise Resource Planning (ERP) program, with a focus on ERP Tranche 1 activities.

8. To form a conclusion against the audit objective, the following high-level criteria were adopted:

  • Has Defence established fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program?
  • Has Defence conducted an effective procurement process for Tranche 1 of the ERP program that contributed to the achievement of a value for money outcome?
  • Has Defence established fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program?

9. In terms of procurement processes and contracting arrangements, the audit scope focused on the contract established with the systems integrator (IBM). This contract is material to the successful delivery of Tranche 1 and represents more than 50 per cent of the total program budget from 2019–20 onward.

10. The audit scope did not include a review of: Tranche 2 and 3 planning and activities undertaken to date for those tranches; and the decision to use SAP for the ERP system.

Conclusion

11. Defence’s administration to date of Tranche 1 of the Enterprise Resource Planning (ERP) program has been largely effective. However, there is scope for Defence to improve program governance arrangements, in particular the management of probity and the management of conflicts of interest in decision-making arrangements for varying contracts.

12. Defence has established largely fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program.

13. Defence conducted a largely effective procurement process for Tranche 1 of the ERP program, which included consideration of value for money. Two breaches of the Commonwealth Procurement Rules and two breaches of Defence policy were identified. A Defence Independent Assurance Review identified a range of issues requiring resolution in the tender process, resulting in tender decisions being set aside and additional steps being added to the tender evaluation process. Subsequently, IBM was identified as the preferred tenderer on the basis of offering best value for money.

14. Appropriate probity and conflict of interest arrangements were established to support Tranche 1 procurement activity. During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally.

15. Defence established largely fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program. While Defence has developed an appropriate contract with the systems integrator, program and contract governance has been undermined by conflicts of interest embedded in decision-making arrangements, with contractors involved in decision-making relating to their contracts.

16. As of May 2021, the ERP program remained at an early stage of implementation and substantial work remains for Defence to fully implement the program.

Supporting findings

Planning, governance, monitoring and reporting arrangements

17. While Defence’s planning for delivery of Tranche 1 of the ERP program up to second pass had significant deficiencies (identified through Department of Finance Gateway Reviews and a Defence Independent Assurance Review) planning following second pass has been largely effective. Defence has developed and updated program and risk management plans, and its planning has been informed by a number of third party reviews and the input of a systems integrator engaged in July 2019. While the planning process involves an iterative approach agreed by government, there have been two extensions of the date for achieving Initial Operating Capability (IOC) and reviews have identified distortions to project planning related to an earlier Defence assessment that a key ICT system (MILIS) required urgent replacement.

18. Defence’s advice to government was largely appropriate. Its December 2020 advice to government did not set out the action Defence had already taken to initiate program changes that would result in an IOC change, including a contract variation for over $14.5 million with the systems integrator in November 2020. The government agreed in December 2019 to adjust IOC from late 2020 to mid-2022, and in May 2021 approved a further change in IOC to late 2022.

19. Defence’s enterprise-level governance arrangements for delivery of the ERP program are largely fit-for-purpose, with initial deficiencies identified by third party program reviews addressed by the department. Key roles and responsibilities for the program are assigned to senior officials with whole of Defence responsibilities, introducing a level of senior management responsibility commensurate with the whole-of-Defence ICT transformation to be delivered by the program. High level program-specific governance is through the Enterprise Transformation Board (ETB) chaired by the Associate Secretary, and the program is also subject to Defence’s business as usual enterprise-level governance arrangements. Program-level governance arrangements include an Internal Program Board supported by a dedicated Program Management Office.

20. Defence has developed largely effective program monitoring and reporting arrangements which provide senior leaders with adequate visibility of progress and emerging risks. At the program level, ERP program managers have access to systems and reports that enable the monitoring of progress and risk. There is regular reporting on the program’s progress and risks to enterprise-level governance committees, the Defence Audit and Risk Committee, the Minister for Defence and the government. However, program risk is not presented consistently in Defence’s internal reporting and its reporting to government.

Procurement

21. Defence conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator selected for Tranche 1. Defence identified high level ERP system requirements prior to conducting a competitive Request for Tender process, and planned to refine its requirements through Offer Definition and Improvement Activity processes conducted as part of the tender process for the systems integrator.

22. Defence conducted a competitive tender process for the systems integrator that largely complied with the formal requirements of the Defence Procurement Policy Manual and Commonwealth Procurement Rules. A 2018 Defence Independent Assurance Review identified a range of issues requiring resolution in the systems integration tender process, resulting in tender decisions being set aside and additional steps in the tender evaluation process. When the tender process was restarted, additional offer definition and parallel negotiation activities were undertaken to address risks identified through the review. A new tender evaluation board prepared a second Source Evaluation Report, which identified IBM as the preferred tenderer on the basis of offering best value for money.

23. Instances of non-compliance with the Commonwealth Procurement Rules include that AusTender reporting was seven days late, and there is no record of consideration of environmental sustainability. There were also two identified instances of non-compliance with financial delegations and Defence financial policy related to the commitment of public money.

24. Defence established appropriate probity and conflict of interest arrangements for Tranche 1 procurement activity. The arrangements included the appointment of a probity advisor, the development of a probity framework and plan, and the conduct of probity briefings. Documented probity requirements were largely adhered to, with the following exceptions: there was no record of conflict of interest declarations in the program register for three of 50 personnel involved in procurement decisions; and personnel were not asked to sign a copy of the probity framework although the policy at the time required this.

25. During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally, and which require attention. These issues pertained to the management of: conflicts of interest; use of panel arrangements for this program; gifts and hospitality; and the use of official information.

Contracting

26. The contractual arrangements that Defence entered into with the systems integrator in July 2019, which included a Deed of Standing Offer and initial Official Order, reflected all relevant authorised negotiation outcomes. Defence achieved its preferred position or better in respect to 51 negotiation outcomes and its minimum fall-back position (identified prior to negotiation) for the remaining 12 negotiation outcomes.

27. The contractual arrangements entered into by Defence for systems integrator services as of May 2021, comprising six Official Orders for work packages under the Deed of Standing Offer, included clear milestones and performance expectations. Systems integrator fees are paid on an instalment basis and are contingent on Defence’s acceptance of clearly defined milestones, comprised of one or more contract deliverables.

28. Contract and program governance arrangements are partially effective in enabling Defence’s monitoring, management and reporting on the achievement of Tranche 1 outcomes. Defence has established a commercial team to manage the systems integrator contract and a system to track contract deliverables. The tracker provides a basis for internal reporting on the achievement of program deliverables and milestones.

29. During the course of the audit, Defence commenced work on improving the clarity and consistency of decision-making arrangements set out in program guidance documentation, particularly the thresholds for involving senior decision-makers and committees in Key Design Decisions and Program Change Requests. Additionally, there is a need for Defence to review decision-making arrangements for the program’s Change Control Board, which decides on Program Change Requests. The Board’s decision-making arrangements give rise to actual conflicts of interest as the systems integrator is part of the decision-making process for variations to its contract.

30. Positional authority issues arise from the program’s delegation and time approval arrangements. The majority of contract delegations are exercised by staff subordinate to the program director, and a number of subordinate APS/ADF personnel are responsible for approving their manager’s timesheets.

31. Tranche 1 is contributing to ERP program objectives and the Defence strategic priority, identified in the 2015 First Principles Review, of implementing an enterprise approach to the delivery of enabling services. However, as of May 2021, the ERP program remained at an early stage of implementation. Tranche 1A was delivered in December 2020 but Tranche 1B is the material activity of Tranche 1 and is yet to reach IOC. Program benefits are not expected to be realised until after IOC for Tranche 1B is reached in Quarter 4 2022. Tranche 2 has been approved and Tranche 3 is yet to be presented to government. Substantial work remains for Defence to fully implement the ERP program.

Recommendations

Recommendation no. 1

Paragraph 2.87

The Department of Defence review arrangements for its internal reporting and its reporting to government on Enterprise Resource Planning program risks, to ensure there is the consistency in program risk reporting.

Department of Defence response: Agreed.

Recommendation no. 2

Paragraph 3.88

The Department of Defence:

  1. review its probity arrangements for the Enterprise Resource Planning program, particularly with respect to its use of contractors, and apply lessons learned to similar programs; and
  2. develop more robust processes for on-boarding contractors, including ensuring awareness of probity and information security requirements.

Department of Defence response: Agreed.

Recommendation no. 3

Paragraph 4.38

The Department of Defence review Enterprise Resource Planning program decision-making arrangements to ensure they:

  1. are clearly and consistently set out in program guidance documentation;
  2. avoid real and perceived conflicts of interest where contractors are potentially involved in decision-making regarding their contracts, including contract variations with financial consequences for the Commonwealth; and
  3. enable delegations to be exercised by individuals who are in positions where they are able to make independent judgements.

Department of Defence response: Agreed.

Recommendation no. 4

Paragraph 4.44

The Department of Defence review contract governance arrangements to ensure timesheet approvals are made by personnel with adequate visibility of work performed.

Department of Defence response: Agreed.

Recommendation no. 5

Paragraph 4.57

The Department of Defence develop benefits realisation plans for each Enterprise Resource Planning program Tranche and the program overall.

Department of Defence response: Agreed.

Summary of the Department of Defence’s response

32. The Department of Defence’s summary response is provided below and its full response is included at Appendix 1.

Defence welcomes the overall findings that management of the Enterprise Resource Planning (ERP) Program has been largely effective. Defence is committed to program improvements that strengthen governance arrangements and ensure this important schedule of works is delivered in the best interests of the Australian Government and defence of the nation.

Defence acknowledges a difference of interpretation on the granularity of reporting obligations to the Australian Government. Defence maintains reporting has been complete throughout the program and Defence has only made administrative decisions, as appropriate, in delivering the ERP Program. All Defence decisions fall within the scope, schedule and budget parameters set by the Australian Government.3

Defence accepts the areas for improvement noted in the report. The matters identified have provoked immediate action in the interests of maintaining efficient, effective and secure delivery of the ERP Program. Where failings by individuals have been identified and verified, steps have been taken, commensurate to the seriousness of the findings, to ensure no repeat of this behaviour. Defence has confidence that, irrespective of the individual actions, Defence’s systems are secure and value for money has been achieved through ERP Program delivery to date.

Defence agrees to implement all recommendations proposed in the report. These actions will ensure any shortcomings at the system level have been identified and appropriately addressed as a priority. Defence has initiated complete reviews of ERP Program operations and taken steps to vary delegations, increase oversight and reconsider the appropriate balance of contractors with APS/ADF personnel in key management positions.

33. Appendix 2 sets outs improvements observed by the ANAO during the course of the audit.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Governance and risk management

Contract management

1. Background

Introduction

1.1 The First Principles Review: Creating One Defence4 was released publicly by the Minister for Defence in April 2015. The central theme of the review was that, for the Department of Defence (Defence) to achieve the outcomes the government expects efficiently and effectively, it needed to cease operating as a ‘federation of separate parts’ and become an integrated organisation.5 This ‘One Defence’ approach was expected to create ‘a more unified and integrated organisation that is more consistently linked to its strategy and clearly led by its centre.’6

1.2 One of the First Principles Review’s six key recommendations was to ‘fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness.’7 The review stated that:

There is general agreement about the nature of the problem. The current organisational model and processes are complicated, slow and inefficient in an environment which requires simplicity, greater agility and timely delivery. Waste, inefficiency and rework are palpable.8

Duplicated systems and processes reflect entrenched resistance to implementing businesslike approaches such as shared corporate services and the empowerment of single accountable officers in areas such as information management. According to the Chief Information Officer, Defence has over 2500 information and communication management applications including 300 financial applications.9

1.3 The Defence White Paper released on 25 February 2016 also identified issues related to Defence information and communications technology (ICT) systems, stating that underinvestment in ICT, and the lack of an enterprise-level strategy for Defence’s ICT requirements:

… has led to serious degradation across the information and communications capabilities of Defence. Key capabilities need urgent remediation, in particular to address the shortcomings of out-dated, and in some cases obsolete, systems that inhibit the conduct of day-to-day business within Defence, with overseas allies and partners, and with industry and the community more broadly.10

1.4 The 2016 Integrated Investment Program, released with the Defence White Paper, committed to enhancements to Defence’s ICT and business processes to support the implementation of First Principles Review recommendations, including by:

… standardising business processes to provide end-to-end visibility of Defence business through streamlined processes and a consolidated Defence Enterprise Resource Planning system that will improve core business functions, including force preparedness planning.11

1.5 The 2016 Integrated Investment Program documented an ‘approximate investment value’ of $1–2 billion (out-turned) for an Enterprise Resource Planning System/Service, now known as the Enterprise Resource Planning (ERP) program, and a program timeframe of 2016–2025.12

1.6 The 2020 Force Structure Plan continued to emphasise the importance of business transformation, setting out that:

Driving improved business processes, enterprise information management, enterprise resource management and broader transformational reforms across Defence will be critical to future Defence effectiveness. The Government is committed to improving enterprise business processes by implementing reform across Defence to maximise business commonality and effectiveness.13

The Enterprise Resource Planning program

1.7 The ERP program involves the streamlining of Defence business processes associated with hundreds of separate Defence ICT applications into one SAP S/4HANA system14, with the intent of enabling better governance, faster processing and lower maintenance and support costs. Functional areas where applications and processes are in scope for the program are: finance, human resources, supply, maintenance, engineering, procurement and estate. The scale of the intended transformation is represented in Figure 1.1 below.

Figure 1.1: Scale of the transformation expected from the ERP program

This figure sets out the various elements impacted by the Enterprise Resource Planning Program, including the number of: groups and services impacted, functional areas in scope, locations, applications to be retired, and users.

Source: Defence records.

1.8 The ERP program is subject to the Australian Government’s first and second pass ICT investment approval process, administered by the Department of Finance.15 The government agreed in May 2017 to consider a series of second pass work packages for the program. This approach, involving a series of second pass submissions, reflects that the program is to be delivered through a number of tranches and that planning work was (and still is) ongoing.

1.9 A timeline of key program events to date is set out below at Figure 1.2.

Figure 1.2: Timeline of key program events to date

This figure sets out a timeline of key dates for the ERP program, including dates related to: the procurement of a systems integrator, Release 1A initial and final operating capability achievement, and second pass Government approval

Source: ANAO analysis of Defence data.

First pass approval

1.10 The government provided first pass approval for the ERP program in May 2017. At first pass, the government agreed to Defence’s preferred option for the program16, which was to standardise and simplify end-to-end business processes, enabled by a new build SAP technology solution (see Appendix 3). The agreed program option would involve:

  • transformation of Defence’s finance, logistics, procurement, engineering, maintenance and estate processes; and
  • Defence engaging private sector partners, including systems integrators, to support delivery.

1.11 The Australian Government’s Digital Transformation Agency (DTA) supported the proposal, including the proposed iterative approach to design and planning. DTA noted that the proposal was consistent with best-practice design, delivery and assurance approaches. The government was also advised that the program risk rating was ‘high’ reflecting the size and complexity of the program.

1.12 The government agreed to a $58.9 million budget (out-turned based on parameters at the time) — including contingency of $10.7 million — from the Defence Integrated Investment Program provision for the ERP program of $1,002.2 million (excluding unfunded contingency of $221.8 million).17 The agreed funding was for the conduct of risk reduction studies, development of business cases for a series of second pass submissions, engagement of strategic partners, and the conduct of Offer Definition and Improvement Activities (ODIA) with potential systems integrators.

1.13 The indicative schedule for the overall program presented to the government included Initial Operating Capability (IOC)18 in February 2020 and Final Operating Capability (FOC)19 in 2026.

Second pass approval: Tranche 1

1.14 The second pass business case for Tranche 1 was approved by the government in June 2018, including $516.1 million (out-turned based on parameters at the time) from the Integrated Investment Program provision, to fund:

  • the delivery of Tranche 1 ($257.9 million);
  • ‘Prepare’ and ‘Explore’ phases for future tranches ($152.1 million); and
  • sustainment over three years ($106.1 million).

1.15 DTA supported the proposal. DTA noted that the proposal would significantly uplift Defence’s internal capabilities, making it a more responsive and efficient organisation and enabling the development of additional functions on the resulting core platform, while mitigating a number of critical risks to its systems, most immediately Defence’s Military Integrated Logistics Information System (MILIS).

1.16 The government approved dates for IOC and FOC at second pass are set out in Table 1.1 below.

Table 1.1: Initial Operating Capability (IOC) and Final Operating Capability (FOC) presented at second passa

Milestone

Date

Initial Operating Capability: to be achieved when the business transformation in scope for Tranche 1 has been successfully adopted, including the delivery of the foundation finance and HR capability, logistics and a land materiel maintenance capability.

Q4 2020

Final Operating Capability: to be achieved when the proposed end-to-end business transformation has been successfully adopted. This includes delivery of the ERP capability for finance, logistics, maintenance, engineering and estate.

Q4 2025

   

Note a: Defence advised the ANAO in March 2021 that the dates for IOC and FOC presented to the government at second pass were for the entire ERP capability (all tranches).

Source: Defence records.

1.17 The government approved the inclusion of the following Tranche 1 package of work:

  • the human resources and finance foundations; and
  • elements of logistics and land materiel capabilities, including the replacement of MILIS.

1.18 The second pass advice set out the delivery approach, including the program’s planned adoption of SAP Activate methodology (an iterative approach to planning, building and testing). The government was advised that SAP Activate is the common methodology globally for SAP implementation programs, and that all program partners were familiar with the methodology.20

Second pass approval: Tranches 2 and 3

1.19 The second pass business case for Tranche 2 was approved by the government in December 2020. This allocated $250 million from the Integrated Investment Program provision for the delivery of a case management system which would replace 16 existing systems. The case management system was initially a stand-alone program, with its own Integrated Investment Program provision, and had received first pass government approval in December 2017.21

1.20 Tranche 3 is intended to deliver the balance of ERP capability through ten ‘Releases’, including force preparedness and planning, asset management, estate, portfolio and project management, supply chain, procurement, human resources, and finance. Defence advised the ANAO in July 2021 that

… to manage risk, the program currently plans to seek Government approval of the full ERP scope, schedule and cost in October 2021, with funds released to deliver Releases 1–5. A second funding release, for Releases 6–10, will be sought in 2023, subject to approval by Government.

Program management and budget

1.21 The ERP program has been managed within the Chief Information Officer Group (CIOG) by a contracted Senior Executive Service (SES) Band 2 Program Director, and an ADF 2 Star Business Lead.

1.22 The program budget breakdown for the financial years 2017–18 to 2022–23 is set out in Table 1.2 below.

Table 1.2: ERP program budget 2017–18 to 2022–23 ($ millions)a

Item

2017–18b

2018–19

2019–20

2020–21b

2021–22

2022–23

Gate 1

N/A

$8.459

$2.200

N/A

$1.122

Contracted services

N/A

$28.066

$27.782

$40.177

$31.815

$32.076

Hardware

N/A

$0.006

$0.300

$0.710

$0.590

$1.054

Software

N/A

$11.886

$1.789

$21.596

$1.799

$1.453

IBM (systems integrator)

N/A

N/A

$45.641

$97.866

$49.117

$43.870

Other/expenses

N/A

$1.891

$3.983

N/A

$4.002

$3.676

Total

$29.689

$50.309c

$81.695

$160.349

$88.445

$82.129

Actual utilisation

$32.213d

$44.686

$82.692

N/A – current year

N/A – future year

N/A – future year

             

Note a: The program budget does not include the cost of Defence APS/ADF personnel. Defence advised the ANAO in 2018, in the context of the annual Major Projects Report, that its IT systems do not provide a direct mapping of personnel to projects, for the purpose of allocating personnel costs to projects.

Note b: The 2017–18 budget did not use the categories listed in the table. The most recent data for the 2020–21 financial year consolidated Contracted Services and Gate 1 into one category of commitment. The majority of the budget for 2017–18 was for contracted services ($29,347,496).

Note c: Numbers do not add up due to rounding.

Note d: The actual utilisation for 2017–18 also includes life to date spend up to 2017–18.

Source: ANAO analysis of Defence records.

Rationale for undertaking the audit

1.23 Effective delivery of key enabling services, including ICT, supports Defence in meeting its purpose: ‘Defend and protect Australia and advance its strategic interests’.22 The ERP program is a major ICT reform initiative of strategic importance to Defence, as it is intended to assist in the delivery of an integrated, enterprise approach to service delivery as set out in the First Principles Review: Creating One Defence.23

1.24 This audit provides independent assurance to Parliament on Defence’s progress to date in implementing Tranche 1 of the ERP program, including governance, monitoring and reporting and the procurement and contract management of a systems integrator.

Audit objective, criteria and scope

1.25 The audit objective was to examine the effectiveness to date of Defence’s administration of the Enterprise Resource Planning (ERP) program, with a focus on ERP Tranche 1 activities.

1.26 To form a conclusion against the audit objective, the following high-level criteria were adopted:

  • Has Defence established fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program?
  • Has Defence conducted an effective procurement process for Tranche 1 of the ERP program that contributed to the achievement of a value for money outcome?
  • Has Defence established fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program?

1.27 In terms of procurement processes and contracting arrangements, the audit scope focused on the contract established with the systems integrator (IBM). This contract is material to the successful delivery of Tranche 1 and represents more than 50 per cent of the total program budget from 2019–20 onward (See Table 1.2 above).

1.28 The audit scope did not include a review of: Tranche 2 and 3 planning and activities undertaken to date for those tranches; or the decision to use SAP for the ERP system.

Audit methodology

1.29 Audit procedures included:

  • reviewing advice provided to government regarding the ERP program;
  • reviewing Defence documentation related to ERP program planning, decision-making, the tender process, and monitoring and reporting of program performance; and
  • discussions with Defence personnel and personnel contracted by Defence to deliver the ERP program.

1.30 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $498,822.

1.31 The team members for this audit were Kelly Williamson, James Woodward, Corne Labuschagne, Nate Wirihana, Tara Rutter and Sally Ramsey.

2. Program planning, governance, monitoring and reporting arrangements

Areas examined

This chapter examines whether the Department of Defence (Defence) has established fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the Enterprise Resource Planning (ERP) program.

Conclusion

Defence has established largely fit-for-purpose planning, governance, monitoring and reporting arrangements to support implementation of the ERP program.

Areas for improvement

The ANAO made one recommendation aimed at improving the consistency of risk reporting related to the ERP program.

2.1 The ERP program involves the streamlining of Defence business processes associated with hundreds of separate Defence ICT applications into one system, to be applied by the whole of Defence, with the intent of enabling better governance, faster processing and lower maintenance and support costs. To support the effective implementation of the ERP program, fit-for-purpose planning, governance, monitoring and reporting arrangements, commensurate with the size and complexity of the program, need to be established.

2.2 This chapter examines whether Defence has:

  • effectively planned for the delivery of Tranche 1 of the ERP program and provided appropriate advice to government following second pass;
  • established a fit-for-purpose governance structure to support delivery of the ERP program, Tranche 1; and
  • developed effective monitoring and reporting arrangements that provide senior leaders with adequate visibility of progress and emerging risks.

Did Defence effectively plan for the delivery of Tranche 1 of the ERP program and provide appropriate advice to government following second pass?

While Defence’s planning for delivery of Tranche 1 of the ERP program up to second pass had significant deficiencies (identified through Department of Finance Gateway Reviews and a Defence Independent Assurance Review) planning following second pass has been largely effective. Defence has developed and updated program and risk management plans, and its planning has been informed by a number of third party reviews and the input of a systems integrator engaged in July 2019. While the planning process involves an iterative approach agreed by government, there have been two extensions of the date for achieving Initial Operating Capability (IOC) and reviews have identified distortions to project planning related to an earlier Defence assessment that a key ICT system (MILIS) required urgent replacement.

Defence’s advice to government was largely appropriate. Its December 2020 advice to government did not set out the action Defence had already taken to initiate program changes that would result in an IOC change, including a contract variation for over $14.5 million with the systems integrator in November 2020. The government agreed in December 2019 to adjust IOC from late 2020 to mid-2022, and in May 2021 approved a further change in IOC to late 2022.

2.3 To establish whether Defence effectively planned for the delivery of Tranche 1 of the ERP Program, the ANAO examined:

  • plans established by Defence to guide the delivery of Tranche 1;
  • systems and processes established to identify and manage risks to the successful implementation of the ERP program;
  • Defence’s management of recommendations from reviews of implementation; and
  • Defence’s advice to government regarding the implementation of Tranche 1 and the program more broadly.

Tranche 1 program plan following second pass approval

2.4 Program management plans were developed in 2015 and 2017 to inform program development at first and second pass. A new program management plan (the plan) was developed by the systems integrator24 and approved by the Program Director on 6 December 2019. The plan was updated and approved by the Program Director on 11 March 2021. These updates reflected:

  • work done on future tranches;
  • schedule changes agreed by Government; and
  • governance changes, such as changes to the risk management approach and key program governance forums.

2.5 The plan sets out program objectives, outcomes, key success factors, high-level scope and schedule. It reflects planning that has taken place with the systems integrator since second pass. The high level scope approved by the government recognises the expectation that the ERP solution’s design will evolve during the ‘Explore’ phase (see Appendix 3).

2.6 There is a suite of planning documents which sits below the plan, including the Integrated Master Schedule, Tranche Implementation Plan, and Master Resource Schedule. These capture detailed planning and changes over time, and are reviewed quarterly as systems integrator deliverables.

2.7 The plan also breaks down Tranche 1 into two component parts:

Release 1A: An element of Tranche 1 Foundation Finance that will provide a central finance reporting capability based on the “to be” Chart of Accounts and Enterprise Structures. Release 1A will use the Tranche 1 core S/4HANA ERP system in conjunction with SAP Central Finance (CFIN). It will deliver a finance solution with enterprise structure and master data objects based on the new Defence Finance Group (DFG) design. Interfaces and mapping rules will be implemented to allow the extraction and loading of required ROMAN and BORIS data. Custom reports will be built to allow DFG to meet Internal and External reporting requirements using the new structure and master data design.

Release 1B: It will deliver a solution for Core Supply Chain Management and Maintenance [MILIS] across Defence. It will also deliver Engineering, Procurement, Finance and HR foundations to support this capability. This release will subsume capability in some legacy systems and interface to others.

2.8 The plan sets out a timeline for Tranche 1, including IOC and FOC dates for Release 1A and Release 1B. See Figure 2.1 below.

Figure 2.1: Tranche 1 timeline as set out in the March 2021 ERP program management plana

This figure presents a timeline for Tranche 1 of the ERP program, including dates for Release 1A, Release 1B and Future Tranches.

Note a: Ticks indicate completion of milestone. The Release 1B IOC and FOC dates in this figure reflect the agreed dates in the Release 1B official order with the systems integrator, discussed in paragraph 2.9 below.

Source: ANAO representation of Defence records.

2.9 Release 1A IOC and FOC have been achieved, as set out in the timeline in Figure 2.1. Release 1A FOC was originally planned for October 2020, and was delivered in December 2020. Deliverables and key milestones for IOC and FOC Release 1B were established as part of the Release 1B Realise and Deploy official order with the systems integrator (IBM), executed on 26 March 2021. The official order sets deliverable milestones for Release 1B IOC to be achieved in October 2022 and FOC in December 2022. Defence advised the ANAO that these delivery dates have been set for IBM to enable Defence to achieve the key dates agreed by government: IOC by Quarter 4 2022 and FOC by mid-2023 (see Table 2.1).

2.10 Requirements definition and process capture and mapping activities have been ongoing. Defence has advised that as at 26 April 2021, 836 persons across the Groups and Services have committed 7,695 workshop days, with a total commitment of 12,300 workshop days when ERP personnel are included. Workshops involved business representatives who assisted in confirming business processes and identifying gaps between business requirements and the SAP solution. Outputs have been formalised into business process documents, which are a systems integrator contract deliverable.

2.11 As discussed in the next section, there is evidence of ongoing planning to mitigate risks and address program challenges. This takes place through regular monitoring and review of the RADDICAL (Risk, Assumptions, Dependencies, Decisions, Issues, Changes, Actions, Lessons) registers maintained in the Enterprise Portfolio Management System (EPMS — an electronic system used for day-to-day program management).

Risk management

2.12 An August 2017 internal audit found that prior to first pass the program had not given sufficient priority to ensuring risks were effectively recognised and reported. The internal audit stated that assurance had been provided that a comprehensive risk management framework was planned to be delivered.

2.13 Subsequently, Defence developed a strategy to address the management of risks, assumptions, issues, dependencies and decisions in the program (known as ‘RAIDD’). The strategy set out the process for risk identification, analysis, reporting and management. Defence further developed this into an approach to manage benefits, risks, actions, dependencies, decisions, issues, change, communications, assumptions, and lessons (known as ‘B-RADDICCAL’), as set out in the program management plan approved in December 2019.

2.14 The ERP program has since moved to a model for managing risks, actions, dependencies, decisions, issues, change, assumptions, and lessons (known as ‘RADDICAL’), as set out in a draft 2020 risk and issue management strategy, the August 2020 ERP program handbook, and the March 2021 updated program management plan.25 Registers for RADDICAL items are embedded in EPMS.

2.15 Defence has established a three-tiered risk management system for the ERP program. The three tiers are: stream26, program and enterprise. Defence advised the ANAO that:

  • ‘stream’ risks are reviewed weekly by the program management office;
  • ‘program’ risks are reviewed monthly by the program executive team; and
  • ‘enterprise’ risks are reviewed quarterly at a Groups and Services risk review session with ADF 1 Star officers, aligning with the monthly 1 Star group meeting.27

2.16 As at 19 May 2021, there were 155 stream, 16 program, and 14 enterprise ‘in progress’ and ‘open’ risks recorded in the EPMS. Six stream and one enterprise risks in the register have a residual rating of ‘high’, and no risks have a residual rating of ‘extreme’.28 Defence has advised that the Chief Information Officer Group operates under the Defence Enterprise Risk Appetite Statement. The Defence Enterprise Risk Appetite Statement states that:

Defence generally prefers to accept low to medium levels of risk, so far as reasonably practical. However, due to its complex operating environment, Defence may need to manage higher levels of risk.

2.17 Defence’s November 2020 internal audit, titled ‘ERP Release 1B Implementation Preparation Maturity Assessment’, recommended that the program consider key person risks within the program. In response to the internal audit, Defence committed to completing an assessment of key person dependencies and reporting on this to the ETB in February 2021. In April 2021 the program presented a paper to the ETB on its response to the audit recommendation. The ETB noted the paper, but requested that the program return in May with a clearer approach to demand and supply of ERP workforce requirements. On 12 May 2021, Defence added a program risk in EPMS related to key persons leaving the program, with an initial and residual rating of ‘medium’. In June 2021 the program returned to the ETB with a paper on how the internal audit recommendations had been addressed, including setting out its mitigation actions for key person risks. The ETB endorsed the actions the program had taken in response to the recommendations.

Management of recommendations from reviews

2.18 As well as the internal audits outlined above, the program has been subject to a number of Department of Finance Gateway reviews and an Independent Assurance Review, which have examined the adequacy of planning to date. Defence’s Audit and Risk Committee has determined that the program will be subject to further internal audits, ‘given its complexity and the effects on many parts of Defence’.

2.19 The Digital Transformation Agency’s comments at second pass (June 2018), while supportive of the program, highlighted the risks of change management, size and the ‘aggressive’ schedule, noting that adoption of SAP’s Model Defence Organisation (MDO)29 and careful, responsive management and top-level support could assist in mitigating risks. At second pass, Defence’s stated reason for the ‘aggressive’ schedule for Tranche 1 was to mitigate the risks posed by MILIS, which at that time was considered to be not fully supportable beyond late 2020 and in need of urgent replacement.

2.20 A number of reviews had raised concerns regarding the planned schedule, including:

  • A December 2017 Gateway review (Gateway reviews are discussed further at paragraph 2.48) recommended that Defence ‘mature dependency, project and schedule planning and adapt the schedule to include sufficient contingency for delays.’
  • A September 2018 Independent Assurance Review considered whether the proposed phasing and rollout for the ERP program best met the needs and priorities of Defence and enabled effective management of risk. The review considered the overall program strategy, planning and resourcing, and found ‘significant issues and concerns’30 which were ‘exacerbated by the current aggressive schedule to Tranche 1’. The review found that relief from the ‘aggressive’ program schedule would allow more time for business transformation, SAP product development and more detailed planning of Tranche 1 functionality. The review raised concerns that the program was shaped primarily by the pressing need to replace one element (MILIS) and would defer other core components of the ERP system, which would ‘necessarily involve complex interfaces and work-around and a level of additional risk, as yet unquantified’. At this stage, the program was waiting on the systems integrator procurement to progress planning.31
  • A May 2019 Gateway review found that the above recommendation from the December 2017 Gateway review was not implemented, with Defence relying on the MDO to accelerate the schedule and de-risk delivery. Defence was aware that it would need to identify gaps in the MDO compared to Defence’s requirements, and that the MDO would need to be incrementally expanded to meet program outcomes.

2.21 Defence actions taken as a result of the 2018 Independent Assurance Review included: re-baselining the program schedule with the systems integrator once the procurement was complete; broadening the role of the responsible governance body (which is now called the Enterprise Transformation Board) to include key business domain owners; and integrating governance of the ERP and Enterprise Information Management programs.32

Program re-baselining: 2019 and 2020

2.22 In late 2019 the program was re-baselined through engagement with senior leaders and the systems integrator, resulting in a planned delay in IOC from Quarter 4 2020 to mid-2022. Through this process, Defence confirmed the program scope and refined the implementation plan. Further, the imperative to replace MILIS — which was originally understood to be unsupportable beyond 2020 and which drove the ‘aggressive’ ERP schedule — was able to be reduced. Work was done to find options that would enable support to MILIS beyond 2020 and delay its decommissioning. Defence recognised that the delay in decommissioning MILIS would lead to an increase in maintenance and support costs. The cost of supporting MILIS was estimated by Defence to be $11,639,321 each year from 2021–22 to 2040–41 (in constant dollars, not out-turned dollars).

2.23 In late 2020 the program was further re-baselined, resulting in a further IOC delay to Quarter 4 2022.33 The request to further delay IOC was initially agreed by Defence’s Enterprise Transformation Board (ETB) on 6 August 2020 (the board is discussed further from paragraph 2.38 below). The ETB had been presented with a proposal to implement: a change to how the ERP program finance scope was delivered; delivery of a ‘basic SAP Transport Management (TM) functionality to enable Cargo Visibility System (CVS) replacement’; and an extension for the ‘explore’ phase of Tranche 1B by three months and increased system testing during its ‘realise’ phase. The ETB was advised that the changes would: result in a three month IOC delay; would not change FOC; and would be delivered within the approved program budget.34 After receiving the ETB’s agreement to implement the proposal, Defence signed a contract variation for $14,594,193.83 with the systems integrator on 30 November 2020.

Advice to government regarding changes to program planning

2.24 In November 2019, Defence advised the government that a delay to IOC35 was required to reduce schedule risk from ‘high’ to ‘medium high’. This delay represented the re-baselining of the program that took place through the planning process with the systems integrator, and the identification of deficiencies in the initial program planning through reviews (discussed above).

2.25 Defence further advised the government that it had not yet assessed the impact on FOC, but that this advice would be provided in mid-2020.36

2.26 The government agreed to the IOC change on 9 December 2019, noting that although it was affordable within the Integrated Investment Program provision37, it would delay benefits in ICT sustainment (such as the potential costs associated with delaying the decommissioning of MILIS), workplace efficiencies, and improved information management capabilities.

2.27 In December 2020, Defence sought approval from government to further delay Tranche 1 IOC by three months.38 The rationale for further IOC change was to keep schedule risk at ‘medium-high’ in response to a range of factors, including:

  • the need for a better understanding of requirements and risks during rollout;
  • the need to develop a strategy to optimise rollout (detailed rollout design was to take place in early 2021);
  • to accommodate lessons learnt from Release 1A around additional test time needed;
  • the emergence of ‘design challenges’; and
  • a delay in a critical SAP software update.

2.28 The government considered and agreed to the request to delay IOC on 10 May 2021.

2.29 The advice provided to government in December 2020, updated by Defence in March 2021, was not complete, as it did not set out the action that Defence had already taken to initiate the program changes in anticipation of government agreeing to an IOC change. This included signing a contract variation for over $14.5 million with the systems integrator in November 2020. The variation included the items agreed by ETB in August 2020, which ETB had been advised would result in a three month delay to IOC (see paragraph 2.23). This included a design extension, with finalisation of R1B design moved from February 2021 to June 2021. Defence advised the ANAO in March 2021 that if government did not approve the IOC change, it would renegotiate the design extension in the systems integrator contract.

2.30 On 21 July 2021, Defence advised the ANAO that:

Defence does not agree that advice provided to government in December 2020 was not complete…

… The contract variation signed in November 2020 with the Systems Integrator was within the Program’s Government Approved Tranche 1 scope and Budget. From Defence’s perspective, contract changes of this nature do not require prior agreement from government.

2.31 While the contract variation was within the agreed scope and budget, it would impact on the schedule approved by government (including the IOC date) at that time. IOC and FOC dates have been approved by the government for this program. Table 2.1 below sets out the Tranche 1 timeframes as approved by government following Defence advice.

Table 2.1: Tranche 1 timeframe changes approved by government

Date of advice to government

Date of government approval

Initial Operating Capability (IOC)a

Final Operating Capability (FOC)a

March 2017 (first pass)

May 2017

February 2020

2026

May 2018 (second pass)

June 2018

Quarter 4 2020

Quarter 4 2025

November 2019 update

December 2019

Mid-2022

Not specified: impact of proposed IOC change unknown

May 2020 update

May 2020

Mid-2022

Mid-2023

December 2020 updateb

May 2021

Quarter 4 2022

Mid-2023

       

Note a: At first and second pass, IOC and FOC dates were for the entire ERP capability, and not specific to Tranche 1. In the November 2019 update, IOC was for Tranche 1. From the May 2020 update (rows shaded), when Defence refers to IOC it relates to Tranche 1 Release 1B.

Note b: Government consideration, initially expected in February 2021, was delayed. The submission was updated in March 2021.

Source: Defence records.

Did Defence establish a fit-for-purpose governance structure to support delivery of the ERP program, Tranche 1?

Defence’s enterprise-level governance arrangements for delivery of the ERP program are largely fit-for-purpose, with initial deficiencies identified by third party program reviews addressed by the department. Key roles and responsibilities for the program are assigned to senior officials with whole of Defence responsibilities, introducing a level of senior management responsibility commensurate with the whole-of-Defence ICT transformation to be delivered by the program. High level program-specific governance is through the Enterprise Transformation Board chaired by the Associate Secretary, and the program is also subject to Defence’s business as usual enterprise-level governance arrangements. Program-level governance arrangements include an Internal Program Board supported by a dedicated Program Management Office.

2.32 This section examines the enterprise and program level governance arrangements in place for the ERP Program. The management of recommendations from internal audits that were completed in August 2017 and November 2020, and an Independent Assurance Review completed in September 2018 are also considered. The Department of Finance’s Gateway Review process has also been examined.

Enterprise-level governance arrangements

2.33 Key roles and responsibilities for the ERP program are assigned to senior officials with whole of Defence responsibilities — the Associate Secretary is the Capability Manager, and the Chief Information Officer is the Senior Responsible Officer.39 These arrangements introduce a level of senior management responsibility commensurate with the whole of Defence ICT transformation to be delivered by the program.

2.34 Defence’s enterprise-level program governance arrangements involve a three tier structure, summarised in Table 2.2 below.

Table 2.2: Enterprise Resource Planning program enterprise-level governance structure

Committee

Chair

Frequency

Enterprise Business Committee (EBC)

Associate Secretary

Monthly

Enterprise Transformation Board (ETB)

Associate Secretary

Monthly

Defence Communications and Information Systems Committee (DCISC)

Chief Information Officer

Monthly

     

Source: Defence records.

2.35 High level program-specific governance is through the ETB chaired by the Associate Secretary.

2.36 The program is subject to Defence’s business as usual enterprise-level governance arrangements through the Enterprise Business Committee and the Defence Communications and Information Systems Committee.

2.37 In addition, there is involvement as needed by: the Defence Committee (DC), the most senior and primary executive committee within the Department of Defence, responsible for setting top-level organisational goals and driving delivery of Defence’s commitments to the government and the community; and the Investment Committee (IC), which is on the same tier as the EBC and is responsible for ensuring that Defence’s investment portfolio for military capability, estate and ICT meets the government’s requirements and the Chief of the Defence Force’s directive.

Enterprise Transformation Board

2.38 An SES Band 3/ADF 3 Star steering group for the ERP program was established in March 2018. It was renamed the Enterprise Transformation Board (ETB) in March 2020. The ETB was established after the December 2017 Gateway Review recommended that a sub-committee of EBC be formed to focus on the ERP program and provide guidance to EBC.40

2.39 The ETB is chaired by the Associate Secretary. The committee meets monthly and its duties as set out in its charter are to:

  • ensure clarity and support for the continuing organisational context for the programs41;
  • review and support resolution of major issues and risks to support the progress of the programs against strategic objectives;
  • actively champion the implementation of the required business transformational change;
  • ensure identified Defence benefits are achieved from the investment;
  • encourage active commitment and support of the program across Defence;
  • advise and support the Program Directors/Managers; and
  • oversee successful delivery and closure of the programs.

2.40 The membership of the earliest iteration of the SES Band 3/ADF 3 Star group comprised the Associate Secretary, Vice Chief of the Defence Force, and the Chief Information Officer. The September 2018 Independent Assurance Review42 of the ERP program recommended that membership be broadened to include key business domain owners. This recommendation was accepted and implemented. The following members were added: the Chief Finance Officer; Deputy Secretary Defence People Group; Deputy Secretary Estate and Infrastructure Group; Deputy Secretary Capability Acquisition and Sustainment Group; Deputy Secretary Chief of Joint Capabilities; First Assistant Secretary Contestability; and Deputy Director General Corporate Capability.

2.41 Concerns have been raised at the ETB regarding adequate representation from the Services (Navy, Army and Air Force) at the ETB, where business transformation decisions are being made. Further to this, a November 2020 Capability Manager Gate Review of the program identified that governance arrangements did not provide for adequate Group and Service representation. In response, in December 2020 the ETB directed the program to develop a terms of reference for a SES Band 2/ADF 2 Star Steering Committee. In February 2021, the ETB determined that rather than establish a new committee, the existing Defence Communications and Information Systems Committee should serve as the steering committee every alternate meeting. The ETB noted the importance of business representation at this meeting.

2.42 The first ERP-focussed Defence Communications and Information Systems Committee was held on 19 April 2021. The committee’s terms of reference were updated at this meeting to reflect that every second meeting would focus on the ERP program.

Program resourcing

2.43 The November 2020 internal audit included a recommendation that the program:

… identify the ongoing people capability requirements for ERP R1B Implementation. This should include seeking Enterprise Transformation Board agreement to the prioritisation of ERP R1B implementation activities and to obtain the commitment of essential resources from the Groups and Services.

2.44 In response to the internal audit recommendation, the program committed to returning to the ETB in February 2021 with a paper explaining the resourcing process and resource demand profile, seeking agreement for the ETB to prioritise ERP activities when making resourcing decisions.43 In April 2021 the program presented a paper on its response to the audit recommendation to DCISC and the ETB. The ETB noted the paper and requested that the program return in May with a clearer approach to demand and supply of ERP workforce requirements. There was no May 2021 meeting. In June 2020 the program presented a further paper to ETB, setting out its approach to managing workforce requirements. The program asked the ETB to: endorse its actions in response to the audit recommendations; agree to its planned next steps — including returning to the ETB where resource constraint issues need to be escalated; and agree to the prioritisation of ERP implementation decisions when making resource decisions. The ETB endorsed the actions the program had taken, agreed to the next steps, and did not agree to the prioritisation of ERP implementation activities when making resource decisions.

Program assurance

2.45 In February 2021 the program presented a paper to the ETB, recommending that the ETB note assurance processes in place for the program. The ETB raised concerns with the model presented, and agreed to explore the engagement of an independent assurance professional to determine an appropriate assurance framework. In March 2021 the ETB Chair requested that the item be discussed again at the May 2021 meeting. In June 2021, the ETB discussed that a consulting firm, Proximity, had been engaged to conduct a review of the ETB. The ETB expect that the review will contribute to any future ERP assurance arrangement. The terms of reference for the review set out that it was expected to commence in late May 2021 and present its findings by mid-June 2021. On 21 July 2021 Defence advised the ANAO that the review was ongoing.

Program-level governance arrangements

2.46 Defence’s program level governance arrangements include a Program Management Office (PMO) and an internal program board. The November 2020 internal audit assigned a ‘medium’ maturity rating to the program governance arrangements, reporting that there were:

… sound governance mechanisms in place for decision making. While decisions are made by the ERP Program following consultation with the Groups and Services, accountability and responsibility for the decisions remains with the Program.

Program management office

2.47 Day-to-day management of the program is the responsibility of a dedicated PMO.44 The PMO maintains EPMS, used to record and monitor all RADDICAL (risk, assumptions, dependencies, decisions, issues, changes, actions, lessons) items, and to generate reports.

Findings of Gateway reviews related to program level governance

2.48 As at November 2020, the program had five Gateway reviews.45 Overall ratings as a result of the reviews were:

  • 0/1 Gateway review of the ERP First Pass Business Case, provided to Defence on 11 December 2015, rated the program ‘Amber’46;
  • 2/3 Gateway review, provided to Defence on 15 December 2017, rated the program ‘Green/Amber’47;
  • Mid Stage Gateway review, provided to Defence on 17 May 2019, rated the program ‘Amber/Red’48;
  • Short Form Gateway review49, provided to Defence on 4 March 2020, rated the program ‘Green/Amber’; and
  • a second Mid Stage Gateway review, provided to Defence on 6 November 2020, rated the program ‘Green/Amber’.

2.49 The 15 December 2017 Gateway review noted that ‘balancing the needs of the Commonwealth and the commercial drivers of the partners will be critical to success’ and ‘a need to balance the role of solution delivery partners and key accountabilities within Defence Groups to ensure that the program is led and supported by Defence leaders’. The review recommended that Defence ‘establish an organisation structure appropriate to the delivery stage including appropriate staffing from ADF/APS in both leadership and decision-making roles to maintain the interests of the Commonwealth during 1st quarter 2018.’ The Department of Finance assessed this recommendation as partially implemented at the 17 May 2019 review, noting that an organisational structure had been implemented, but required revision. In response, the program noted that: it planned to have two program leads, with a program director (SES Band 2 level) focussed on the technical aspects and a business lead (SES Band 2/ADF 2 Star level) focussed on culture and change aspects; and it had commenced recruiting ADF/APS personnel in key positions in capability delivery teams, who would have private sector counterparts to lead and support their roles.

2.50 The structure described above had been implemented at the time of the ANAO’s audit fieldwork, and is illustrated in Figure 2.2 below. During the audit the program had:

  • an ‘above the line’ SES Band 2 Program Director;
  • an ADF 2 Star Head Business Transformation Lead;
  • functional streams led by:
    • ADF/APS business leads;
    • ‘above the line’ delivery leads (SAP specialists); and
    • ‘below the line’ IBM capability leads (the contracted systems integrator).

2.51 Box 1 below summarises Defence’s use of ‘above the line’ and ’below the line’ descriptors for the program’s contractor workforce.50

Box 1: ‘Above the line’ and ’below the line’ program contractor arrangements

Defence advised the ANAO in May 2021 that the ‘contractor workforce includes contractors who perform an “above the line” function on behalf of Defence and the Program, to supplement Defence’s APS and ADF workforce.’ Defence provided the following explanation of the difference between ‘above the line’ and ‘below the line’ contractors:

These “above the line” contractors provide specialist skills and support to Defence and are engaged through normal Defence Program funding arrangements. “Above the line” contractors operate on the ‘client side’ on behalf of Defence and assist with managing the ‘below the line’ contractors (primarily the systems integrator and technology partner) in delivering the required ERP capability outcomes. A number of the ‘above the line’ contractors are in key Program roles or perform key Program functions (e.g. Delivery Leads working with APS/ADF Business Leads).

The “above the line” contractors are not sourced from the system integrator or technology partner organisations, and follow clear conflict of interest arrangements as specified in their services contracts and the Program’s probity framework [discussed in paragraph 3.60].

At the time of the audit, none of the ‘above the line’ contractors had been engaged as ‘officials’ for the purposes of the Public Governance, Performance and Accountability Act 2013, section 13.

Figure 2.2: ERP program structure as at April 2021

This figure presents an organisational chart of the ERP program.

Source: ANAO analysis of Defence information, as at 1 April 2021.

Internal Program Board

2.52 An ERP Internal Program Board was established in March 2019.51 The Board is chaired by the ERP Program Director and includes program executives, representatives from SAP and the systems integrator (IBM). Eight of the ten board members are contractors and two are APS/ADF.52 The board meets monthly. The ERP program management plan sets out that the board’s role is to:

  • review monthly status reports;
  • review key change management issues and risks;
  • approve Program Change Requests53 (as per delegation); and
  • prepare for DCISC and ETB meetings.

2.53 The board met 16 times between May 2019 and April 2021. The board fulfilled the functions described above, with the exception of Program Change Request approval.54 Defence advised the ANAO on 16 April 2021 that the Change Control Board was established in February 2020 to ‘progress and approve all Program Change Requests’.55 As discussed in Box 2 on page 72, the program management plan was updated to reflect the Change Control Board’s role in March 2021. The March 2021 program management plan set out that the objective of the Change Control Board was to:

  • ‘Review Project Change Requests’
  • ‘Provide direction on Project Change Requests’
  • ‘Decision making on Project Change Requests’
  • ‘Approval / reject Project Change Requests’.

2.54 Terms of reference for the Change Control Board were approved by the CIO on 7 July 2021. The terms of reference state that the board is now ‘an advisory committee that reviews and provides guidance on Program Change Requests (PCRs) in order to provide endorsed recommendations to a Commonwealth decision-authority for approval.’ The Change Control Board’s role is further discussed in paragraphs 4.24–4.32.

Engagement with Defence business areas

2.55 The government was advised at second pass (May 2018) that key risks to the ERP program included organisational acceptance of new process, and integration with other information reform initiatives. Stakeholder engagement is a mechanism to mitigate these risks. Defence has developed a range of consultative forums to support ERP program governance and implementation.

2.56 In June 2017, Defence contracted Ernst and Young (EY) as an Organisational Change Manager (OCM) to assist with engagement with Defence business areas. The OCM’s role has involved development of stakeholder management and communications plans to support engagement.56 Stakeholder engagement activities set out in the plans include three consultative forums (see Table 2.3 below).

Table 2.3: ERP consultative forums

Forum

Date established

Purpose

Membership

Meeting frequency

SES Band 1/ADF 1 Star

October 2019

  • Provides specialist advice to the program.
  • Enables, within their respective Group or Service, Defence ERP to be successfully implemented.
  • Provides resources to support the delivery of the program.

SES Band 1/ADF 1 Star representation from each of the Groups and Services.

Monthly

Business Readiness Network

June 2020

Leads, plans, coordinates and monitors the execution of business readiness activities within the Groups and Services to drive the successful implementation of the Defence ERP program.

O6/EL2a representatives from the Groups and Services, ERP program Business Process Leads, and OCM Engagement Leads.

Monthly

Organisational Change Management Network

January 2020b

Provides a forum for the ERP OCM team and the Groups and Services to communicate and collaborate in relation to ERP OCM activities.

Representatives from each of the Groups and Services as well as ERP program Business Process Leads and OCM Engagement Leads.

Monthly

         

Note a: An O6 is the ADF equivalent of an APS EL2.

Note b: Defence advised the ANAO that the Organisational Change Management Network existed in an earlier form, the ERP Comms Network, and first met in October 2019.

Source: ANAO analysis of Defence records.

2.57 Defence advised the ANAO in January 2021 that each forum has been implemented as planned. As set out in Table 2.3, the SES Band 1/ADF 1 Star Group is the highest level forum, intended to support governance through provision of advice to the ERP program, and support program implementation through resourcing. Once implemented, the SES Band 2/ADF 2 Star steering committee (see paragraph 2.41) is expected to ensure the program and the ETB are receiving strengthened support in these areas.

2.58 Although the forums have been implemented, the November 2020 internal audit found that the relationship between the governance committees and consultative forums could be more clearly defined. Defence advised the ANAO in May 2021 that it would commence work in mid-2021 to review, update and approve Terms of Reference documentation for ERP program committees and consultative groups to ensure relationships are more clearly defined.

Did Defence develop effective monitoring and reporting arrangements that provide senior leaders with adequate visibility of progress and emerging risks?

Defence has developed largely effective program monitoring and reporting arrangements which provide senior leaders with adequate visibility of progress and emerging risks. At the program level, ERP program managers have access to systems and reports that enable the monitoring of progress and risk. There is regular reporting on the program’s progress and risks to enterprise-level governance committees, the Defence Audit and Risk Committee, the Minister for Defence and the government. However, program risk is not presented consistently in Defence’s internal reporting and its reporting to government.

Internal monitoring and reporting

2.59 There is regular reporting on the ERP program’s progress to relevant senior leaders, including the CIO as the Senior Responsible Officer, and to enterprise-level Defence committees with a governance role in the program.

2.60 To support monitoring by Defence senior leaders, the following reports are provided:

  • internal status reports; and
  • the Chief Information Officer Group Quarterly Performance Report.

2.61 In addition, the CIO provides scheduled reports to the Defence Audit and Risk Committee on the status of the program. These reports were included in the committee’s forward work plan from 2020. The 31 March 2021 report included advice that ‘the ERP has progressed Tranche 1, received government approval for Tranche 2 (Case Management) and progressed planning for Tranche 3’.

Internal Status reports

2.62 The ERP program handbook sets out reporting arrangements to support program governance. These reporting arrangements are described in Table 2.4 below.

Table 2.4: Enterprise Resource Planning program reports

Report

Content

Audience

Commenced

Weekly Stream Status Report

The report is sourced directly from the ERP program’s Electronic Project Management System (EPMS).

For each of the program streams, the reports provide a traffic light snapshot of scope, schedule, risks and issues and overall status, as well as qualitative information on progress, high and extreme risks, high and extreme issues, decisions requiring action and changes requiring action.

  • Program Executive
  • Program Team

July 2019, with a break until September 2019a

Weekly Program Status Report

The report is prepared by the Stream leads.

Provides an overall update, including information on whether the program is on track to meet timeframes.

Provides a summary of progress against Key Performance Indicators (KPIs).b

  • Chief Information Officer

September 2019

Monthly Program Status Report

Provides an overall update, including information on whether the program is on track to meet timeframes.

Includes: an overall update, information on progress against KPIs and key milestones and deliverables, which is manually entered; information on work stream activities, high and extreme risks, high and extreme issues, project change requests, and key design decisions, which is uploaded directly from the Enterprise Portfolio Management System; and information on timeline status, which is sourced from the program’s plan on a page.

  • ERP Internal Program Board
  • Enterprise Transformation Board
  • 1 Star Group
  • Defence Communications and Information Systems Committee
  • Enterprise Business Committee
  • Other forums are required as requested
  • Program Executive
  • Program Team

September 2019

Monthly Financial Status Report

Provides a short narrative on budget achievement, a diagram to demonstrate level of commitment against forecast budget, and a graph demonstrating year to date achievement against planned and forecast budget.

  • Enterprise Transformation Board
  • Defence Communications and Information Systems Committee
  • Enterprise Business Committee

March 2018

       

Note a: This was early in the program’s implementation, with the systems integrator and a new Program Director commencing in mid-2019.

Note b: KPIs include: benefits management, scope, schedule, resource, budget, security and dependencies. KPIs are measured with a status indicator (Red, Amber and Green), and do not involve performance targets.

Source: Defence records.

2.63 The following section examines the reports set out in Table 2.4. Reports were examined by the ANAO to determine whether the content was appropriate for supporting relevant decision-makers, including whether there was information on progress, risks and issues. The accuracy of the data used in the reports was not tested.

Weekly stream status reports

2.64 The ANAO examined examples of weekly stream status reports, including all 83 reports submitted in October 2020. The reports involve detailed stream level information, appropriate for the audience.

Weekly program status reports

2.65 The ANAO examined examples of weekly program status reports, including all nine reports submitted in September and October 2020. The reports involve a narrative update, along with a traffic light indicator on matters such as benefits management, scope, schedule, resource, budget, security, and dependencies. The reports provided management information to support the CIO in the role of Senior Responsible Officer.

Monthly program status reports

2.66 The ANAO examined the 18 monthly program status reports issued between October 2019 and April 2021. The reports examined provided: a timeline with key milestones, including IOC and FOC for Release 1A and Release 1B; a high level summary of current status; and a traffic light indicator of progress against schedule.

2.67 The ANAO observed that:

  • the timeline was often illegible in the format included in the monthly report57;
  • information on workstreams tended to be technical, with prevalent use of acronyms. Senior leaders would require detailed knowledge of the deliverables to interpret the report58; and
  • there was scope for improved information to assist the reader to understand whether deliverables and milestones have been delivered on time (further discussed in paragraphs 4.21–4.23).

2.68 The monthly program status reports reviewed by the ANAO initially presented only enterprise and program risks that have a residual ‘high’ and ‘extreme’ rating after treatment of the risk. As there were no residual ‘high’ or ‘extreme’ enterprise or program risks between July 2020 and January 202159, there was limited visibility of enterprise and program risks to senior leaders. From November 2020, Defence commenced reporting on enterprise and program risks with a ‘high’ and ‘extreme’ risk rating before treatment. This provides improved visibility of the program risk profile to senior leaders.

2.69 In January 2020 the Enterprise Business Committee was provided with a list of ERP program strategic risks separate to the monthly report. The committee noted the approach, agreed to the ERP related enterprise-level risks under active management, and agreed that Group and Service representatives continue to develop and transition relevant risks into their risk management practices.

Monthly financial status reports

2.70 The ANAO examined all examples of monthly financial status reports provided to the ETB. Between September 2019 and April 2021, financial reports were presented at 11 out of 17 meetings. The reports provide a summary of issues, and track spending against the program budget. They provide visibility to senior leaders of key issues and what has been spent.

Chief Information Officer Group Quarterly Performance Reports

2.71 The Chief Information Officer Group quarterly performance report commenced in June 2019, and followed the format of the Capability and Sustainment Group’s (CASG) quarterly performance report.60 The ANAO examined all six quarterly reports issued between June 2019 and September 2020. Each of these reports included the ERP program as a key ICT project, with information on risk (to capability, schedule and cost), progress against timeframes, and progress against budget presented. Defence advised the ANAO that recipients of the report include the Minister for Defence, the Secretary and Associate Secretary. The ERP program has used the narrative section of the quarterly report to present some additional information on risk.61

2.72 The report also includes categories for projects of interest and projects of concern.62 The ERP program has not been included in either category.

2.73 In the initial June 2019 quarterly performance report the ERP program was reported as ‘amber’ (some concerns, being managed, requires monitoring). In all subsequent reports the ERP program was reported as ‘green’ (on track to deliver approved scope). This included the September 2019 report, which presented the approved IOC date of 31 October 2020, and the forecast IOC date of 31 July 2022. At this stage, the revised (July 2022) IOC date was yet to be approved (see paragraphs 2.24–2.26 for further information on the change in IOC date). Quarterly reporting has been on hold since Quarter 4 2020. Defence advised the ANAO that a new ‘Capability Report’ is expected to commence in September 2021, with the aim of improving the timeliness and quality of advice to government.

Reporting to government

2.74 Defence also provides quarterly program updates to the Defence Minister and six monthly reports to the government. A number of quarterly reform priority reports have also been provided to government.

Quarterly program updates to the Minister

2.75 Quarterly reports were provided to the Defence Minister for: Quarter 3 2018; Quarters 1, 2, 3 and 4 2019; and Quarter 3 2020.63 On 17 April 2020, the ETB agreed that quarterly updates to the Minister would not be required in the quarters where the six monthly reports for the government were produced, and that the Quarter 1 update was not required. For that reason, there were also no Quarter 2 or 4 2020 reports to the Minister. Defence has further advised that there was no Quarter 1 2021 update as there was a six monthly report waiting to be considered by government during Quarter 1, and a change in the Minister. The new Minister commenced 30 March 2021.

2.76 The reports provided a high level summary of status, which did not include financial information. The reports presented limited information on progress against timeframes.

Quarterly reform priority reports

2.77 In December 2019, the Minister for Defence provided a letter to the Prime Minister with an update on Defence’s reform agenda, setting out the lines of effort and initiatives expected to build on the implementation of the First Principles Review. This reporting included the ERP program, which was said to:

… provide an opportunity to eliminate complicated and unnecessary structures, processes, systems and tools, and to replace them with a single, trusted source of accurate, near real-time information.

2.78 The Defence Minister received five quarterly reports on Defence’s reform program from Quarter 3 2019 to Quarter 3 2020. The ERP program was included in these reports from Quarter 1 2020 (three reports). The reports that included the ERP program advised that the overall program was ‘on track’, and that some activities within the program were ‘off track’ or ‘at risk’.

2.79 Quarterly reform priority reporting ended in Quarter 3 2020, with the agreement of the Defence Minister. Defence advised the Minister that new reform reporting arrangements would be considered for the Lead the Way: Defence Transformation Strategy64 launched on 27 November 2020. The ERP is not an area of focus in Lead the Way, although the strategy notes the importance of reforming processes ahead of future tranches of the ERP program to ‘obtain maximum value and benefit as early as possible’.

Six monthly reporting to government

2.80 At first pass, the government agreed that Defence would report to the Minister for Defence every four months, and provide routine reporting to the Secretaries’ Committee on National Security at key milestones. At second pass government agreed that Defence would report on progress quarterly to the Minister for Defence and six monthly to the government.

2.81 Six monthly reporting to government commenced in November 2019, with further reports provided in May and December 2020. The ERP program advised Defence senior leaders that the delay in commencing six monthly reporting was due to the timing of the 2019 federal election and changes to the Ministry.

2.82 Six monthly reporting includes a high level summary of progress, an attachment with further information on matters such as risk and schedule, and recommendations. In the three reports provided to the government as at December 2020, there was limited information on progress against budget.65 The three reports provided a statement as to whether the program was on track against agreed timeframes. While the timelines provided as part of the reporting did not set out Release 1A or Release 1B IOC or FOC dates, some of this information was included in the report narrative.66

2.83 The six monthly reports have included information on risk. Risk levels have largely remained stable, with the following exceptions:

  • cost risk increased from ‘medium-high’ to ‘high’ in November 2019. The relevant report stated that the reason was the re-baselining of IOC from late 2020 to mid-2022; and
  • schedule risk decreased from ‘high’ to ‘medium-high’ in November 2019. The reported reason was the re-baselining of IOC, with a further delay to IOC recommended by Defence in December 2020 to maintain a medium-high schedule risk.

2.84 Changes to the program’s risk profile over time are set out in Table 2.5 below.

Table 2.5: Risk changes since the second pass business case

Risk type

May 2018

November 2019

May 2020

December 2020

Overall Second pass

High

High

High

High

Technical

Low

Low

Low

Low

Cost

Medium-High

High

High

High

Schedule

High

Medium-High

Medium-High

Medium-High

Workforce

Medium-High

Medium-High

Medium-High

Medium-High

Implementation

High

High

High

High

Environmental

Low

Low

Low

Low

Cyber Security

Medium

Medium

Medium

Medium

         

Source: Defence records

2.85 The ANAO’s review of the risks reported to government compared to the risks reported internally (see paragraph 2.68) indicates that the risk profile presented to government is higher than the risk profile presented internally.

2.86 Defence advised the ANAO that it is aware that risk is not presented consistently in its internal reporting and its reporting to government, and advised that it had begun work on a solution. In May 2021 Defence provided evidence that it has included the risks reported to the government in EPMS, with links to corresponding program and stream level risks. Defence has also advised that these risks will be reviewed monthly as part of the regular monthly risk review sessions with ERP program senior leadership (discussed in paragraph 2.15) and directly extracted from EPMS for reports to the government. Defence should monitor its internal reporting and its reporting to government on ERP program risks, to ensure there is consistency in program risk reporting.

Recommendation no.1

2.87 The Department of Defence review arrangements for its internal reporting and its reporting to government on Enterprise Resource Planning program risks, to ensure there is the consistency in program risk reporting.

Department of Defence response: Agreed.

2.88 Defence agrees to recommendation.

3. Procurement

Areas examined

This chapter examines whether the Department of Defence (Defence) conducted an effective procurement process for Tranche 1 of the Enterprise Resource Planning (ERP) program that contributed to the achievement of a value for money outcome. In particular, this chapter focuses on Defence’s process to identify requirements, risks and the procurement approach for the key procurement in Tranche 1, the systems integrator, and the tender and probity processes applied by the department.

Conclusion

Defence conducted a largely effective procurement process for Tranche 1 of the ERP program, which included consideration of value for money. Two breaches of the Commonwealth Procurement Rules and two breaches of Defence policy were identified. A Defence Independent Assurance Review identified a range of issues requiring resolution in the tender process, resulting in tender decisions being set aside and additional steps being added to the tender evaluation process. Subsequently, IBM was identified as the preferred tenderer on the basis of offering best value for money.

Appropriate probity and conflict of interest arrangements were established to support Tranche 1 procurement activity. During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally.

Areas for improvement

The ANAO made one recommendation aimed at ensuring probity and on-boarding arrangements for the ERP program are appropriate.

The ANAO identified two areas for improvement to facilitate Defence’s compliance with procurement requirements related to record keeping and timely and accurate reporting on AusTender.

3.1 The second pass business case for Tranche 1 of the ERP program was approved by the government in June 2018. The approved funding for acquisition for Tranche 1 was $257.9 million (out-turned).67 Tranche 1 activities included the procurement of a systems integrator. Systems integration services are material to the successful delivery of Tranche 1 and represent more than 50 per cent of the total program budget from 2019–20 onward.

3.2 This chapter examines Defence’s procurement process for Tranche 1, with a focus on the systems integrator procurement. The ANAO examined whether Defence:

  • conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator;
  • conducted a tender process for the systems integrator that complied with relevant procurement requirements; and
  • established appropriate probity and conflict of interest arrangements for the systems integrator procurement activity.

Did Defence conduct an effective process to identify its requirements, risks and procurement approach for the systems integrator for Tranche 1?

Defence conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator selected for Tranche 1. Defence identified high level ERP system requirements prior to conducting a competitive Request for Tender process, and planned to refine its requirements through Offer Definition and Improvement Activity processes conducted as part of the tender process for the systems integrator.

3.3 To assess whether Defence conducted an effective process to identify its requirements, risks and procurement approach for the systems integrator function in the context of Tranche 1, the ANAO reviewed Defence’s identification of:

  • program level technical requirements and risks; and
  • requirements, risks and approach for procuring a systems integrator.68

3.4 Defence’s procurement of the systems integrator is discussed in paragraphs 3.24 to 3.57.

Initial identification of program-level technical and procurement requirements

3.5 Defence developed high-level requirements for the ERP program during the Program Definition Phase.69 These were drawn from sources including the Defence ERP Strategy and Roadmap, other ERP implementations70, the Defence Finance Systems Strategy, other Defence ICT programs that were planned or underway, and a modelling tool. Key documents developed by Defence that record the technical and procurement requirements identified for the ERP program include:

  • two ‘Business and Technical Architecture’ documents which define and describe the ERP solution (the Program Definition Phase Business and Technical Architecture document, and Business and Technical Architecture document); and
  • a Requirements Management Strategy which describes how the ERP program will collect, use and manage requirements to manage stakeholder needs and deliver program outcomes, and defines the role of requirements in supporting program decision-making.

3.6 These documents are discussed in more detail below.

Business and Technical Architecture documents

3.7 The Program Definition Phase Business and Technical Architecture document, dated 11 August 2015, was developed in the context of the pre-first pass business case Program Definition phase of the program. It states that it:

… defines and describes the Defence Enterprise Resource Planning (ERP) solution from a business and technical architecture perspective.

3.8 The document was developed based on the following inputs:

  • business reference architectures developed between February 2014 and May 2015;
  • stakeholder engagement across Defence;
  • use of a modelling tool as a baseline from which to develop Defence’s standardised business processes;
  • 14 process and functional requirements workshops held between 7 April and 11 May 2015 facilitated by an external consultancy (Deloitte), and 41 hours of stakeholder and subject matter expert consultation throughout Defence, to design and validate high level processes and associated requirements derived from the modelling tool baseline, and to identify related business requirements;
  • a collection of non-functional requirements for business processes developed through four workshops held between 14 and 20 May 201571; and
  • six end-to-end processes and two other process groups that were developed to provide a view of how the ERP would enable business processes across Defence.

3.9 Process and functional requirements workshops resulted in the development and validation of 70 high level business requirements associated with business processes, the capture of a repository of 1260 detailed business requirements for logistics and finance, and the development and validation of 47 high level non-functional requirements. The Business and Technical Architecture set out that future solution design work would include the development and validation of detailed business process and associated business and non-functional requirements.

3.10 The high level requirements were endorsed in principle by the ERP Reference Group72 on 8 May 2015. High level non-functional requirements were endorsed by the ERP Reference Group in July 2015.

3.11 The Business and Technical Architecture document, finalised on 23 November 2017, was developed as part of the second pass business case, as well as a supporting document for the Request for Tender for a systems integrator. Its stated purposes were as follows:

This document defines and describes the Defence Enterprise Resource Planning (ERP) solution from a business and technical architecture perspective. It establishes the scope of the ERP solution, identifying the key information elements, applications, and technology considerations to deliver the target state. It incorporates work previously completed by the Solution Definition Stream during the ERP Program Definition Phase updated by Defence’s Strategic Partner based on changes that have occurred since the document was originally published.

3.12 The document further outlined 12 program outcomes (discussed in paragraph 4.50) and outcome characteristics that were to provide the strategic requirements for the ERP Business Architecture. It also stated that the high-level capabilities of the ERP solution were to be further developed to drive development of business and non-functional requirements.

The Requirements Management Strategy

3.13 The Requirements Management Strategy was finalised on 4 December 2017. The strategy set out the pre-first pass requirements baseline from which initial scope boundaries for the program were identified: Finance; Logistics; Engineering; Maintenance; and Estate. The strategy set out that the program would inherit high-level requirements from the SAP Model Defence Organisation (MDO, see Appendix 3), which Defence would expand on through:

  • SAP’s development of the S/4HANA solution; and
  • the identification of gaps and changes in the processes when the SAP MDO was compared with how Defence conducts its business.

3.14 The requirements were to be mappable to one or more of the 12 program outcomes including outcome characteristics.73 Additionally, detailed requirements as needed in specific areas were to be developed during the Explore phase of Tranche 1.

3.15 Requirements were further refined through the tender process through Offer Definition and Improvement Activities (ODIAs) (discussed further below).

Identification of systems integrator procurement risks

3.16 Defence prepared a risk assessment specific to the systems integrator procurement as part of preparing an Endorsement to Proceed prior to the procurement. Once advised that an Endorsement to Proceed was not required74, these risks were integrated into the program’s risk and issue registers and managed in those documents. The register evidences that assessment of systems integrator procurement risks began at least as early as December 2015. The Endorsement to Proceed risk register also outlined risk treatments for each risk, the status of these, and the residual post-treatment risk assessment. Additionally, the concept procurement plan outlined how the procurement and contracting method would mitigate certain risks relevant to achieving a successful systems integrator procurement.

3.17 Defence maintained program risk and issue registers during the procurement, dating between July 2016 and March 2017. The registers included risks and issues specific to the systems integrator procurement, evidencing risk assessment and management activity by the program in relation to the procurement. These risks and issues were substantially similar to those outlined in the risk register prepared for the Endorsement to Proceed.

3.18 The 2017 RAIDD (Risks, Assumptions, Issues, Dependencies and Decisions) risk management strategy outlined processes for identification, analysis, allocation and treatment of risk applicable to the procurement. Risks identified during the 2018 Independent Assurance Review resulted in a Post-Tender Submission ODIA process and a second tender evaluation for the systems integrator procurement.75

Development of the procurement approach

3.19 The procurement approach for the systems integrator was developed and refined through several key program documents prior to and during the procurement, as set out below in Table 3.1 below. Initial development of the approach in 2015 was undertaken by external consultants (Deloitte) as a strategic partner, with work continued by the ERP Implementation Office.

Table 3.1: Key procurement planning documents

Document

Date issued

Option/s considered

Option/s chosen/recommended

Enterprise Resource Planning Program Definition Phase Acquisition (Sourcing) Strategy Document

 

14 August 2015

  • Managed services; strategic partner; network prime, in-house and augmentation; and in-house acquisition options.
  • Build internal capability, leveraging existing panel arrangements, direct acquisition and open approaches to market as market engagement options.
  • Contracting options: open book; gain and risk sharing; pay for performance; and fixed price.a
  • Network prime or self-prime acquisition approach, to be further investigated through market testing activities.b
  • Open tender approach to market, including developing a market testing framework; issuing an Invitation to Register (ITR); conducting workshops with shortlisted providers from the ITR process; revising the procurement approach for a restricted pre-qualified Request for Tender (RFT), and issuing a restricted RFT to shortlisted providers.
  • Gain and risk sharing and pay for performance contracting approaches.

System Service Concept Procurement Plan–Phase One

30 September 2016

  • Market engagement options: open approach; selective approach (including qualifying process); joint solutioningc open approach; joint solutioning selective approach; joint solutioning open approach with request for tender; and joint solutioning selective approach with request for tender.
  • Network prime and self-prime acquisition options.
  • Panel and long-term contracting options.
  • Self-prime delivery with Defence maintaining program accountability and control and managing up to two systems integrators contracted to deliver work packages, supported by Strategic Partner.
  • Open approach including joint solutioning (three stages – ‘capability fit’, ‘solution option’, and ‘commercials and pricing’) activities with shortlisted vendors, pre-First Pass market briefing; market sounding activities; an Invitation to Register process; best and final offer; and contract negotiation.
  • Long-term incremental contracting.

Procurement Plan

28 July 2017

  • N/A
  • Procurement approach including initial RFT, two ODIA processes (technical: capability fit and solution option; and commercial: commercials and pricing), and final RFT refined through the ODIA.
       

Note a: Open book refers to price being dependent on cost of delivery plus a fixed or known fee or margin. Gain and risk sharing refers to allowable costs plus an agreed fee adjusted to reflect the ‘pain/gain’ share mechanism. Pay for performance refers to a fixed price being paid incrementally upon successful completion of milestones. Fixed price refers to an unalterable price being paid for a strictly defined scope of work.

Note b: The network prime delivery model involved Defence engaging with specialist providers and selecting one as the prime. The self-prime model involved self-delivery primarily utilising internal capabilities, augmented by selected providers as required, with advice and guidance provided by the Strategic Partner.

Note c: As noted in paragraph 3.21, joint solutioning refers to Offer Definition and Improvement Activities.

Source: ANAO analysis of Defence procurement planning documentation.

3.20 Defence conducted market testing activities between December 2015 and January 2016 and held a market briefing on 17 December 2015. Defence published a follow-up AusTender notice on 21 December 2015, which included the slideshow from the 17 December 2015 briefing, a Market Briefing Information Pack for potential suppliers, and the answers to questions raised during the market briefing. On 13 January 2016 Defence published a Market Sounding Questionnaire on AusTender seeking insight and feedback from the market as it related to ideas and concepts described in the Market Briefing Information Pack. An August 2017 internal audit noted that there was no evidence that feedback from the Market Sounding Questionnaire had been incorporated into the development of the procurement approach.

3.21 The procurement approach chosen for the systems integrator was an open tender, involving an Invitation to Register, followed by a shortlisting process and a Request for Tender (RFT) process with the shortlisted vendors. Shortlisted vendors would engage in competitive ODIAs conducted in parallel (described as ‘joint solutioning’).

3.22 Defence planned to adopt a ‘long-term incremental contracting’ approach through a Deed of Standing Offer and work packages contracted under Official Orders.76 Defence outlined in the System Service Concept Procurement Plan–Phase One that the rationale for long-term incremental contracting was based on:

a. Providing Defence with greater negotiation leverage due to the long term nature of the agreement without lock-in, while encouraging System Integrator(s) to offer the greatest value due to increased likelihood of securing future increments (capability drops).

b. Reducing ongoing procurement effort and complexity as primary and secondary System Integrator(s) are pre-qualified in contrast to Panel [Statement of Work] arrangements.

c. Effectively managing risk associated with acquiring the ERP system where the risk of acquiring the entire functionality in one step is considered unacceptable.

d. Enabling Defence to manage its risk exposure because its commitment is limited to the current set of capability drops.

e. Facilitating agile development and acquisition.

f. Aligning Defence’s acquisition and contracting process to the System Integrator(s)’s development processes.

3.23 The August 2017 internal audit found that the program’s proposed sourcing model was adequate, with governance measures and controls in place to ensure alignment with the Commonwealth Procurement Rules (CPRs) and the Defence Procurement Policy Manual.

Did Defence conduct a tender process for the Tranche 1 systems integrator that complied with relevant procurement requirements?

Defence conducted a competitive tender process for the systems integrator that largely complied with the formal requirements of the Defence Procurement Policy Manual and Commonwealth Procurement Rules. A 2018 Defence Independent Assurance Review identified a range of issues requiring resolution in the systems integration tender process, resulting in tender decisions being set aside and additional steps in the tender evaluation process. When the tender process was restarted, additional offer definition and parallel negotiation activities were undertaken to address risks identified through the review. A new tender evaluation board prepared a second Source Evaluation Report, which identified IBM as the preferred tenderer on the basis of offering best value for money.

Instances of non-compliance with the Commonwealth Procurement Rules include that AusTender reporting was seven days late, and there is no record of consideration of environmental sustainability. There were also two identified instances of non-compliance with financial delegations and Defence financial policy related to the commitment of public money.

Tender process

3.24 Defence shortlisted potential vendors for the provision of systems integrator services through an open Invitation to Register (ITR) process conducted between May 2016 and April 2017. The ITR opened on 20 May 2016 and closed on 23 June 2016.

Invitation to Register Statement of Requirement

3.25 The Statement of Requirement included in the Invitation to Register (ITR) documents outlined that Defence intended to engage two systems integrators to deliver systems integration services using a consolidated, predominately SAP platform with integration to a limited number of ‘bolt-on’ applications where required. Defence stated that the scope of services would be refined during the Offer Definition and Improvement Activity stages of the Request for Tender selection process described in the Statement of Requirement. The proposed allocation of functional capability to three planned tranches was set out in attachments to the Statement of Requirement, with eight end-to-end processes to be enabled by the tranches.77

3.26 Defence received six responses to the ITR, two of which were excluded due to non-compliances with the conditions of registration. The remaining four responses were assessed against evaluation criteria included in the ITR documents published on AusTender. The assessment process resulted in Accenture and IBM being shortlisted.78

3.27 An initial RFT79 was released on 31 July 2017 to the two shortlisted tenderers, following which three Offer Definition and Improvement Activities (ODIA) processes were conducted between 4 September and 27 October 2017. Program level and systems integrator requirements were further developed and refined through the ODIAs.

Refinement of requirements through Offer Definition and Improvement Activities (ODIAs)

3.28 ODIA workshops were attended by program staff, representatives of the shortlisted tenderers, Defence subject matter experts, the Organisational Change Management Partner (an external consultant, Ernst and Young), the Strategic Partner (an external consultant, KPMG Australia), as well as the Probity Advisor (Maddocks, a law firm) when necessary.80

3.29 Through ODIA 1, Defence considered that it identified functional areas (transformation targets) that were fundamental to how Defence operates and had the potential to provide ‘extensive benefits’ if transformed. A report detailing ODIA outcomes was produced for each ODIA.

3.30 ODIA 2 included discussion of requirements in relation to rolling out Initial Operating Capability (IOC) and Final Operating Capability (FOC) and Defence’s requirements relating to transition and support. During ODIA 2, the ‘transformation targets’ were used to develop ‘outcome characteristics’ for the program, which when finalised would describe the end state that Defence expected the program to deliver. The outcome characteristics were to be used to: help develop Defence’s requirements; advise the process of identifying risks, benefits, business and organisational change; and develop the list of prerequisite tasks that Defence needed to action to enable the characteristics to be delivered. The ODIA 2 Report noted that the program intended to continue to engage with subject matter experts that attended the workshops as the characteristics were refined and broken into more precise requirements statements.

3.31 Prior to and in parallel with ODIA 2, Defence developed and refined a draft statement of requirement (scope of services) document, which was included in an updated RFT issued in October 2017.81 Responses were received on 22 November 2017.

3.32 At this stage it was envisioned that work packages would be divided between the two preferred tenderers on the basis of three tasking statements, as well as the Transformation Targets. The source evaluation report, finalised by the tender evaluation board on 19 January 2018, recommended that IBM be selected as preferred tenderer for Tasking Statement 1 and the Transformation Targets, IBM and Accenture be selected for Tasking Statement 2, and neither tenderer be selected at that time for Tasking Statement 3.82 Subsequently on 31 January 2018, at the request of the delegate (the ERP Program Manager), the tender evaluation board met with the delegate, who determined that neither tender be appointed for the Transformation Targets at that time.

3.33 On 26 March the Associate Secretary appointed the CIO as the new delegate for the systems integrator procurement. On 18 March 2018, the CIO asked the program to put additional scenarios to the tenderers to assist in making a decision on the appointment of a systems integrator to the program.83 On the basis of the tenderer responses and advice received from the Program Manager on 23 May 2018, the new delegate determined on 31 May 2018 that Accenture be appointed to the program as systems integrator for Tasking Statement 1, with a decision on appointing a preferred tenderer(s) for Tasking Statements 2 and 3 and the Transformation Targets deferred to a later date.

3.34 On 17 July 2018 (prior to the appointment of a systems integrator) an Independent Assurance Review (2018 IAR) of the program as a whole commenced. The review stated that it was ‘an appropriate time, immediately following second pass approval and while essential contractors are being engaged and detailed plans are being developed, to review the overall Program strategy, planning and resourcing, to ensure that it is best placed to deliver successfully in accordance with Government decisions and Defence expectations.’ The systems integrator selection process was considered by the review. Engagement with the tenderers was suspended pending the outcome of the review. On 1 August 2018, the reviewers provided a minute to the acting Associate Secretary recommending all procurement decisions and recommendations to that date be set aside and that a lead negotiator, new Tender Evaluation Board and new delegate be appointed. The recommendation was based on:

  • a ‘significant number of non-compliances, remaining uncertainties and areas requiring negotiation’ in the tender responses84;
  • ‘issues that have arisen during the [systems integrator] source selection process’ and ‘the varying judgements made about the preferred way ahead’; and
  • the reviewers’ view that the source selection ‘would best be made on a more solid and informed basis than is available from the material received to date from the tenderers.’

3.35 Further, the IAR reviewers observed more generally that ‘there is a need to schedule a formal Assurance Board meeting, to address a number of elements of the Program’s organisation, strategy, resourcing and governance.’

3.36 On 27 August 2018 the acting Associate Secretary formally agreed85 that the Chief Financial Officer (CFO) would be the new delegate and agreed to set aside all decisions and recommendations to date. The ERP Program Manager was tasked with actioning the recommendations.

Restarting the tender assessment

3.37 On 18 September 2018, the new delegate (CFO) determined that:

… neither tenderer’s offer, in its current form, is capable of being accepted without exposing the Commonwealth to unacceptable technical, delivery, financial and commercial risks and therefore neither offer currently represents value for money to the Commonwealth.

3.38 The delegate decided that the program would undertake a Post-Tender Submission (PTS) ODIA and parallel negotiations with the tenderers. PTS ODIA workshops were conducted between 26 November 2018 and 31 March 2019. The aims of the workshops were to:

a. address the solution issues identified through the evaluation of the RFT and drive down assumptions;

b. workshop the commercial non-compliances raised by IBM and Accenture in their 2018 tender responses and collaboratively develop contractual arrangements with the tenderers to reflect both Program evolution and solution workshop developments; and

c. workshop the pricing models to be applied across the Program and undertake a financial investigation of the price build up, including the build-up of labour rates.’

3.39 A Request for Updated Tender was released on 1 April 2019, closing on 30 April 2019.

3.40 A second source evaluation report, based on the updated tenders provided, was approved by the delegate on 4 June 2019. The report recommended that IBM be selected as the preferred tenderer for all tasking statements, but Accenture was not to be set aside at that stage, pending successful negotiations with IBM. IBM was considered to represent the best value for money based on its response against both the solution and financial evaluation criteria. There was no differentiation between tenderers against the commercial evaluation criteria.

3.41 A Deed of Standing Offer was signed with IBM on 19 July 2019, commencing on 19 July 2019 and expiring on 18 July 2025. On 19 July 2019 Defence signed an Official Order for the ‘Prepare’ and ‘Explore’ phases of Tranche 1 with IBM, valued at approximately $95.5 million, with the arrangement commencing on 22 July 2019. Defence outlined in its section 23 commitment approval documentation86 a rationale for the value for money expected to be achieved. The rationale focussed on IBM’s experience in delivering similar programs; the skills of key people; how costs would be managed including the agreement for fixed price element and for benchmarking and open book pricing on other elements; as well as incremental work package contracting and Defence’s contractual right to terminate the agreement and engage a third party for the remainder of the program.

Defence Procurement Policy Manual Compliance

3.42 Defence was largely compliant with Defence Procurement Policy Manual requirements, with the exception of an identified instance of non-compliance with financial delegations and Defence policy designed to meet the requirements of the Public Governance, Performance and Accountability Act 2013 (PGPA Act): subsection 23(3) delegation; and subsection 23(1) delegation.87

3.43 The Defence Procurement Policy Manual requires that these delegations are exercised in the following order, unless the procurement does not involve the commitment of relevant money in which case only the enter into an arrangement delegation is required:

  • subsection 23(3) — commitment approval; and
  • subsection 23(1) — enter into an arrangement.

3.44 The Defence official who signed the section 23 commitment approval documentation for Official Order 1 on 18 July 2019 was acting in the role of Program Manager ERP, and did not have the necessary delegation to approve the commitment of $95,413,351.37.

3.45 Defence advised the ANAO that between 10 July and 18 July 2019 (inclusive), the role of Program Manager ERP was an O6/EL2 equivalent position, with a standard Commitment Approval delegation of $5 million as per the Department of Defence Public Governance, Performance and Accountability Delegation (FINMAN 2). The purported exercise of the delegation was contrary to the Defence Procurement Policy Manual and the relevant version of Accountability Authority Instruction 15, which stated that Defence officials:

… must not exercise any of the financial delegations in FINMAN 2 unless they are a person holding, or are for the time being acting in, or have been directed to perform the duties of a position or class of positions listed in the relevant delegations schedule …

… must comply with the terms and conditions of the relevant delegations schedule in FINMAN 2 and … must not depart from any requirements, directions or limits set out in FINMAN 2.

3.46 The delegations issue was brought to Defence’s attention during the course of this audit. In response, Defence completed an investigation of the exercise of delegations in the ERP program. Defence advised the ANAO that no further instances of improper exercise of delegations have been identified.

3.47 The Defence official (the CIO) who subsequently entered into the financial arrangement with the systems integrator, did have the necessary delegation to both commit the Commonwealth to expenditure under subsection 23(3) and enter into an arrangement under subsection 23(1). However, FINMAN 2 sets out that the subsection 23(1) delegate must not enter into an arrangement that commits relevant money unless it has been approved by a subsection 23(3) commitment approver. Accordingly, the invalid exercise of the commitment approval delegation meant that the requirements of FINMAN2 and Accountable Authority Instruction 15 with respect to the ‘enter into an arrangement’ delegation were not satisfied.

3.48 An additional breach of Defence policy was identified during the course of the audit. FINMAN 2 sets out that a subsection 23(1) delegate must not enter into an arrangement that commits relevant money if the value of the arrangement exceeds the amount approved by the subsection 23(3) delegate. The commitment approval for IBM Official Order 3, signed on 19 June 2020, does not provide sufficient funds for the maximum contract value, with $263,963.07 less than the total contract price estimate of $4,591,282.17 approved, resulting in a breach of FINMAN 2 and Accountable Authority Instruction 15.

Commonwealth Procurement Rules compliance and AusTender reporting

3.49 Defence has largely complied with the mandatory requirements of the Commonwealth Procurement Rules (CPRs) during the systems integrator procurement process, with two exceptions.

3.50 The exceptions were: there was no evidence of consideration of the environmental sustainability of the proposed goods and services; and Official Order 3 was published on AusTender seven days after the required date.

Consideration of financial and non-financial costs

3.51 The CPRs require officials, when conducting a procurement, to consider the relevant financial and non-financial costs and benefits of each submission including, but not limited to the environmental sustainability of the proposed goods and services (CPR 4.5). There was no evidence of consideration of the environmental sustainability of the proposed goods and services.88

3.52 In relation to the requirement to consider relevant financial and non-financial costs and benefits of tender submissions when assessing value for money, the ANAO sought advice from the Department of Finance (as the CPR policy owner), which advised that the following interpretation was consistent with the intention of the CPRs:

The statement of the section notes that “an official must consider the relevant financial and non-financial costs and benefits of each submission including, but not limited to…” and then proceeds to list the six conditions that are to be included. The important point is that officials are directed that they must consider these costs and benefits.

As they must be considered, it is a reasonable assumption that to comply with record keeping requirements of how value for money was considered and achieved (CPR 7.2 & 7.3.c) for a procurement, a documented consideration in some form should be kept even if to state that in a particular case a condition was not relevant in determining value for money.

3.53 Defence guidance reviewed by the ANAO did not include instructions for Defence officials regarding record keeping expectations for requirements that were considered but determined to not be relevant to the particular procurement. To ensure a complete record of decisions made, there is scope for Defence to improve its guidance for officials on how to document decisions about CPR requirements that are considered or determined to be not relevant to a particular procurement activity.

Reporting on AusTender

3.54 The Commonwealth Procurement Rules require that all contracts or amendments by non-corporate Commonwealth entities (such as Defence) at or above $10,000 be reported on AusTender within 42 days of entering into (or amending) a contract (CPR 7.18–19).

3.55 Official Order 3 was published on AusTender seven days after the required date. Defence advised the ANAO that this was due to a technical issue with the automatic upload of information to AusTender from ROMAN.89 The program had completed all documentation to enable timely AusTender reporting.

3.56 Further, ANAO analysis of AusTender entries during the audit identified a number of additional issues with Defence’s AusTender reporting:

  • dates were incorrect, with the start dates recorded in AusTender for Official Order 2 and variations 3, 4 and 6 to Official Order 1 preceding the execution dates. Defence advised the ANAO that the contract and amendment start dates in AusTender were erroneous;
  • the total contract value for Official Order 1 in AusTender did not reflect all variations to date, with the value in AusTender being $254,908.24 higher than Defence’s records90; and
  • inaccurate values were reported, including the value of Official Order 3 reported on AusTender being $263,889.87 lower than the Official Order.

3.57 For the purpose of providing timely and accurate reporting on AusTender, Defence should consider reviewing its system(s) for uploading data to AusTender and undertake quality checking of reported data.91

Did Defence establish appropriate probity and conflict of interest arrangements for the Tranche 1 systems integrator procurement activity?

Defence established appropriate probity and conflict of interest arrangements for Tranche 1 procurement activity. The arrangements included the appointment of a probity advisor, the development of a probity framework and plan, and the conduct of probity briefings. Documented probity requirements were largely adhered to, with the following exceptions: there was no record of conflict of interest declarations in the program register for three of 50 personnel involved in procurement decisions; and personnel were not asked to sign a copy of the probity framework although the policy at the time required this.

During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally, and which require attention. These issues pertained to the management of: conflicts of interest; use of panel arrangements for this program; gifts and hospitality; and the use of official information.

Probity advice

3.58 Defence appointed an external probity advisor (Maddocks, a law firm) for the ERP program on 15 December 2015, with the procurement taking place through the Defence Professional Services Standing Offer Panel. There are four contracts for these probity advice services disclosed on AusTender, dated 10 March 2016 to 30 April 201692, 31 May 2016 to 30 September 2016, 25 July 2017 to 30 June 18, and 1 July 2018 to 30 June 201993, which cover the entire period that Defence undertook systems integrator tender processes.

3.59 There is evidence that the probity advisor conducted 23 probity briefings between December 2015 and May 2019. While attendance records cannot be located, Defence has advised that probity briefings were held for all evaluation team members. The advice and recommendations provided by the probity advisor during the procurement are recorded in the conflict of interest register discussed at paragraph 3.62.

Probity framework and plan

3.60 The ERP probity framework was developed by the probity advisor. It was issued in January 2016, revised in September 2018 and updated in 2020. The framework states that it applies to:

… personnel who may be involved in [the program] or who may have access to information relating to [the program], including: APS employees; Australian Defence Force personnel; and any Advisers or consultants (including sub-contractors) to Defence working on, considering, or providing services in relation to [the program].

3.61 A probity plan, intended to build on the probity principles in the framework in relation to specific issues, was first issued in June 2016 and revised in September 2017. The plan outlines communications protocols and business as usual protocols and also includes protocols for:

  • tender evaluation;
  • gifts and hospitality;
  • offers of employment;
  • business meetings and social functions; and
  • confidentiality and information security.
Implementation of the probity framework and plan

3.62 A conflict of interest register and a contact register were established, in line with the 2016 probity framework requirement.94

3.63 The 2018 version of the framework set out that all program personnel were to be provided with a copy of the probity framework. Personnel were required to:

  • complete a confidentiality acknowledgement and a declaration of interests form; and
  • return a signed copy of the probity framework acknowledging they have read and understood their obligations, roles and responsibilities under the framework.

3.64 The probity contact officer was responsible for maintaining a register of acknowledgements of the probity framework and a register of confidentiality agreements. Defence advised the ANAO that rather than requiring staff to return the signed probity framework, personnel were agreeing to the probity framework and confidentiality agreement by signing the conflict of interest declaration.95 The change in practice was not reflected in the probity framework in place during the procurement process. The probity framework was further updated in October 2020 (during the course of this audit) and reflects the change. Personnel are now required to read, understand and comply with the framework, rather than sign and return it.

3.65 The ANAO’s review of the conflict of interest register showed that of 50 project personnel with decision-making roles, three personnel were not listed in the conflict of interest register. The three personnel were members of evaluation working groups and the tender evaluation steering group. During the course of this audit, Defence provided the ANAO with: statutory declarations from the two tender evaluation working group members stating that declarations were completed and no conflicts were declared; and a declaration of interest form from the tender evaluation steering group member, which listed interests which were not relevant to the ERP program.

3.66 Of the 50 project personnel with decision-making roles, ten registered a conflict. For nine of these there was a documented strategy to manage the conflict. There was no management strategy for the tenth, as the declared conflict of interest related to a different procurement, in which the individual was not involved.

Other probity issues

3.67 The ANAO’s review focussed on probity in the procurement process for the Tranche 1 systems integrator, as discussed above. During the course of the audit, a number of specific probity issues were identified which relate to the management of probity in the program more generally. These issues pertained to the management of: conflicts of interest; use of panel arrangements for this program; gifts and hospitality; and the use of official information.

Conflict of interest
Chief Information Officer

3.68 On 23 October 2019, there was questioning in the Senate Foreign Affairs, Defence and Trade Legislation Committee regarding an allegation that the son of Defence’s CIO worked at a company (Sinapse Pty Ltd) contracted by Defence.96 The concern was that the CIO had a financial delegation for the letting of the contract. The CIO was present at the hearings and stated that although there had been no financial benefit for his son, ‘On reflection, yes. I think there was a chance of a perceived conflict of interest. I do acknowledge that’. When asked about the nature of the capability, product or service provided by Sinapse Pty Ltd, the CIO stated that:

It was a range of services, but predominantly with respect to an ability to undertake an independent assurance review around the ERP program, with particular respect to organisational change management plans.

3.69 The ANAO reviewed the contract signed by the delegate (the CIO) and Sinapse Pty Ltd on 19 December 2018. Under the contract, Sinapse was engaged to undertake ‘an independent expert assessment of the Enterprise Resources Planning Program Organisation Change Management approach’. Defence advised the ANAO that from 26 August 2019, Sinapse has also been engaged under the 2017 Ernst & Young Standing Offer (through three official orders) as an approved contractor for the purposes of ‘Change Leadership, Business Adoption and Stakeholder Management & Communications and Engagement work streams’.

Senior ERP program contractor

3.70 In September 2020, a potential improper use of position and conflict of interest issue in the ERP program was brought to the attention of Defence senior leaders. The issue involved an allegation that a senior ERP program contractor had discussed their partner’s employment situation with another contractor in the program. Subsequent to the discussion, the senior ERP contractor’s partner was employed by that contractor’s company. This resulted in an investigation by Defence’s Audit and Fraud Control Division. The investigation identified that there had been a breach of Defence policy by the contractor through a failure to identify and declare a perceived conflict of interest, as required by the contractor’s contract. The investigation found that the contractor’s conduct:

… can be described as naïve, technical breaches of Defence policy, however of a low level of seriousness and lacking intent or malice.

3.71 The investigation identified that the key contributor to the breach occurring was a lack of support through induction and training, and that there was a need to ensure contractors are educated to have a full understanding of Defence policy and governance requirements.

3.72 Subsequent to the investigation, the individual concerned completed a new conflict of interest declaration. However, this was not completed until five months after the investigation concluded, and the declaration had not been registered in the Conflict of Interest declaration log for the ERP program provided to the ANAO by Defence on 25 May 2021.

3.73 The ANAO also identified that this individual was a company director of a SAP consulting firm, and that the directorship had not been documented in Defence’s conflict of interest documentation provided to the ANAO. Defence advised the ANAO in July 2021 that it had taken action on this matter.

3.74 On 24 May 2021, Defence advised the ANAO that contractors to Defence did not have a conflict reporting requirement other than what was included in a contract and those related to specific projects. Defence further advised that the Chief Information Officer Group has developed a new contractor integrity framework. The framework was approved by the CIO on 1 July 2021 and issued as a Chief Information Officer Directive, applicable to all Chief Information Officer Group personnel. The framework requires contractors to complete:

  • a ‘contractor conflicts deed’ at the commencement of the engagement with Defence;
  • a ‘contractor conflicts declaration’ at the commencement of the engagement with Defence, at each anniversary of the commencement of the engagement, at any time there is a concern that a conflict of interest may, or has arisen, and where there is a material change in the engagement; and
  • the ‘Contractor Conflicts Training Module’ within one month of the commencement of the engagement with Defence and then annually.
Use of panel arrangements

3.75 A panel arrangement is ‘the end result of a procurement process, where a number of suppliers are appointed through a contract or deed of standing offer.’97 Where there is a panel arrangement, procurement can take place directly with any supplier on the panel.

3.76 ‘Above the line’ contractors in the ERP program98 are largely recruited through labour hire firms included on Defence procurement panels. The ANAO identified instances of individuals being recommended for a role or roles in the ERP program by a program contractor they had previously worked with. The available evidence indicates that these individuals were subsequently matched by the program with labour hire firms appearing on Defence procurement panels, and subsequently engaged as contractors on the program through those firms. This approach reduces competition for roles in the program.99 Paragraph 5.1 of the 2020 Commonwealth Procurement Rules states that:

Competition is a key element of the Australian Government’s procurement framework. Effective competition requires non-discrimination and the use of competitive procurement processes.

3.77 Auditor-General Report No. 4 2020–21 Establishment and Use of ICT Related Procurement Panels and Arrangements noted that:

Panels are a common procurement mechanism in the Australian Government sector and procurements from panels and similar arrangements are often perceived as requiring less time and effort, particularly when compared to the cost and time required to undertake an open approach to market. However, panels should not be used to eliminate competition. In addition … the CPRs state that procurements from standing offers (which most panels are) are not subject to the rules in Division 2 of the CPRs. As the Division 2 rules (which require high value procurements to be available to the open market unless certain conditions are met) no longer apply, buyers can approach a single provider from a panel and still comply with the CPRs. This occurs irrespective of the procurement value, provided the officials responsible for the procurement are satisfied, after reasonable enquires, that the procurement achieves a value for money outcome. It is essential to bear in mind that when using a panel, each use of the panel is a separate procurement and entities need to ensure they adopt processes that are not only technically compliant with the CPRs but are also consistent with the intent of the CPRs, which is to drive value for money through competition…100

3.78 Defence advised the ANAO in July 2021 that:

Defence is undertaking a review into its procurement, commercial and financial practices within the Enterprise Resource Planning Program. This has identified areas for improvement and we will capture lessons learned that can be applied to other procurement/commercial and financial activities across the department.

Defence takes every opportunity to openly compete contracted work. Sometimes this requires approaching the market using alternative methods. In this instance, suitable candidates were located from across Australia with significant experience in the delivery of an extremely complex ERP Program. The category of SAP skills required are limited in the marketplace as it is a highly contested segment, with few suitably experienced resources to assist in the successful delivery of the program.

Undertaking this procurement approach means that the Department placed more weight on capability, than any other criteria, in the delivery of contracted services. The resources selected are highly regarded in their field of expertise. The procurement approach significantly reduced both delivery and financial risks. This is in line with the Commonwealth Procurement Rules, where both financial and non-financial benefits/risks must be taken into account in any Commonwealth purchasing decision.

Gifts and hospitality

3.79 Defence’s Financial Policy on Gifts and Benefits sets out that offers of hospitality can only be received:

… if it assists Defence to develop and maintain constructive relationships with stakeholders and does not give rise to either an actual, potential or perceived conflict of interest or compromises the reputation of the Australian Government, the APS or Defence.

3.80 As the ERP Program Management Office is responsible for managing contracts, accepting hospitality from a vendor can give rise to a conflict of interest. This risk arises for both Defence personnel and contracted program personnel. The policy requires gifts and benefits, including hospitality, to be recorded on the Defence Gifts and Benefits Register, excluding ‘working meals’.101 The policy does not require declarations of offers of hospitality that have been declined.

3.81 An extract of Defence’s Gifts and Benefits register, provided to the ANAO on 24 May 2021, documents eight instances of hospitality received by program personnel from vendors between August 2019 and February 2021.

3.82 The ANAO’s review of Defence records identified a further 10 instances of program personnel being invited to lunch, dinner or drinks by a vendor, on dates other than those recorded in the register. In respect to these 10 instances, Defence advised the ANAO that:

  • in one instance a contractor provided hospitality estimated at under $100 for each member of Defence personnel in attendance. Permission was granted by the CIO, and there was an oversight in recording this event in the Defence Gifts and Benefits register;
  • in one instance a contractor provided hospitality estimated at under $100 for each member in attendance. Defence advised that there was a misinterpretation of Defence policy, and personnel involved did not believe the instance required reporting in the Defence Gifts and Benefits register;
  • in three instances no hospitality was offered or provided to Defence personnel, with individuals paying for themselves; and
  • in five instances records do not confirm whether the event took place.
Use of official information

3.83 In the course of reviewing Defence documentation the ANAO identified a number of instances of senior program personnel sending Defence information relating to the ERP program from their Defence email accounts to external email accounts.

3.84 The ANAO analysed the email caches of senior ERP personnel and identified instances where information was not handled in accordance with Defence requirements.

  • The ANAO identified 423 emails, up to the classification of ‘protected’, sent from the Defence accounts to non-defence email accounts.
    • Emails were sent to their personal, business and/or other government entity email accounts.
  • The emails related to the monitoring, reporting and operations of the program, as well as some documents relating to technical architecture.
    • Three emails related to another Defence ICT program.
    • Most of the emails included attachments.
    • The misuse of security classifications in a number of emails enabled them to be sent to an external email account.

3.85 For the analysed senior ERP personnel, the ANAO identified 29,961 instances of program personnel transacting or being transacted with using external or personnel email accounts rather than their Defence email accounts.

3.86 The ANAO brought these matters to the attention of Defence’s Audit and Fraud Control Division in May 2021 and provided an update in June 2021. Defence advised the ANAO in July 2021 that it had taken follow-up action.

3.87 The probity issues discussed in this section relate to the management of probity in the program as a whole. Defence should review its probity arrangements for the ERP program and apply lessons learned to similar programs.

Recommendation no.2

3.88 The Department of Defence:

  1. review its probity arrangements for the Enterprise Resource Planning program, particularly with respect to its use of contractors, and apply lessons learned to similar programs; and
  2. develop more robust processes for on-boarding contractors, including ensuring awareness of probity and information security requirements.

Department of Defence response: Agreed.

3.89 Defence agrees to recommendation.

4. Contracting

Areas examined

This chapter examines whether the Department of Defence (Defence) established fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the Enterprise Resource Planning (ERP) program. In particular, the chapter examines: the inclusion of authorised negotiation outcomes in the systems integrator contract; milestones and performance expectations in the contract; contract governance arrangements; and the contribution of Tranche 1 to program objectives and Defence’s strategic priorities.

Conclusion

Defence established largely fit-for-purpose contracting arrangements that support the achievement of Tranche 1 outcomes and its strategic priorities under the ERP program. While Defence has developed an appropriate contract with the systems integrator, program and contract governance has been undermined by conflicts of interest embedded in decision-making arrangements, with contractors involved in decision-making relating to their contracts.

As of May 2021, the ERP program remained at an early stage of implementation and substantial work remains for Defence to fully implement the program.

Areas for improvement

The ANAO made three recommendations aimed at improving clarity and consistency in guidance documentation, the avoidance of conflicts of interest in decision-making, improving arrangements for timesheet approval, and the timely development of benefits realisation plans for each Tranche of the ERP program.

The ANAO also identified that contracting arrangements could be improved by: regular quality assurance of data in the deliverables tracker, as this data is used for monitoring and reporting; and inclusion of deliverable due dates and delivery dates through the ERP program monthly report.

4.1 The First Principles Review: Creating One Defence was released publicly by the Minister for Defence in April 2015. One of the First Principles Review’s six key recommendations was to ‘fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness.’ Robust contract management arrangements are important in ensuring this strategic priority and ERP program objectives are achieved.

4.2 This chapter examines Defence’s contracting arrangements for Tranche 1 of the ERP program, with a focus on the systems integrator procurement and the achievement of program objectives and Defence strategic priorities. The ANAO examined whether:

  • the systems integrator contract reflects Defence’s authorised negotiation outcomes;
  • Defence has developed contractual arrangements for systems integrator services with clear milestones and performance expectations;
  • Defence’s contract and program governance arrangements enable monitoring, management and reporting on the achievement of Tranche 1 outcomes; and
  • Tranche 1 is contributing to ERP program objectives and Defence strategic priorities.

Does the systems integrator contract reflect Defence’s authorised negotiation outcomes?

The contractual arrangements that Defence entered into with the systems integrator in July 2019, which included a Deed of Standing Offer and initial Official Order, reflected all relevant authorised negotiation outcomes. Defence achieved its preferred position or better in respect to 51 negotiation outcomes and its minimum fall-back position (identified prior to negotiation) for the remaining 12 negotiation outcomes.

4.3 The Contract Negotiation Directive for the systems integrator procurement was approved by Defence’s Chief Information Officer (CIO) on 11 June 2019. The directive authorised a lead negotiator (Ngamuru Advisory), whose services were procured from a Defence panel arrangement for negotiation services, to negotiate the appointment of a systems integrator to deliver the ERP program. The objective of the negotiation was to finalise, and recommend to the delegate for signature, a Deed of Standing Offer and an Official Order between the Commonwealth and IBM (Defence’s preferred tenderer for the systems integrator role) for the ‘Prepare’ and ‘Explore’ phases of Tranche 1. The intention was to enter into a Deed of Standing Offer and Official Order 1 by 1 July 2019.102

4.4 A Negotiation Issues Matrix was included as an annex to the directive. This matrix divided 63 negotiation issues into three categories — commercial, financial and solution — with each characterised as minor, significant or critical.103 IBM’s stated position and the Commonwealth’s negotiation objective and minimum fall-back position were set out for each issue.

4.5 Contract negotiations were conducted in accordance with the directive, with face-to-face negotiations taking place between 24 June 2019 and 5 July 2019. The negotiation outcomes against the Commonwealth position and the minimal fall-back position were set out in detail in a matrix included as an annex to a Contract Negotiation Report.

4.6 The ANAO reviewed the contract negotiation outcomes achieved for the 63 negotiation issues identified. Of the 63 issues set out in the directive and the Negotiation Issues Matrix, the negotiation team achieved Defence’s preferred position or better on 51 issues, and the minimum fall-back position for the remaining 12 issues. For eight of the 63 issues, the outcome achieved was clarification and agreement (for example, clarification of which partner had responsibility for data cleansing of legacy information technology applications). These matters did not require changes to be made to the key documents.

4.7 The negotiation team provided the Contract Negotiation Report to Defence’s CIO on 5 July 2019. The CIO was the Public Governance, Performance and Accountability Act 2013 (PGPA Act) section 23 delegate for entering into the arrangement. The report recommended a Deed of Standing Offer and Official Order between the Commonwealth and IBM. Contract negotiation outcomes were reflected in the Deed of Standing Offer and the Official Order. The report was endorsed by the CIO on 8 July 2019. The report stated that the negotiation team had updated the delegate on negotiations through regular briefings, and that the CIO had attended the Head Table sessions with the IBM leadership team.104 Defence and IBM entered into the Deed of Standing Offer and the Official Order on 19 July 2019, 18 days after the intended date of 1 July 2019.

Has Defence developed contractual arrangements for systems integrator services with clear milestones and performance expectations?

The contractual arrangements entered into by Defence for systems integrator services as of May 2021, comprising six Official Orders for work packages under the Deed of Standing Offer, included clear milestones and performance expectations. Systems integrator fees are paid on an instalment basis and are contingent on Defence’s acceptance of clearly defined milestones, comprised of one or more contract deliverables.

4.8 As of 3 May 2021, the contractual arrangements entered into by Defence for systems integrator services comprised an overarching Deed of Standing Offer agreement and six Official Orders for each individual work package.

4.9 The six Official Orders, which were executed between July 2019 and March 2021, had an original combined value of $257,677,968.13. Their combined value as at 3 May 2021 was $274,511,082.94, reflecting a $16,833,114.81 (6.5 per cent) increase resulting from contract variations (see Table 4.1 below).

Table 4.1: Official Orders under the Deed of Standing Offer as at 3 May 2021

Official order no.

Date executed

Work package

Original value

Value as at 3 May 2021a

1

July 2019

Tranche 1 Prepare and Explore and High Level Design for future tranches

$95,413,351.36

$111,662,084.50

2

November 2019

Release 1A Realise and Deploy

$11,723,439.64

$12,307,821.32

3

June 2020

Release 1A Sustainmentb

$4,591,282.17

$4,591,282.17

4

October 2020

Service-Oriented Architecture Integration

$1,367,030.25

$1,367,030.25

5

December 2020

Case Management

$16,982,864.70

$16,982,864.70

6

March 2021

Release 1B Realise and Deploy

$127,600,000.00

$127,600,000.00

Total 

$257,677,968.13

$274,511,082.94

         

Note a: The difference between the original contract value and the value as at 26 March 2021 is the result of contract variations discussed later in this chapter.

Note b: Official Order 3 expires on 15 July 2022, the last milestone being due 7 July 2022.

Source: ANAO analysis of Official Orders and variations.

Contract milestones and deliverables

4.10 Of the six Official Orders established as of March 2021 and reviewed by the ANAO, each set out key milestones that IBM must meet, including for initial operating capability (IOC) and final operating capability (FOC), with clearly defined acceptance criteria. The orders also list deliverables and work products.105 The due dates of deliverables and milestones are set according to the Integrated Master Schedule106, which Defence has advised is submitted by IBM quarterly and reviewed by Defence.

4.11 The payment is fixed price, with payments linked to the completion of milestones. Payment of the fixed price component of each Official Order is contingent on acceptance of the relevant milestone set out in the relevant Official Order, with a payment schedule set out in each Official Order.

4.12 Across the six Official Orders there are 100 milestones, 52 of which were due before the end of April 2021. Defence data indicates that 51 of the milestones due have been approved. Of these, 40 (78 per cent) were approved late by Defence. On average they were 28 days late. Defence advised the ANAO that the date recorded as the milestone acceptance date is the date that a final acceptance certificate is provided to the systems integrator (IBM), rather than the date all deliverables in the milestone have been completed and approved by Defence. This means that some timeliness issues are due to the time taken to issue the formal acceptance certificate.

Contract performance expectations

4.13 The Deed of Standing Offer and the Official Orders include clear performance expectations, detailing the systems integrator scope and the Defence scope for each service under the contract. For example, the Official Orders set out required business outcomes to be achieved, applicable plans107 and standards108, a summary of services required and a statement of work outlining the scope of IBM’s work under the Official Order.

Do contract and program governance arrangements enable monitoring, management and reporting on the achievement of Tranche 1 outcomes?

Contract and program governance arrangements are partially effective in enabling Defence’s monitoring, management and reporting on the achievement of Tranche 1 outcomes. Defence has established a commercial team to manage the systems integrator contract and a system to track contract deliverables. The tracker provides a basis for internal reporting on the achievement of program deliverables and milestones.

During the course of the audit, Defence commenced work on improving the clarity and consistency of decision-making arrangements set out in program guidance documentation, particularly the thresholds for involving senior decision-makers and committees in Key Design Decisions and Program Change Requests. Additionally, there is a need for Defence to review decision-making arrangements for the program’s Change Control Board, which decides on Program Change Requests. The Board’s decision-making arrangements give rise to actual conflicts of interest as the systems integrator is part of the decision-making process for variations to its contract.

Positional authority issues arise from the program’s delegation and time approval arrangements. The majority of contract delegations are exercised by staff subordinate to the program director, and a number of subordinate APS/ADF personnel are responsible for approving their manager’s timesheets.

Contract management meetings

4.14 Defence has established a commercial team within the ERP program to manage the systems integrator contract.109 The commercial team holds fortnightly contract management meetings with IBM. These follow a consistent agenda, covering: deliverable status, milestone status and invoicing, issues impacting the Deed of Standing Offer or Official Orders, variations, contract notifications, government furnished equipment and dependency status, and obligations status.

Contract deliverable tracking and acceptance

4.15 The Program Management Office110 maintains a tracker for all ERP contract deliverables, including for the systems integrator contract.

4.16 As of 17 June 2021, there were 367 contract deliverables for the program with 308 deliverables relating to the contract with IBM for Tranches 1 and 2. The staff responsible for updating the tracker meet fortnightly with the commercial team to ensure the information in the tracker is up to date.

4.17 According to the data recorded in the tracker, IBM had 237 deliverables due to be provided to Defence by the end of April 2021. Of the 237 deliverables, 227 had been submitted, as follows:

  • 134 deliverables (59 per cent) were submitted on time, on average three days early; and
  • 93 deliverables (41 per cent) were not submitted on time, on average 16 days late.

4.18 The ANAO’s review of the deliverables tracker indicated that data entry errors111 meant that more deliverables appeared overdue than was the case. There is a risk that data entry errors have reduced the accuracy of the tracker and related reporting. Defence advised the ANAO that it is reviewing the tracker to improve data quality.

4.19 The deliverables tracker also documents the approval of deliverables.112 According to data in the tracker, of the 227 deliverables due by the end of April 2021 and submitted, 209 had been approved as of 17 June 2021. Of the 209:

  • 83 deliverables (40 per cent) had been approved on time, on average four days early; and
  • 126 deliverables (60 per cent) had been approved late, on average 12 days late.

4.20 Of the remaining 18 deliverables, thirteen had conditional approval, and five had yet to receive conditional approval or approval.

Reporting on achievement of deliverables

4.21 Achievements against IBM deliverables and milestones are reported monthly across the senior governance committees, through the ERP monthly report discussed in Table 2.4.

4.22 From October 2019 to April 2020, the report set out expected deliverable due dates, and provided a traffic light indicator to advise whether the deliverable was on track, with completed deliverables marked ‘complete’. Completion dates were not provided in the report.

4.23 From May 2020 to April 2021, the report provided deliverable approval dates, but not deliverable due dates or submission dates. To assist the users of this report (including users from senior governance committees) to monitor the timeliness of deliverables, the ERP program should consider including the due dates and submission dates for comparison.

Program change request process

4.24 A Program Change Request (PCR) is to be generated for any proposed program change, and a Change Control Board has been established as the governing body to progress and approve all PCRs (see Box 2 below).

Box 2: Change Control Board

The Change Control Board meets fortnightly or as required. Its membership comprises 14 program personnel, including one APS, two ADF and 11 contractors (four IBM and seven non-IBM). The role of the Board was set out in the March 2021 version of the ERP program management plan as follows:

  • ‘Review Project Change Requests.’
  • ‘Provide direction on Project Change Requests.’
  • ‘Decision making on Project Change Requests.’
  • ‘Approval/reject Project Change Requests.’

Defence advised the ANAO on 16 April 2021 that the Board was established in February 2020 as the governing body to progress and approve all Program Change Requests (PCRs):

Early on in the program, the early intention was for the Internal Program Board to approve Program Change Requests. Over time, as [Program Change Requests] became more frequent and increased in volume, there was a need to establish a more frequent and appropriate forum for PCRs to be approved. This led to the establishment of the Change Control Board. The Change Control Board has the appropriate attendees for the Program Change Requests to be discussed and approved.

While updates on [Program Change Requests] are discussed at the Internal Program Board meetings, the Change Control Board is the governing body to progress and approve all Program Change Requests.

Change Control Board (CCB) is responsible for deciding whether to proceed with PCRs, not for approving contract changes, this a financial delegate function which is executed separate to the CCB.

As discussed in paragraphs 4.28 and footnote 117 of this audit, in respect to PCRs: ‘Approved and proceeding to CVP [contract variation proposal]’ or ‘approved and endorsed for commercial assessment’ are the expressions used consistently in the board minutes reviewed by the ANAO.

On 19 July 2021 Defence advised the ANAO — contrary to the ERP program documentation discussed above, the evidence in the board minutes, and Defence’s repeated written advice — that the Board was advisory in character:

Defence acknowledges the outward perception that the CCB is an authorised decision making body and has produced terms of reference which clearly set out the solely advisory role undertaken by the CCB. The CCB is an important element in the bringing to light relevant aspects of change requests (potential timeline changes, financial and scope impacts) that require consideration by all parties before a separate, commercial business case can be developed for consideration by the correct Defence delegate.

Defence first advised the ANAO that it was developing terms of reference for the Board on 16 April 2021. The terms of reference were approved by the CIO on 7 July 2021. The terms of reference stated that the board is ‘an advisory committee that reviews and provides guidance on Program Change Requests (PCRs) in order to provide endorsed recommendations to a Commonwealth decision authority for approval.’ Defence further advised the ANAO, on 21 July 2021, that board has been renamed, to ‘Program Change Advisory Board’. The new name was not reflected in the terms of reference provided to the ANAO, which continued to refer to the ‘Change Control Board’.

4.25 As at 3 May 2021, the ERP program’s register of PCRs included 99 items, of which 55 were marked closed (that is, approved by the Change Control Board and the required action taken).113

  • Of the 55 closed items, 41 involved a systems integrator contract change, resulting in 14 contract variations to Official Order 1, five variations to Official Order 2, one variation to Official Order 4, and one variation to Official Order 5.114
  • The 14 items which did not require a contract change related to changes to items delivered by Defence or minor changes to systems integrator deliverables which could be absorbed without any change to cost or schedule.115

4.26 Ten of the 21 variations involved a change in value. The original Official Order 1 value was $95,413,351.37. As at 3 May 2021 the value was $111,662,084.50, a total increase of $16,248,733.13 (17 per cent). The largest single variation, in November 2020, was for $14,594,193.83, an increase of 15 per cent of the previous value. This variation related to extension of the design phase of the program, and additional scope items.

4.27 The original value of Official Order 2 was $11,723,439.64. As at 3 May 2021 the value was $12,307,821.32, an increase of $584,381.68 (five per cent) of the previous value.

Decision-making processes

4.28 The ANAO’s review of Defence processes for managing Program Change Requests (PCRs) indicates that the Change Control Board was substantively responsible for deciding whether to proceed with PCRs, and for their final resolution.116 The ANAO examined minutes from 31 Board meetings from February 2020 to May 2021. Decisions relating to PCRs were documented on 26 occasions, including that PCRs were to be:

  • approved and proceed to a contract variation proposal;
  • approved and relevant program documentation be updated;
  • approved and endorsed for commercial assessment;
  • endorsed for an impact assessment;
  • cancelled/rejected; or
  • put on hold.

4.29 The ETB, as the senior committee responsible for the program, has visibility of recent PCR decisions and pending PCRs through the monthly program report. Defence advised the ANAO on 16 April 2021 that the Program Director determines whether a PCR is to be escalated to the Defence Information Systems Committee (DCISC) or the ETB. No instances have been identified of escalation of PCRs to senior committees, other than the PCRs associated with the Finance Central Payments and Banking Change Key Design Decision. As discussed below at paragraph 4.34, this Key Design Decision was approved by ETB. Defence records indicate that after a PCR relating to a contract variation is approved at the Change Control Board117, a designated SES Band 1/ADF 1 Star financial and contract delegate (the Deputy Program Director) or the ADF 2 Star Head Business Transformation, is then responsible for deciding on the associated PGPA Act section 23 commitment approval. Contract variations have then been signed by Defence delegates.118

4.30 The ANAO’s review identified the risk of a positional authority issue resulting from Defence’s delegation arrangements, as the majority of delegations were exercised by staff subordinate to the SES Band 2 equivalent Program Director and the ADF 2 Star Head Business Transformation.119 While not inconsistent with Defence policy, a similar risk was identified in Auditor-General Report No.33 2015–16 Defence’s Management of Credit and other Transaction Cards, in which the ANAO noted that for review to work effectively:

… the reviewer must be in a position to exercise independent judgement … this means that they cannot be in a position which would constrain unreasonably their capacity to question transactions that appear inappropriate; for example, this may be difficult for a person junior to the cardholder … (paragraph 2.42).120

4.31 The PCR process set out in the ERP program handbook during audit fieldwork is illustrated in Figure 4.1 below. This was the process followed in all but one case reviewed by the ANAO. This process was inconsistent with the approved 2019 program management plan, which set out that the Internal Program Board is responsible for approving PCRs. The program management plan was updated in March 2021, in the course of this audit, to align with the process set out in the handbook, setting out that the Change Control Board was responsible for approving PCRs.

Figure 4.1: ERP Program Change Request procedure as at March 2021

This figure sets out the process for Program Change Requests in the ERP program. It presents a flow chart for the identification, assessment and resolution of Program Change Requests.

Note: The PMO is the Project Management Office. This was the process in place during audit fieldwork. This process was revised in the April 2021 version of the program handbook. The revised process indicates that the Change Control Board may escalate PCRs to the ETB or other bodies. Defence provided a further revised process on 21 July 2021. The further revised process indicates that the Change Control Board provides ‘direction or recommendation on presented requests’, and that the Program Director then presents the ‘recommendation for approval by Commonwealth decision-making authority’.

Source: Defence records.

4.32 At the time of audit field work there was no charter or terms of reference for the Change Control Board. As discussed in Box 2 above, the board’s functions were described briefly in the March 2021 version of the Program Management Plan, and terms of reference have been approved on 7 July 2021.

4.33 The ANAO also identified inconsistent advice in the program management plan regarding the approval process for Key Design Decisions, in particular, the instances where these should come to the ETB. The ERP program management plan in place during audit fieldwork, as well as the version updated in March 2021, indicate that Key Design Decisions are approved internally by the program, while the ETB charter indicates that the ETB approve Key Design Decisions unless there is a timeliness issue. During the course of the audit, Defence reviewed the Key Design Decision process to improve clarity on DCISC and ETB decision-making responsibilities. The new process, which sets out the role for DCISC and ETB, was approved by the ETB in February 2021. In May 2021 the program handbook was updated to include the approved process, stating that there is a requirement to obtain approval for Key Design Decisions above the Program Executive level. The program management plan could also be reviewed to reflect that Key Design Decisions are to be made above the program level.

4.34 Key Design Decisions and Program Change Requests (PCRs) can affect program scope, cost and schedule and can result in contract variations. While the program management plan sets out that the ETB is the decision-making authority for ERP scope, schedule and budget, in practice, Key Design Decisions and PCRs are largely approved at the program level. An exception includes a Key Design Decision, approved by the ETB in August 2020. This resulted in a contract variation of $14,594,193.83 (see paragraphs 2.23 and 4.26).

4.35 A strategic RACI (a document that sets out which stakeholders are responsible, accountable, consulted and informed) was approved by the ETB in August 2019. It sets out decision-thresholds for the program in areas of scope, schedule and funding, and when decisions should go to the Defence Investment Committee, Enterprise Business Committee, ETB, or DCISC. It also sets out when they can be approved at the program level. However, this information is not reflected in current program guidance materials, including the program management plan and handbook. Decision-making responsibilities should be clearly and consistently set out in program guidance documents, to support compliance with requirements.

Managing conflicts of interest in the program change request process

4.36 The ERP program involves a large number of contractors working across all parts of the program and at all levels of program decision-making.121 Defence has interspersed its officials across the ERP program to discharge its departmental responsibilities.122 For example, the Change Control Board has three departmental officials working alongside 11 contractors, including four IBM personnel, on program changes, including changes to the IBM contract. In four of the 26 Change Control Board meeting minutes examined by the ANAO, where decisions were made, there were no departmental staff present. The average level of Commonwealth representation at Change Control Board meetings was one departmental official present, with an average total attendance number of nine attendees across the examined meeting minutes. A low level of Commonwealth representation across a contractor-led program can create oversight risk for the responsible entity, which remains accountable for the proper use and management of public resources under the PGPA Act and for decision-making in the Commonwealth interest.

4.37 Further, Defence’s program change arrangements give rise to both real and perceived conflicts of interest, as contractors are involved in the department’s substantive decision-making processes relating to their contracts, including contract variations with financial consequences for the Commonwealth. As discussed in Box 2 above and at paragraphs 4.28–4.31, Defence has documented, and the board minutes indicate, that the Change Control Board is a decision-making and approval body for Program Change Requests (PCRs). The board had IBM representation at each of the 31 meetings reviewed by the ANAO. The meeting minutes document that PCRs relating to price-impacting variations to IBM’s contract were approved to progress to the next stage (commercial assessment or contract variation proposal) at six of the 31 meetings, resulting in four contract variations with a total value of $483,902.77. The relevant meeting minutes do not record any conflict of interest declarations being made by any participants. Further, the minutes do not document any participants recusing themselves from the Board’s deliberations and decision-making.

Recommendation no.3

4.38 The Department of Defence review Enterprise Resource Planning program decision-making arrangements to ensure they:

  1. are clearly and consistently set out in program guidance documentation;
  2. avoid real and perceived conflicts of interest where contractors are potentially involved in decision-making regarding their contracts, including contract variations with financial consequences for the Commonwealth; and
  3. enable delegations to be exercised by individuals who are in positions where they are able to make independent judgements.

Department of Defence response: Agreed.

4.39 Defence agrees to recommendation.

Contract governance arrangements

4.40 As discussed in paragraphs 2.50–2.51 and Box 1, Defence uses an ‘above the line/below the line’ model in the ERP program, which involves ‘above the line’ contractors in program administration roles alongside ADF and APS personnel. Defence’s contract administration process for the ERP program involves ‘above the line’ contractors submitting timesheets, which are approved by another program team member. The timesheets are then used by the ERP commercial team to validate the contractors’ invoices prior to payment.

4.41 In late 2019, the program identified a contractor whose personnel were invoicing the program for more days than those documented in their timesheets. The program engaged a strategic advisor to investigate the issue (Ngamuru Advisory Pty Ltd). The resultant May 2020 report included the findings of an internal financial investigation that the Commonwealth had been invoiced $51,769.39 above what was demonstrated in the relevant timesheets. The strategic advisor’s report found that the contractor believed there was an ‘informal working arrangement’ in place with the program. The report recommended various actions, including potential price adjustment. The contractor agreed to Defence’s proposed actions to remediate the issue.

4.42 Following identification of this issue, in June 2020 the ERP Deputy Program Director was provided with information on the program’s contract administration process. This included a table setting out which personnel were responsible for approving each contractor’s timesheets. This table demonstrated issues of positional authority123, with three subordinate APS/ADF staff responsible for approving their manager’s timesheets, and two instances where the contractor and approver were in the same team and at the same level (see Table 4.2 below). Further issues related to whether there was adequate visibility of work performed and an ability to confirm timesheets, with 13 instances of the contractor and approver being in different teams.

Table 4.2: Contractor timesheet approvers as at June 2020

Relationship between contractor and timesheet approver

Identified instances

Contractor and approver are in different teams

13

Contractor’s approver is their manager

5

Contractor’s approver is subordinate

3

Contractor’s approver is in the same team, at the same level

2

Unable to determine from available data

11

Total

34

   

Source: ANAO analysis of Defence records.

4.43 The information provided in June 2020 to the ERP Deputy Program Director also indicated that some ‘above the line’ contractors were responsible for approving the timesheets of other ‘above the line’ contractors. The Deputy Program Director requested advice on ‘having contractors sign for contractors’. The APS commercial lead responded that ‘from a governance point of view, contractors are not making a commitment or spending Defence money when they sign a timesheet’. In July 2021 Defence advised the ANAO that ‘Defence acknowledges the advice provided by this official is incorrect. Defence does not support this advice and recognises the requirement that actions that lead to expenditure of Public Money can only be exercised by authorised Officials under the PGPA Act’. Defence further advised that it is ‘reviewing governance arrangements that may have led to a misunderstanding of this principle.’

Recommendation no.4

4.44 The Department of Defence review contract governance arrangements to ensure timesheet approvals are made by personnel with adequate visibility of work performed.

Department of Defence response: Agreed.

4.45 Defence agrees to recommendation.

Is Tranche 1 contributing to ERP program objectives and Defence strategic priorities?

Tranche 1 is contributing to ERP program objectives and the Defence strategic priority, identified in the 2015 First Principles Review, of implementing an enterprise approach to the delivery of enabling services. However, as of May 2021, the ERP program remained at an early stage of implementation. Tranche 1A was delivered in December 2020 but Tranche 1B is the material activity of Tranche 1 and is yet to reach IOC. Program benefits are not expected to be realised until after IOC for Tranche 1B is reached in Quarter 4 2022. Tranche 2 has been approved and Tranche 3 is yet to be presented to government. Substantial work remains for Defence to fully implement the ERP program.

Progress in delivering Tranche 1

4.46 An indicative schedule for the delivery of Tranche 1 was set out in the second pass business case provided to government. Defence subsequently refined Tranche 1 with the systems integrator — in accordance with the SAP prepare, explore, release and deploy methodology — and split Tranche 1 into Release 1A and Release 1B after second pass (see paragraph 2.7). To review progress in delivering Tranche 1, the ANAO examined 18 ERP monthly reports, 21 Enterprise Transformation Board (ETB) minutes, six quarterly reports to the Minister and three six monthly reports to the government for the period March 2019 to April 2021.

Release 1A

4.47 As set out in the initial Integrated Master Schedule for the program developed by the systems integrator, and approved by the ERP program director on 5 November 2019, Release 1A IOC was planned for 20 July 2020, and FOC was planned for 20 October 2020. The ETB was advised in the July 2020 ERP monthly report that Release 1A IOC was delivered on time on 20 July 2020. The ETB was advised in the November 2020 Monthly report that the Release 1A FOC was proposed for delay to 14 December 2020. The change in FOC was managed through a Program Change Request and approved by the program’s Change Control Board. The January 2021 monthly report set out that the Release 1A FOC milestone had been approved on 17 December 2020. Although FOC had been delayed, in Defence’s December 2020 six monthly update, the government was advised that Release 1A was delivered on schedule.

Release 1B

4.48 Delivery of Release 1B was underway as at May 2021.

4.49 In May 2020 Defence advised the government that IOC for Release 1B was planned for mid-2022 and FOC was planned for mid-2023. In December 2020, Defence returned to government and requested a further delay to IOC for Release 1B to Quarter 4 2022. The government considered and approved this request in May 2021. Release 1B FOC remained at mid-2023. As at March 2021, the program reported to Defence’s Audit and Risk Committee that it was nearing the end of the ‘Explore’ (design) phase, which was expected to be complete by June 2021.

Program Objectives

4.50 The program management plan (developed in 2019 and updated in 2021) includes 13 program objectives, and 12 program outcomes, set out in Table 4.3 below. Defence intends to track progress against these through the ERP benefits management system.

Table 4.3: Program objectives, outcomes and benefit areas, as set out in the March 2021 Program Management Plan

Program objectivesa

  • Improved force preparedness, planning, and reporting
  • Improved advice to Government
  • Improved resource management across the enterprise
  • Seamless transition to and from operations
  • Fewer applications
  • Improved system availability, including deploy ability under communications constraint
  • Exploitation of modern, innovative technology
  • Trusted, secure data (protected and secret)
  • Improved decision-making support
  • Consistent, simplified end-to-end processes
  • Assured compliance
  • Reduced sustainment cost and risk
  • Increased workforce capacity

Program outcome areas

  • Capability and resource planning
  • Preparedness and planning
  • Deployability
  • Engineering
  • Estate
  • Finance
  • Supply chain
  • Maintenance
  • Procurement
  • Human Resources Management
  • Case management
  • Training and sustainment
   

Note a: There is a fourteenth program objective set out in the second pass business case: ‘reduced training burden and enhanced ease of use.’ Defence advised this was omitted from the program management plan in error.

Source: Defence records.

Benefits management strategy

4.51 Defence finalised a benefits management strategy on 8 December 2017 to support the second pass business case. A new version of the strategy was approved by the ETB on 6 August 2020. The strategy’s purpose is to ensure that the program’s:

Benefits Management Methodology, including associated processes; roles and responsibilities; governance and approval mechanisms; tools and artefacts are clearly articulated and understood.

4.52 The strategy sets out the relationship between the program objectives, program outcomes and benefit areas.124 The strategy defines benefits as:

… the measureable improvement from outputs and outcomes expected of a Program or Project.

4.53 A benefits management team has been established within the ERP program. Its role is to: provide benefits managements advice; develop the benefits management strategy; identify, plan and analyse benefits; support Defence Groups’ and Services’ measurement and realisation activities125; report on measurement and realisation data; and hand over benefits management documentation and activities once the ERP end-to-end solution has been delivered. After benefits realisation plans are developed126, six monthly Group and Service reporting to the benefits management team is planned. This is to be collated by the benefits management team into an ERP program benefits dashboard.127

4.54 The program returned to the ETB in March 2021 with recommendations on benefit ownership and rebalancing. The ETB did not agree to the recommendations and requested that the program ensure alignment with the Defence Transformation Strategy, whole-of-government transformation and Government Enterprise Resource Planning. On 14 May 2021 Defence advised the ANAO that it was working with GovERP128 and the Defence Transformation Office on this.

4.55 The benefits management team has developed a draft benefits map that traces program outcomes to benefit owners, end benefits, benefit areas and intermediate benefits. As at 21 July 2021, intermediate benefits were yet to be defined, and were undergoing review with the program and Defence Groups and Services. Defence advised the ANAO that it continues to ‘engage with Groups and Services to shape and define ERP intermediate benefits as they correlate to the eight end benefits referenced in the second pass business case.’ The strategy states that ERP benefits management is an iterative process, and that the approach will evolve over time. The strategy also states that benefits will not be realised until the change activities occur. As Tranche 1 Release 1B IOC is not expected until Quarter 4 2022, the program is not yet at the stage of benefits realisation.

4.56 The benefits realisation framework is intended to support Defence in measuring, monitoring and reporting on the extent to which outcomes have been achieved. While acknowledging that there is ongoing planning work occurring in the program, early establishment of a benefits realisation plan for each Tranche and for the program overall will better position Defence to collect the data necessary to demonstrate that outcomes have been delivered.

Recommendation no.5

4.57 The Department of Defence develop benefits realisation plans for each Enterprise Resource Planning program Tranche and the program overall.

Department of Defence response: Agreed.

4.58 Defence agrees to recommendation.

Strategic Priorities

Australian Industry Capability

4.59 At second pass, Defence undertook to achieve an estimated level of Australian industry involvement (also known as Australian industry capability (AIC)) of 80 per cent of program expenditure.

4.60 For the systems integration component of the program, the Deed of Standing Offer with IBM includes an AIC strategy. The strategy sets targets of a minimum 90 per cent local industry involvement and a minimum of 30 per cent small to medium enterprise involvement. IBM’s quarterly AIC progress report, covering December 2020 to March 2021, indicated that IBM was exceeding its AIC target, with an average of 99.93 per cent local industry involvement and 41.87 per cent small to medium enterprise involvement achieved to date.

4.61 Defence advised the ANAO on 21 July 2021 that:

To date Defence has achieved 75% Australian Industry Content (AIC) across all program expenditure. This is a cumulative total program expenditure to date and should not be taken as a projection of the total anticipated AIC component for the program on completion.

First Principles Review

4.62 As discussed in paragraph 1.2, one of the six key recommendations of the 2015 First Principles Review: Creating One Defence was to fully implement an enterprise approach to the delivery of corporate and military enabling services to maximise their efficiency and effectiveness. The 2016 Defence White Paper also identified issues related to Defence information and communications technology (ICT) systems resulting from underinvestment in ICT and the lack of an enterprise-level strategy for Defence’s ICT requirements. The 2016 Integrated Investment Program, released with the Defence White Paper, committed to enhancements to Defence’s ICT and business processes to support the implementation of First Principles Review recommendations, including by standardising business processes to provide end-to-end visibility of Defence business through streamlined processes and a consolidated Defence enterprise resource planning system intended to improve core business functions. The 2020 Force Structure Plan continued to emphasise the importance of business transformation in Defence, setting out that:

Driving improved business processes, enterprise information management, enterprise resource management and broader transformational reforms across Defence will be critical to future Defence effectiveness. The Government is committed to improving enterprise business processes by implementing reform across Defence to maximise business commonality and effectiveness.129

4.63 At first and second pass, the business case prepared by Defence to inform the Minister for Defence’s submission to government set out that the ERP program would assist in addressing deficiencies identified in the First Principles Review, by standardising business processes.

4.64 As of May 2021, some six years after the First Principles Review identified the need to implement an enterprise approach to the delivery of enabling services, the ERP program remained at an early stage of implementation. Tranche 1A was delivered in December 2020 but Tranche 1B is the material activity of Tranche 1 and is yet to reach IOC. Tranche 2 has been approved and Tranche 3 is yet to be presented to government. Substantial work remains for Defence to fully implement the ERP program.

Appendices

Appendix 1 Department of Defence response

Page 1 of the response from the Department of Defence. You can find a summary of the response in the summary and recommendations chapter of this report.

Page 2 of the response from the Department of Defence. You can find a summary of the response in the summary and recommendations chapter of this report.

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

  • strengthening governance arrangements;
  • introducing or revising policies, strategies, guidelines or administrative processes; and
  • initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

5. In respect to a range of issues identified by the ANAO in the course of the audit — particularly the issues reported in paragraphs 3.67–3.87 and 4.40–4.43 — Defence advised in its response to the audit report that:

Defence has taken actions to address the areas for improvement identified by the audit, including the implementation of a Contractor Integrity Framework [ANAO comment: see paragraph 3.74] and review of contractor on-boarding processes. Defence is conducting reviews of ERP Program operation and governance, with a particular focus on improving transparency in the selection of delegates [ANAO comment: see paragraph 4.30] and appropriate approval of timesheets [ANAO comment: see paragraphs 4.40–4.43]. In addition, mandatory training and periodic probity briefings have been introduced to ensure contractor awareness of their obligations.

Defence accepts the areas for improvement noted in the report. The matters identified have provoked immediate action in the interests of maintaining efficient, effective and secure delivery of the ERP Program. Where failings by individuals have been identified and verified [ANAO comment: see paragraph 3.86], steps have been taken, commensurate to the seriousness of the findings, to ensure no repeat of this behaviour. Defence has confidence that, irrespective of the individual actions, Defence’s systems are secure and value for money has been achieved through ERP Program delivery to date.

Defence agrees to implement all recommendations proposed in the report. These actions will ensure any shortcomings at the system level have been identified and appropriately addressed as a priority. Defence has initiated complete reviews of ERP Program operations and taken steps to vary delegations, increase oversight and reconsider the appropriate balance of contractors with APS/ADF personnel in key management positions.

Appendix 3 Approach to the use of the SAP application for the ERP program

1. At first pass on 22 May 2017, the Australian Government agreed to Defence’s preferred option for the program, which was to standardise and simplify end-to-end business processes, enabled by a new build SAP technology solution. Box 3 below provides further information on Defence’s approach to the use of the SAP application and program methodology in relation to the ERP program.

Box 3: Defence’s approach to the use of the SAP application and program methodology in relation to the ERP program

  1. In June 2015, the Defence Committee agreed that the SAP Defence Force and Public Security (DFPS) application be used as the platform for the ERP program. The rationale for choosing SAP was a high level of alignment between SAP DFPS and Defence business processes, including for its finance and logistics systems. The SAP DFPS solution was agreed by the government on 22 May 2017 at first pass.
  2. Defence advised the ANAO that in October 2019 SAP released the SAP Defence and Security application, which replaced SAP DFPS. SAP Defence and Security uses the S/4HANA platform.
  3. At second pass, on 12 June 2018, the government was advised that SAP’s S/4HANA software would be used to develop a Model Defence Organisation (see Figure A.1). Defence intended to use the MDO to manage risks and accelerate delivery.

Figure A.1: ERP Program Change Request procedure as at March 2021

This figure sets out the relationship between key elements of the ERP program, including the S/4HANA software, Model Defence Organisation, Enterprise Resource Planning solution and Defence business model.

Source: ANAO representation of Defence records.

  1. Defence is using the SAP Activate program methodology to implement the ERP program. This program methodology includes the following phases:
    • Prepare: involves a series of activities to help identify and plan the primary focus areas, establish a common understanding of the program method, and develop workshop content to support the ‘explore’ phase.
    • Explore: involves facilitation of workshops to validate business processes and key decisions identified during the ‘prepare phase’. SAP S/4HANA Model Defence Organisation is to be used as the baseline model, with the overarching principle of ‘adopt not adapt’ — which means that personnel are expected to adopt the new system rather than customise the system to fit existing business practices.
    • Realise: involves work to document, build and test the solution, based on the approved design.
    • Deploy: involves the rollout of the system, ‘hypercare’ — a period where a high level of support is available, and transition to sustainment of the system.

Defence has advised the ANAO that the nature of the SAP Activate methodology is that during the explore phase, key design questions are considered which need to be addressed to deliver the solution, rather than the solution being determined prior to the commencement of the program.

Appendix 4 ERP program schedule as at May 2021

Table A.1: Planned timelines and actual events for the Enterprise Resource Planning program

Date

May 2017 timelinea

First Pass

June 2018 timelinea

Second Pass

December 2019 timelineb

May 2020 timelinec

May 2021 timeline

Mid 2018

Second pass consideration of Tranche 1 by government

Actual: Government considered and approved Tranche 1 at second pass

Mid 2018

Delivery phase to commence

Quarter 1 2019

Tranche 1 ‘Prepare’ and ‘Explore’ phases to be completed

July 2019

Actual: Contract signed with systems integrator (commencing delivery of Prepare and Explore phase)

February 2020

Initial Operating Capability (IOC) for the ERP program

July 2020

Tranche 1 Release 1A connection to existing Defence financial systems

Actual: Tranche 1 Release 1A IOC delivered

Quarter 4 2020

IOC for the ERP program (when the business transformation in scope for Tranche 1 has been successfully adopted)

Tranche 1 Release 1B financial reports released to Defence users

Actual: Tranche 1 Release 1A FOC delivered

January 2021

Tranche 1 ‘Prepare’ and ‘Explore’ phases to be completed (as per contract with systems integrator)

June 2021

Tranche 1 ‘Prepare’ and ‘Explore’ phases to be completed (as per November 2020 variation to contract with systems integrator)

Mid 2022

IOC for Tranche 1 (technical go-live and pilot rollout)

IOC for Tranche 1 Release 1B

Quarter 4 2022

IOC for Tranche 1 Release 1B

Mid-2023

FOC for Tranche 1 Release 1B

FOC for Tranche 1 Release 1B

Quarter 4 2025

FOC for the ERP program (when the end-to-end business transformation in scope for the ERP program has been successfully adopted)

2026

Final Operating Capability (FOC) for the ERP program

2050

ERP end of life

      

Key:  planned dates; actual dates

Note a: In May 2017 (first pass) and July 2018 (second pass) the schedule was ‘indicative’, and to be refined through the SAP Activate process (prepare, explore, realise and deploy).

Note b: In December 2019 the impact of the proposed change in Initial Operating Capability was being considered, so no Final Operating Capability date was presented.

Note c: In May 2020 the government was advised that Tranche 1 had been split into Release 1A and Release 1B.

Source: ANAO analysis.

Footnotes

1 Department of Defence, 2019–20 Annual Report, p. 3.

2 Defence has advised the ANAO that the ERP will also address the root cause of two ANAO ‘B’ category findings (previously ‘A’ category findings) and three ‘C’ category findings made in the 2019–20 audit of Defence’s financial statements. For descriptions of audit finding categories see: Australian National Audit Office, Financial statement audit information, ANAO, available from https://www.anao.gov.au/financial-statement-audit-information [accessed 4 March 2021].

3 ANAO comment: this paragraph relates to the discussion in paragraphs 2.29–2.31 of the audit report. Defence’s decision to amend a contract anticipated a change in the Initial Operating Capability date before the revised date was agreed by the government. IOC and FOC dates for this program have been set by the government.

4 Department of Defence, First Principles Review: Creating One Defence, April 2015.

5 Auditor-General Report No.34, 2017–18 Defence’s Implementation of the First Principles Review, p.16. As part of that performance audit the ANAO reviewed implementation of the review’s recommendations as at early 2018.

6 Department of Defence, First Principles Review: Creating One Defence, April 2015, p. 5. In its Lead the Way: Defence Transformation Strategy announced in late 2020, Defence sought to update the ‘One Defence’ concept to ‘reaffirm the organisation’s commitment to achieving its intent and benefits’ (Department of Defence, Lead the Way: Defence Transformation Strategy, November 2020, p. 15).

7 ibid., p. 9.

8 ibid., p. 13.

9 ibid., p. 14.

10 Department of Defence, Defence White Paper, February 2016, p. 105.

11 Department of Defence, Integrated Investment Program, February 2016, p. 56.

12 ibid., p. 61, Table 4. Table 4 also noted that: ‘The figures in the table cover the acquisition element of the programs. There will be additional investment in whole-of-life sustainment and operating costs for each program. All figures are calculated on an out-turned price basis.’

13 Department of Defence, 2020 Force Structure Plan, July 2020, p. 83.

14 See Appendix 3 for further information on S/4HANA. In essence it is an enterprise resource planning software package meant to cover the day-to-day processes of an enterprise.

15 Further information on the first and second pass ICT investment approval process can be found at Department of Finance, ICT investment approval process, Finance, available from www.finance.gov.au/government/assurance-reviews-and-risk-assessment/ict-..., [accessed 29 March 2021].

16 Four options had been presented to government: continue with evolution of current independent systems; re-platform existing technology (technology refresh); standardise and simplify end-to-end processes, enabled by a new build SAP technology solution; or a managed service model to outsource specific service delivery functions.

17 The Defence Integrated Investment Program (IIP) is a ten year expenditure plan covering activities and projects that have been approved for inclusion in the IIP by the government. An IIP provision sets out what funding has been provisioned (including for acquisition and sustainment), whether the funding is approved or unapproved by the government for the project and release of funds, and in which financial years the funding is currently allocated to. Defence advised the ANAO that unfunded contingency is ‘an agreed contingency budget (provision) which is not programmed or funded in cash terms. If a project needs to access contingency to mitigate a contingent risk, the funding of that contingency will need to be identified via offsets from either within the Program or Defence investment program more broadly’.

18 The advice to government was that IOC would be when the scope of the first tranche is complete.

19 Defence advised the ANAO on 19 April 2021 that the detail and scope of FOC were yet to be defined at first pass. This is in accordance with Defence’s advice to government at first pass that capability would be developed and delivered in a series of tranches, with associated second pass submissions.

20 Further information is set out in Appendix 3.

21 In July 2020, Defence investigated whether the case management program could align with the ERP program, and found that SAP’s case management product could meet Defence’s needs. In September 2020, the Defence Investment Committee approved the program to proceed to second pass as part of the ERP program.

22 Department of Defence, 2019–20 Annual Report, p. 3.

23 Defence has advised the ANAO that the ERP will also address the root cause of two ANAO ‘B’ category findings (previously ‘A’ category findings) and three ‘C’ category findings made in the 2019–20 audit of Defence’s financial statements. For descriptions of audit finding categories see: Australian National Audit Office, Financial statement audit information, ANAO, available from https://www.anao.gov.au/financial-statement-audit-information [accessed 4 March 2021].

24 Chapter 3 discusses the procurement of the systems integrator. The systems integrator was engaged in July 2019.

25 Defence advised the ANAO on 21 July 2021 that it was ‘reviewing a suite of governance documents, including the Risk and Issue Management Strategy, Program Management Plan and terms of reference of the Program Change Advisory Board (formerly Change Control Board) to ensure adherence to the Managing Successful Programs framework.’

26 The program structure involves streams organised into areas of core ERP capability (Military Planning and Operations, Supply Chain Management, Enterprise Asset Management, Finance, Procurement, Human Resource Management, Estate and Infrastructure, and Case Management) led by an APS/ADF business lead, IBM capability lead, and contracted delivery lead who supports contract delivery.

27 The 1 Star group is described in Table 2.3.

28 The ‘high’ enterprise risk relates to the ability of the program to contain scope. The ‘high’ stream risks relate to: integrating historical legal data; schedule; late design completion; test completion; stakeholder alignment; and a procurement solution, for specific function areas of the program.

29 The MDO is a product provided by SAP and intended to be used as a ‘building block’ for the design and build of the SAP S/4HANA software (see Appendix 3). The S/4HANA software would then support the ERP solution. The second pass business case set out that the MDO would accelerate program delivery, enable Defence to adopt standard SAP configurations, and minimise the need for customisation.

30 These included that ‘senior Defence decision-makers have not been brought on the ERP journey sufficiently well, and that there is a lack of confidence amongst them about the Program’s status and direction’, as well as a lack of business readiness, with the program focus on achieving second pass approval rather than stakeholder engagement.

31 The Independent Assurance Review is further discussed in Chapter 3 in the context of the systems integrator procurement.

32 The Enterprise Information Management program is another Defence ICT program with interdependencies with the ERP program.

33 Defence advised the ANAO in July 2021 that ‘In late 2020, the program was further re-baselined through identification of emerging Defence risks and opportunities, resulting in a further IOC delay to Quarter 4 2022’.

34 The impacted contract related to the ‘Prepare’ and ‘Explore’ components of Tranche 1B, with the ‘Realise’ and ‘Deploy’ contract yet to be finalised at the time (see Appendix 3 for further detail on the program methodology). Defence advised the ANAO that as the FOC date was remaining stable, by extending the timeframe in the ‘Prepare’ and ‘Explore’ contract, Defence expected savings in the ‘Realise’ and ‘Deploy’ contract.

35 IOC is defined in an attachment to the November 2019 submission as the ‘Tranche 1 technical go-live and pilot rollout’.

36 As indicated in Table 2.1, the government was advised in May 2020 that FOC would be achieved in mid-2023.

37 Defence advised the ANAO on 18 May 2021 that the Integrated Investment Program provision is referring to the ERP Tranche 1 budget approved by the government at second pass.

38 The Minister for Defence had agreed on 6 October 2020 that a submission go to government that included the request for the change in IOC date. As set out in paragraph 2.23, ETB had agreed to the change on 6 August 2020.

39 There was initially a SES Band 2/ADF 2 Star program manager in this role. Responsibility for program oversight transferred to the CIO on 3 September 2018.

40 The review stated that ‘given the volume of information coming before the EBC, the Program would benefit from a sub-set of the EBC focused on the Program’.

41 In August 2018 Defence expanded the role of the ETB to include governance of another ICT program, the Enterprise Information Management program.

42 See paragraph 3.34 for further discussion of the Independent Assurance Review.

43 Risks related to securing adequate resources to progress the ERP program have been documented in the program risk register.

44 Defence advised the ANAO that the first PMO Director was appointed in August 2016.

45 The government’s Gateway Review Process, administered by the Department of Finance, is intended to help entities deliver major projects or programs by providing third party advice through development and implementation of a project or program. Department of Finance, Gateway Review Process, Finance, available from https://www.finance.gov.au/government/assurance-reviews-and-risk-assessment/gateway-reviews-process [accessed 19 February 2021].

46 The review defines ‘amber’ as follows: ‘Successful delivery of the project to time, cost, quality standards and benefits realisation appears feasible but significant issues already exist requiring management attention. These need to be addressed promptly’.

47 The review defines ‘green/amber’ as follows: ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears probable however consistent attention will be needed to ensure risks do not become major issues threatening delivery’.

48 The review defines ‘amber/red’ as follows: ‘Successful delivery of the project to time, cost, quality standards and benefits realisation is in doubt with major issues apparent in a number of key areas. Urgent action is needed to address these’.

49 The Short Form Gateway Review was limited to assessing progress against recommendations of the Mid Stage Gateway Review, and found improvement in delivery confidence as a result of previous recommendations being accepted and actioned, although the review notes that some were still works in progress.

50 Based on the ERP organisational chart, as at 1 April 2021 there were: 116 ‘above the line’ contractors, 233 ‘below the line contractors’ and 107 APS/ADF personnel in the program.

51 There is evidence that an ERP program-led SES Band 2/ADF 2 Star steering group existed prior to this, as early as November 2016.

52 The Internal Program Board’s terms of reference do not include quorum requirements, but set out that: members, attendees and advisors are required to attend the meetings if invited; in the event that an invitee is unable to attend, they are to appoint a delegate to attend in their place, who has their full authority with respect to tranche and project matters or specific agenda items; and non-attendance of the meeting will not be grounds for disputing the decision of the board at a later date. Of the 16 program board minutes examined by the ANAO, in four instances one of the two APS/ADF members was not present, and in one instance both APS/ADF members were not present. In these five cases, there was no delegate recorded in the minutes for the absent APS/ADF members. No program decisions were documented at these meetings, other than an amendment to a status report.

53 A Program Change Request is a mechanism required by the program to initiate potential changes to the program’s baseline scope, schedule or cost.

54 Internal Program Board minutes do not document Program Change Request approvals, with the exception of one in November 2019 (of 55 approved Program Change Requests, as at 5 May 2021). Program Change requests are a standing agenda item at Internal Program Board meetings, with a table of Program Change Requests presented and discussed.

55 Defence advised the ANAO in April 2021 that:

Early on in the program, the early intention was for the Internal Program Board to approve Program Change Requests. Over time, as [Program Change Requests] became more frequent and increased in volume, there was a need to establish a more frequent and appropriate forum for PCRs to be approved. This led to the establishment of the Change Control Board. The Change Control Board has the appropriate attendees for the Program Change Requests to be discussed and approved.

While updates on [Program Change Requests] are discussed at the Internal Program Board meetings, the Change Control Board is the governing body to progress and approve all Program Change Requests.

In July 2021 Defence advised the ANAO, contrary to its earlier advice and Defence documentation, that the Board had a more advisory character. This matter is discussed further at paragraph 4.24 and Box 2 on page 72.

56 Prior to this, change management and communications plans were developed by Deloitte, Defence’s contracted strategic partner, in August 2015.

57 In April 2020 the monthly report was reformatted. In this format, the timeline included was not legible in six of the 13 reports reviewed. In May 2021 Defence advised the ANAO that feedback from the ETB has been received on this issue, and that the page is often attached separately and printed in A3 for readability.

58 Defence advised the ANAO that work stream updates in the monthly report are provided directly from work streams through EPMS to ensure a ‘single source of truth’ to the ETB. Defence also advised that avoiding acronyms and technical language in these updates is an area of ongoing improvement.

59 From February 2021 there has been a residual high enterprise risk related to the ability of the ERP program to contain scope.

60 The CASG quarterly performance reporting process was examined in Auditor-General Report No. 3 of 2019–20, Defence’s Quarterly Performance Report on Acquisition and Sustainment, available at https://www.anao.gov.au/work/performance-audit/defence-quarterly-performance-report-acquisition-and-sustainment [accessed 19 October 2020].

61 For example, the Quarter 1 2020 narrative section of the report states that schedule risk is medium-high, consistent with the reporting to government. Other risk levels reported to government are not discussed.

62 Defence acquisition projects with issues and risks raised against schedule, cost, and/or capability performance that warrant heightened senior management attention may become a Project of Interest or a Project of Concern. See: Auditor-General Report No.31 2018–19, Defence’s Management of its Projects of Concern.

63 The Quarter 4 2018 update was not provided to the Minister and was superseded by the Quarter 1 2019 update.

64 Department of Defence, Lead the Way: Defence Transformation Strategy, Defence, available from https://www1.defence.gov.au/sites/default/files/2020-11/Defence-Transformation-Strategy.pdf [accessed 24 March 2021].

65 Two of the three reports stated that the program was within budget and provided the approved acquisition figures and acquisition spend to date. Financial information was otherwise not presented.

66 The November 2019 report narrative discusses a proposed change in IOC dates, but does specify whether this is for Release 1A or Release 1B, or provide separate IOC and FOC dates for Release 1A or Release 1B. The May 2020 report includes Release 1B IOC and FOC dates in the narrative, but not Release 1A IOC or FOC dates, although a Release 1A ‘go live’ date is provided. The December 2020 report discusses a proposed change to Release 1B IOC and specifies the planned Release 1B FOC dates. The report states that Release 1A ‘go-live’ has been achieved on schedule, but does not provide the FOC date. Release 1A FOC had been delayed from October to December 2020. See paragraph 4.47.

67 As discussed in paragraph 1.14, funding was also approved for the ‘Prepare’ and ‘Explore’ phases for future tranches ($152.1 million) and sustainment over three years ($106.1 million).

68 A systems integrator in information technology integrates computing systems and software applications physically or functionally to act as a coordinated whole. As set out in Table 1.2, the contract with the systems integrator represents more than 50 per cent of the total program budget in 2019–20 and 2020–21. Defence contracted IBM Australia for $95,413,351.37 on 19 July 2019 for the ‘Prepare’ and ‘Explore’ phases of Tranche 1, including high level design of future tranches. Defence has signed additional Official Orders for Tranche 1 and varied two Official Orders so that the total contracted cost for IBM’s Tranche 1 work as systems integrator is $274,511,082.94 (as of April 2021).

69 The ERP program commenced in February 2015 with a Program Definition Phase, the period leading up to first pass (considered by government on 22 May 2017). This work was done with the assistance of Deloitte, the program’s strategic partner between 30 March 2015 and 31 October 2016. KPMG was selected as a new strategic partner through an open tender approach, with the contract commencing on 25 July 2017.

70 The primary implementations examined were the United Kingdom and Canadian implementations.

71 Non-functional requirements define the overall qualities or attributes of a system.

72 Defence advised the ANAO in March 2021 that the ERP reference group involved Group and Service representatives who informed the development of the business cases.

73 Outcome characteristics would be developed through the Offer Definition and Improvement Activities (ODIA) that were a part of the systems integrator procurement.

74 An Endorsement to Proceed is a form required to be prepared and then approved by the Defence Chief Procurement Officer prior to releasing request documentation to the market for Defence non-materiel procurements valued at more than $200,000 (including GST). This requirement excludes ICT major projects, such as the ERP program.

75 The conditions of tender for the procurement set out that Defence could conduct post tender submission ODIAs at its discretion, in order to clarify, improve and maximise value for money.

76 The System Service Concept Procurement Plan–Phase One explains that the long-term incremental contracting method would consist of a head agreement with orders under the head agreement pertaining to specific functionality or capability.

77 These processes were: procure to pay; source to distribute; plan to report; project concept to close; develop to retire materiel/assets; estate and service delivery; other finance processes; and other logistics processes.

78 The Shortlisting Evaluation Report for the ITR rated IBM and Accenture equal first, on the basis of their experience as prime systems integrators on comparable programs at a low technical risk to Defence, as well as other factors such as commercial innovativeness and soundness of understanding of the requirements of the Defence ERP program. The other two tenders were assessed as posing higher technical risk, with an inability to demonstrate the requisite degree of prime systems integration experience with SAP DFPS or S/4 HANA on comparable programs.

79 The initial RFT involved a conditions of tender document, which set out that tenderers were not likely to be required to submit a tender prior to the completion of the Offer Definition and Improvement Activity (ODIA) process, and that details of tender requirements would be issued following the completion of the ODIA.

80 The appointment of the Probity Advisor and probity and conflict of interest arrangements for the systems integrator procurement are discussed from paragraph 3.58 onwards.

81 See paragraphs 3.39–3.40 for discussion of the updated RFT.

82 The tasking statements divided the systems integration services for the program as a whole into three packages of work, with each including a statement of work. Tasking Statement 1 described Defence’s requirements for systems integration services to support the SAP implementation within Tranche 1. Tasking Statement 2 described Defence’s requirements for systems integration services to support the SAP implementation within the Prepare and Explore phases for all other tranches. Tasking Statement 3 described Defence’s requirements for systems integration services to support the SAP implementation within the Realise, Deploy and Run phases for all other tranches.

83 Although the CIO had not been formally appointed delegate by the Associate Secretary, available evidence indicates there was an understanding at the working level that the CIO would be the delegate.

In response to the CIO’s request, on 6 April 2018 the tenderers were asked to provide information on the impact on their tender response if: the Model Defence Organisation was delivered four months after the systems integrator started work; the Model Defence Organisation was used by Defence as reference value, but not provided to the systems integrator; or if some of the deliverables to be provided by Defence were reassigned to the systems integrator.

84 The non-compliances related to proposed amendments by the shortlisted tenderers to provisions of the draft Deed of Standing Offer provided as part of the October 2017 RFT. For example, the two shortlisted tenderers both proposed a time and materials cost model, rather than a fixed price pay-for-performance model.

85 These intentions had been communicated in writing on 16 August 2018.

86 Section 23 of the Public Governance, Performance and Accountability Act 2013 (PGPA Act) authorises the accountable authority of an entity (in Defence’s case the Secretary, who may delegate the power) to enter, on behalf of the Commonwealth, into commitments of public money (called ‘relevant money’ in the Act) and financial arrangements such as contracts.

87 The ANAO examined all section 23 commitment approval documents and their corresponding Official Orders or contract variations for Tranche 1 of the program which were signed between 19 July 2019 and 26 March 2021.

88 There is evidence that Defence assessed environment risk as low for the overall program and advised the government accordingly. See Table 2.5. Defence advised the ANAO in July 2021 that its position is that environmental sustainability was taken into account. Defence further advised the ANAO that ‘It is Defence’s position that a software upgrade, with no new facility requirements, poses extremely low, if any, environmental risk.’

89 ROMAN is Defence’s finance system, which will be replaced through the ERP program.

90 Defence advised the ANAO that the error in AusTender resulted from Defence’s automatic AusTender reporting tool ignoring the correct total contract value. One variation involved a reduction in price, below the reporting threshold for AusTender. The form submitted to AusTender for a further variation provided the correct total contract amount, but this was not reflected on AusTender.

91 The Defence tool for uploaded data is in Defence’s finance system, ROMAN, which is in scope for replacement through the ERP program.

92 The date range on AusTender is incorrect for this contract, and should be 15 December 2015 to 30 April 2016. This contract was published late to AusTender, being executed on 8 January 2016 and published on 22 March 2016, 74 days after execution. The AusTender requirement is to publish 42 days after execution.

93 The date range on AusTender is also incorrect for this contact, which was extended to 30 June 2020. The additional funds were reflected in AusTender, but not the extension date.

94 Although a contact register was not mentioned in the 2018 updated probity framework, Defence continued to maintain a contact register.

95 Agreement to the confidentiality agreement is documented in the conflict of interest register.

96 See evidence of the CIO as documented in the Hansard record: Commonwealth, Senate Foreign Affairs, Defence and Trade Legislation Committee, 23 October 2019, pp. 43–49.

97 Department of Finance, Procuring from a Panel – Panels 101, Finance available from https://www.finance.gov.au/government/procurement/buying-australian-government/panel-arrangements [accessed 21 May 2021].

98 See paragraphs 2.50–2.51 and Box 1 for information on the ERP program’s ‘above the line/below the line’ contractor model.

99 One relevant Defence email exchange in November 2020 relating to the procurement of contractors stated that: ‘We need to do a very short and sharp “market research” exercise to satisfy our Commonwealth procurement rules around marketing testing/sole sourcing – our commercial team is being audited by the ANAO at the moment so we’re trying to be squeaky clean. This involves asking a couple of other suppliers whether they have anyone with equivalent skills and experience who’s ready immediately (not likely to happen – we just need to do it to cover the audit trail requirements). I sent the role description off to them this morning and asked for a response by Monday. With any luck, we’ll have the contract documents finalised for [name of contractor] by end of next week and then [they will] be able to start as soon as possible after that.’

100 Auditor-General Report No. 4 2020–21 Establishment and Use of ICT Related Procurement Panels and Arrangements, paragraph 4.34, available at https://www.anao.gov.au/sites/default/files/Auditor-General_Report_2020-21_4.pdf [accessed 23 July 2021].

101 The policy defines ‘working meals’ as meals that: ‘are meals to a simple standard (i.e. morning and afternoon teas and light meals); do not include alcoholic beverages; are for the purposes of refreshment rather than entertainment; and are appropriate when cost and/or time advantages exist in continuing work through the normal meal break (i.e. there are efficiency and effectiveness benefits).’

102 As discussed in paragraph 3.22, Defence’s intent was to engage the systems integrator under separate contracts for different work packages.

103 The categorisation of issues reflected that adopted for the tender assessment and initial ODIA process. ‘Solution’ refers to issues which go to IBM’s approach to the Systems Integrator role and the ERP solution. ‘Financial’ refers to pricing and related matters. ‘Commercial’ refers to commercial/contractual matters including the terms of the Deed of Standing Offer and Official Order 1.

104 Defence advised the ANAO that Head Table discussions took place during the Post Tender Submission ODIA period, and involved weekly meetings with tenderer and Commonwealth leads to undertake a status check, discuss issues of concern and clarify questions arising.

105 Deliverables are assessed and approved according to a documented acceptance process. Work products are documents that are not required to follow the formal approvals processes that other deliverables are subject to, as they are constructs used by the systems integrator to deliver smaller parts of formal deliverables.

106 Official Order 1 required the Systems Integrator to develop and maintain an integrated project management plan for Tranche 1 aligned to the ERP program methodology and schedule, including an integrated master schedule. The integrated master schedule determines the due dates for all deliverables under each Official Order in line with the ERP program schedule.

107 For example, the Tranche 1 Plans listed in the orders included: the Integrated Master Schedule; Implementation Plan; Master Resource Schedule; Quality Management Plan; Risk Management Plan; Transition-In Plan; and Release 1A Sustainment Services Governance Plan.

108 Examples of the standards to be applied include: SAP Activate methodology; Managing Successful Programs methodology; and applicable Defence and other Commonwealth standards and reference documents.

109 As at 1 April 2021, the commercial team is comprised of a director (contracted), and eight other personnel (three APS and five contractors).

110 See paragraph 2.47.

111 Seven data entry errors (3 per cent of the total number of deliverables reviewed) were identified by Defence during the ANAO’s review of the tracker.

112 Approvers are APS, ADF and contractors, including: business leads (embedded APS and ADF from the Groups and Services, responsible for supporting program design and implementation in their relevant business areas); data, deployment, design and integration directors; the deputy program director; and the program director. The process for accepting deliverables has been documented in the ERP program handbook, dated August 2020. The process involves a 10 or 20 day approval cycle depending on complexity, impact, risk, internal and external stakeholders and commercial arrangement and ownership.

113 Of the remaining 44, 13 were cancelled (no longer required, rejected, or withdrawn); 18 were in progress (endorsed for impact assessment or commercial assessment by the Change Control Board); six were on hold (change has been put on hold, with potential for future reinstatement); and seven were open (identified, registered, and undergoing assessment). Defence advised the ANAO that the PCR register is used for internal management purposes, as well as for reporting in weekly and monthly status reports.

114 Note that one contract change can pertain to multiple PCRs. The 41 PCRs involving a contract change primarily related to: revised scope in an existing deliverable (19); change in a deliverable due date (13); combining of deliverables (four); additional deliverables (two); deliverable clarification (two); and splitting of a deliverable (one).

115 There was one exception to this: an item which Defence advised the ANAO should have been marked ‘cancelled’.

116 There was one exception to this, with a PCR approved at the Internal Program Board in November 2019. The Internal Program Board and Change Control Board membership is largely the same, including the contracted Band 2 Program Director, ADF 2 Star Head Business Transformation, and IBM delivery executive. The role of the Change Control Board is discussed in Box 2 on p. 72.

117 ‘Approved and proceeding to CVP [contract variation proposal]’ or ‘approved and endorsed for commercial assessment’ are the expressions used consistently in the board minutes reviewed by the ANAO.

118 The financial approval by the delegate is a separate process flowing from the requirements of the PGPA Act and Defence’s Accountable Authority Instructions and FINMAN 2. It follows the substantive deliberative and decision-making processes of the Change Control Board documented in Defence’s ERP Program Management Plan and Program Handbook. See paragraphs 4.36–4.37 for discussion of managing conflicts of interest in the Program Change Request process.

119 Of the 15 contracts and variations requiring section 23 commitment approvals (as at May 2021), 12 commitment approvals were signed by the Deputy Program Director or acting Program Manager (as discussed in paragraph 3.44, one of these approvals was not a valid exercise of delegations). In 10 of these 12 instances, the contract or variation was also signed by the same individual. The remaining section 23 commitment approvals and contract variations were signed by other individuals with SES Band 1/ADF 1 Star delegations or above, including the ADF 2 Star Head Business Transformation. The Head Business Transformation role was described as ‘reports to [the Program Director]’ in an email from the CIO to staff, on 20 September 2019. The Head Business Transformation provided assurance to the ANAO on 19 May 2021 that he receives adequate information to inform decision-making.

120 Further issues relating to positional authority are discussed in paragraph 4.42 below.

121 See paragraph 2.49 and Figure 2.2.

122 The PGPA Act allows contractors to be prescribed as officials if they will exercise a power, perform a function or discharge a duty conferred by the PGPA Act or Rule. However, contractors holding key positions in the ERP program have not been designated as officials for the purposes of the PGPA Act.

123 Issues relating to positional authority were also discussed in paragraph 4.30.

124 Program objectives and outcomes are set out in Table 4.3.

125 SES Band 3/ADF 3 Star personnel are Benefit Owners, accountable for business transformation activities, and the realisation and reporting of benefits to the ERP program.

126 Defence advised the ANAO that benefit realisation planning is due to commence in Quarter 2 2021.

127 Defence advised the ANAO that while governance arrangements for ERP benefits reporting are yet to be determined, the reporting is likely to go to the ETB.

128 GovERP is a planned common whole-of-Government ERP system, to be administered by the Department of Finance.

129 Department of Defence, 2020 Force Structure Plan, July 2020, p. 83.