The objective of the audit was to assess whether the Australian Bureau of Statistics (ABS) has established effective risk management arrangements to support the implementation of the Statistical Business Transformation Program.

Summary and recommendations

Background

1. The Australian Bureau of Statistics (ABS) is an independent statutory body, providing official statistics on a wide range of economic, social, population and environmental matters. The primary functions, duties and powers of the ABS are set out in the Australian Bureau of Statistics Act 1975, the Census and Statistics Act 1905 and the Public Governance, Performance and Accountability Act 2013.

2. The Statistical Business Transformation Program (Program) is a major business re-engineering program to address a significant risk of statistical system failure resulting in an inability to deliver quality, relevant and timely data to ABS customers. It is intended to replace a large number of disparate systems and processes with an integrated, enterprise-wide business architecture solution that would reduce the risk of system failure, increase efficiency and improve access to data. In December 2014, the Government approved $257 million to implement the Program. An additional $13 million was provided in the 2018–19 Budget.

Rationale for undertaking the audit

3. The 2016 Australian Census of Population and Housing highlighted the need for the ABS to have a strong risk management framework. The Statistical Business Transformation Program represents a significant investment of public resources and there are major risks involved with such a large and complex program of work. The audit will provide assurance regarding the adequacy of risk management arrangements underpinning the delivery of the Program. Any suggestions for improvement or recommendations from the audit could usefully inform the delivery of the remaining elements of the Program and assist the ABS to improve its approach to risk management.

Audit objective and criteria

4. The objective of the audit was to examine whether the ABS has established effective risk management arrangements to support the implementation of the Statistical Business Transformation Program.

5. To form a conclusion against the audit objective, the Australian National Audit Office (ANAO) adopted the following high level criteria:

  • Has the ABS established an effective enterprise-level risk management framework?
  • Are identified Program risks being effectively managed?

Conclusion

6. Risk management arrangements to support the implementation of the Statistical Business Transformation Program are effective except for the requirement to monitor and assess risk treatments and take corrective action. The ABS enterprise-level risk management framework is not fully effective.

7. The ABS has established an enterprise-level risk management framework that partly meets the minimum requirements set out in the Commonwealth Risk Management Policy. The framework has not been fully embedded into business processes and procedures. The ABS cannot demonstrate that it actively manages its strategic risks or provides regular executive oversight of strategic risk.

8. The ABS has established an appropriate risk management framework for the Statistical Business Transformation Program, which outlines the strategy and processes for managing risk within the Program. Program risks have been identified and assessed and treatments planned in accordance with the framework. The ABS has not always met the requirement to monitor the effectiveness of treatments and take action where responses do not meet expectations. The ABS has taken steps to improve risk management capability within the Program.

Supporting findings

Establishing an effective enterprise-level risk management framework

9. The ABS has developed a risk management framework that partly meets the requirements of the Commonwealth Risk Management Framework. The ABS cannot demonstrate that it actively manages its strategic risks and has not implemented recommended improvements to the risk management framework.

10. Reviews of the ABS have identified that its risk management framework is embedded into some parts of the organisation. However, inconsistencies in risk management guidance and work practices, and limited executive level oversight of strategic risk indicates that the framework is not fully embedded into all business processes and procedures.

11. The ABS has identified a target level of risk management capability and, in 2018, assessed that it meets that target level. However, key risk management activities relating to capability have been identified that have not yet been completed.

Managing Statistical Business Transformation Program risks

12. Sound Program governance arrangements have been articulated in the Program governance plan. Risk is primarily the responsibility of the Program Delivery Board, which regularly addresses risk related matters.

13. The ABS has developed an appropriate Program risk management framework and implementation is largely effective, but treatments are not always managed in accordance with the framework. The ABS has not updated the Program risk appetite statement since 2015.

14. The risk that the ABS will not have sufficient funds to fully implement the Program has not been managed effectively. The ABS has not quantified the scale of funding issues or revised the Program costs to reflect changing circumstances.

15. The ABS has identified a shortage of project and program management capability within the Program and taken steps to increase the level of skill in this area.

16. The ABS has arrangements in place to communicate with stakeholders and report on Program risks. Internal reporting includes detailed information about the status of risks and issues. Communication to external stakeholders is more general in nature and focuses on broader ABS transformation and associated risks.

Recommendations

Recommendation no.1

Paragraph 2.26

The ABS:

  1. finalise its risk management framework and ensure that the revised framework complies with the Commonwealth Risk Management Policy and is embedded into its processes and procedures; and
  2. implement an effective process to manage strategic risks.

Australian Bureau of Statistics response: Agreed.

Recommendation no.2

Paragraph 3.49

The ABS update the total Program cost estimate, incorporating all work yet to be completed in accordance with the revised Program schedule, and effectively manage the Program budget to ensure that the Program achieves the intended benefits and meets Program outcomes.

Australian Bureau of Statistics response: Agreed.

Recommendation no.3

Paragraph 3.51

The ABS monitor Program risk treatments and take action when treatments are not effective.

Australian Bureau of Statistics response: Agreed.

Summary of the Australian Bureau of Statistics’ response

17. A summary of the ABS’ response is below and the full response is at Appendix 1.

The ABS has been managing risk over its 113 year history with external expectations that we deliver perfect statistics with perfect processes. The challenge for the ABS has been to maintain this standard of delivery for around 500 statistical releases a year using increasingly complex technical solutions in the face of tightening resourcing. The threats associated the new technologies are themselves growing at an accelerating rate. The ABS has meanwhile been assiduous in managing the risks associated with its Statistical Business Transformation Program. The Department of Finance Gateway review of December 2017 found that “the Program is proceeding well. Significant progress has been made in many areas including governance, transition planning, risk management, benefits management and change management. The Program is well placed to meet its original outcomes … Since the last review some risks have been effectively resolved while some new risks have emerged. The ABS has recognised these risks and is working to mitigate them as far as possible.” The Gateway Review noted in relation to the Independent Assurer for the program that “the external assurance provided by KPMG continues to contribute to effective Program risk management”.

The Statistical Business Transformation Program will be an important enabler for the future of ABS. The Program is the subject of unrelenting vigilance at every level of governance, not least the ABS Executive Board itself where risk is inherently the focus of every decision within the Program. The challenge throughout has been to balance the technical challenge of business transformation with resources — a matter of ongoing concern, constant revision, and regular consultation with Government - and the inevitable uncertainties associated with migrating from aged legacy systems to new platforms while managing statistical risk. ABS will continue to manage the budget for the Statistical Business Transformation Program, consulting closely with Government, and to monitor and manage the broader suite of risks associated with the Program.

At the same time ABS has taken steps to formalise its enterprise level management of risk in accordance with the Commonwealth Risk Management Policy and will continue its efforts to improve. This follows action in preceding years to improve our management of statistical risks for Australia’s essential national statistics, starting with our main economic and population statistics and progressing through our statistical program.

Key learnings for all Australian Government entities

Below is a summary of key learnings, including instances of good practice, which have been identified in this audit that may be relevant for the operations of other Commonwealth entities.

Governance and risk management

Program Implementation

1. Background

1.1 The Australian Bureau of Statistics (ABS) is an independent statutory body, providing official statistics on a wide range of economic, social, population and environmental matters. The data produced by the ABS is used by governments in the development and implementation of public policy, and by business, non-government organisations and the wider community. Key categories of statistical information provided by the ABS are shown in Table 1.1.

Table 1.1: ABS statistical categories

Economy

Key Economic Indicators, Business Indicators, Finance, Government, International Trade, National Accounts, Price Indexes and Inflation.

Industry

Industry Overview, Agriculture, Building and Construction, Energy, Mining, Retail and Wholesale Trade, Technology and Innovation, Tourism and Transport.

People

Aboriginal and Torres Strait Islander Peoples, Crime and Justice, Culture and Recreation, Education, Housing, People and Communities, Population.

Labour

Earnings and Work Hours, Employment and Unemployment.

Environment

Environmental Management.

Health

Causes of Death, Disability, Health Conditions and Risk Factors, Health Services, Mental Health.

Snapshots of Australia

Australian Social Trends, Data by Region, Gender Comparisons, Measures of Australia’s Progress, Wellbeing of Individuals and Communities, Year Book Australia, Historical Releases.

   

Source: ABS website, accessed 6 July 2018.

1.2 The primary functions, duties and powers of the ABS are set out in the Australian Bureau of Statistics Act 1975, the Census and Statistics Act 1905 and the Public Governance, Performance and Accountability Act 2013. The ABS’ purpose is to inform Australia’s important decisions by partnering and innovating to deliver relevant, trusted, objective data, statistics and insights. The ABS also works in partnership with other countries and international organisations on statistical matters, including ensuring consistency with internationally accepted frameworks.

The Statistical Business Transformation Program

1.3 In its 2012–13 Annual Report the ABS announced ‘ABS 2017’, an organisational transformation program to address concerns about the impact of ageing and increasingly fragile business processes and supporting infrastructure. This program sought to: address the risks associated with the ageing processes and infrastructure; enable sustainment of statistical programs; support statistical demands of the future; and establish more easily accessible statistical information. The core of ABS 2017 was the introduction of a robust information management framework and supporting infrastructure, for which the ABS would require government funding.

1.4 In 2014 the ABS sought $292 million in government funding for an integrated package of reform measures, which included investment in critical infrastructure to replace the ageing systems and business processes previously identified. In December 2014, the Government approved $257 million to implement the Critical Statistical Infrastructure Program.1 Funding was included in the 2015–16 Budget and phased over five years from 2015 to 2020 (see Table 1.2). An additional $13 million was provided in the 2018–19 Budget.

1.5 The Critical Statistical Infrastructure Program was described, in the ABS Second Pass Business Case, as a major business re-engineering program to address a significant risk of statistical system failure resulting in an inability to deliver quality, relevant and timely data to ABS customers. It is intended to replace a large number of disparate systems and processes with an integrated, enterprise-wide business architecture solution that would reduce the risk of system failure, increase efficiency and improve access to data.

1.6 The Critical Statistical Infrastructure Program was subsequently re-named the Statistical Business Transformation Program (Program).

Table 1.2: Statistical Business Transformation Program budget, as at 30 June 2018

Source

2015–16

$000

2016–17

$000

2017–18

$000

2018–19

$000

2019–2020

$000

Total

$000

2015–16 Budget

47,537

66,646

60,942

44,518

36,786

256,429

ABS contribution

5,924

8,275

10,719

15,327

4,290

44,535

2018–19 Budgeta

 

 

8,985

4,048

 

13,033

Total

53,461

74,921

80,646

63,893

41,076

313,997

             

Note a: Funding of $15.8 million was approved in the 2018–19 budget. The ABS informed the ANAO that $13 million is for additional Program costs. The remainder is for other areas in the ABS.

Source: ANAO analysis of ABS documentation.

1.7 The Program, which commenced in early 2015, consists of 27 independent components that, when complete, are expected to form an integrated system. A diagrammatic representation of the components of the Program is included at Appendix 2.

1.8 The Program applies to all ABS data collections except for the Australian Census of Population and Housing.2 The Program’s business case outlined five objectives, now referred to as benefit areas, which are detailed in the ABS’ Benefits Management Plan (see Figure 1.1).

Figure 1.1: Program benefits

A table outlining the five benefit areas the Program is expected to deliver: reduced statistical risk; reduced cost of ABS operations; Improved client responsiveness; improved capability to grow the business; and reduced red tape

Source: ABS, Benefits Management Plan, 9 November 2017, p. 6.

Risk management

1.9 The ABS Risk Management Framework defines risk as the ‘effect of uncertainty on objectives (positive and/or negative).’3 An issue is defined as ‘something that has happened and must be managed. It is a risk that has been realised, which may or may not have been an identified risk.’4

1.10 The 2016 Australian Census of Population and Housing (eCensus5) highlighted the need for robust risk management practices to ensure that treatments for identified risks are effective. On 9 August 2016, the ABS online lodgement system for eCensus data suffered a series of external system attacks that resulted in public unavailability of the system and later suspension of the eCensus website by the ABS. A review by the Department of the Prime Minister and Cabinet found that the risk of an attack of the kind that occurred on Census night had been specifically identified, but the impact of such an attack was underestimated. Further, there was a lack of focus on the effectiveness of treatments that were implemented to mitigate the risk.6 In December 2016 the Australian Statistician stated that ‘on the surface, we had a regime for risk management in place’, including that the risk of an attack was identified and assessed, and mitigations documented and reported. He then stated ‘however, the mitigations were not adequate.’7

Rationale for undertaking the audit

1.11 The 2016 eCensus events highlight the need for the ABS to have a strong risk management framework. The Statistical Business Transformation Program represents a significant investment of public resources and there are major risks involved with such a large and complex program of work. The audit will provide assurance regarding the adequacy of risk management arrangements underpinning the delivery of the Program. Any suggestions for improvement or recommendations from the audit could usefully inform the delivery of the remaining elements of the Program and assist the ABS to improve its approach to risk management.

Audit approach

1.12 The objective of the audit was to examine whether the Australian Bureau of Statistics has established effective risk management arrangements to support the implementation of the Statistical Business Transformation Program.

1.13 To form a conclusion against the audit objective, the Australian National Audit Office (ANAO) adopted the following high level criteria:

  • Has the ABS established an effective enterprise-level risk management framework?
  • Are identified Program risks being effectively managed?

1.14 The audit team examined the risk frameworks, policies and procedures established at the enterprise-level and the ABS’ processes for identification and management of risks associated with the delivery of the Statistical Business Transformation Program. The audit did not examine the management of risks throughout planning and implementation of eCensus 2016.

1.15 In undertaking this audit, the ANAO analysed departmental records and data, and interviewed key managers and personnel at the ABS.

1.16 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $389,000.

1.17 The team members for this audit were Jennifer Myles, Veronica Clement-Jones and Deborah Jackson.

2. Establishing an effective enterprise-level risk management framework

Areas examined

This chapter considers whether the Australian Bureau of Statistics (ABS) has: established an effective enterprise-level risk management framework and incorporated it into its processes; and maintained an appropriate level of risk management capability.

Conclusion

The ABS has established an enterprise-level risk management framework that partly meets the minimum requirements set out in the Commonwealth Risk Management Policy. The framework has not been fully embedded into business processes and procedures. The ABS cannot demonstrate that it actively manages its strategic risks or provides regular executive oversight of strategic risk.

Areas for improvement

The ANAO has made one recommendation aimed at ensuring the ABS embeds a compliant risk management framework into its processes and procedures and manages its strategic risks.

Has the ABS established an enterprise-level risk management framework?

The ABS has developed a risk management framework that partly meets the requirements of the Commonwealth Risk Management Framework. The ABS cannot demonstrate that it actively manages its strategic risks and has not implemented recommended improvements to the risk management framework.

The ABS risk management framework

2.1 The ABS Risk Management Framework was endorsed in October 2015 and remains in force as at 26 June 2018. It consists of two parts: the Risk Policy and the Risk Guidelines.

The Risk Policy

2.2 The Risk Policy contains high-level statements about the ABS’ approach to risk management, its risk appetite, culture and responsibilities, as follows:

  • approach — an integrated approach to risk management encompassing policy, governance, planning and reporting and assurance activities; risk controls will take into account interdependencies and assess the effectiveness of controls;
  • risk appetite — includes risk appetite statements for statistical risk, enabling services risk and the Statistical Business Transformation Program (Program);
  • risk culture — develop and maintain a culture with strong leadership, collaborative and cooperative approaches and responsible decision-making;
  • responsibilities — assigns a range of specific risk management responsibilities.
The Risk Guidelines

2.3 The Risk Guidelines contain information about the risk management process, risk categories and thresholds, and actions for managing risks. It includes an overview of the risk management process (see Figure 2.1).

Figure 2.1: ABS risk management process

A figure showing the seven elements of risk management, including five consecutive activities: establish the context; identify the risks; analyse the risks; elevate the risks; treat the risks, and two continuous activities that apply throughout the proces

Source: ANAO analysis of ABS documentation.

2.4 The ABS risk management process is further expanded in the document and relevant examples are included. The Risk Guidelines also include a brief section on likelihood and consequence thresholds, risk categories and risk assessment matrices. The guidelines specify a preference for the use of a four by four matrix to analyse risks and assign a risk rating (see Figure 2.2). Risks rated at medium or higher are to have a treatment plan developed and be included in a risk register.

Figure 2.2: ABS Risk Management Framework risk rating matrix

A figure depicting the matrix used to analyse risks and assign a risk rating. The matrix includes four likelihood ratings ranging from rare to almost certain and four consequence ratings ranging from insignificant to catastrophic. The cross-section provid

Source: ABS Risk Management Guidelines, 2015, p. 27.

Compliance with the Commonwealth Risk Management Policy

2.5 As a non-corporate Commonwealth entity, the ABS is required to implement the Commonwealth Risk Management Policy, which includes 22 specific requirements organised in nine policy elements. The ANAO assessed the ABS Risk Management Framework’s level of compliance with the Commonwealth Risk Management Policy. The results are summarised in Table 2.1. A more detailed evaluation of compliance against each requirement can be found at Appendix 3.

Table 2.1: ABS’ Risk Management Framework’s compliance with the Commonwealth Risk Management Policy

Element

Number of requirements

Level of compliance

Establishing a risk management policy

4

Met

Establishing a risk management framework

9

4 requirements met

3 requirements partly met

2 requirements not met

Define responsibility for managing risk

3

Met

Systematic management of risk is embedded in business processes

1

Partly met

The framework must support the development of positive risk culture

1

Partly met

Communicate and consult about risk with stakeholders

1

Not met

Contribute to the management of shared risks

1

Partly met

Maintain an appropriate level of capability

1

Not met

Review risks, risk management framework and the application of risk management practices and implement improvements

1

Partly met

     

Source: ANAO analysis of the ABS Risk Management Framework.

Comcover Risk Management Benchmarking Survey — overall maturity rating

2.6 The Comcover annual Risk Management Benchmarking Survey enables an entity to self-assess the maturity of its risk framework against the nine elements of the Commonwealth Risk Management Policy.8 A six level maturity model is used to assess the level of alignment an entity has achieved with the Policy and allows entities to analyse their risk management capability and track their performance. Survey results include the entity’s current overall maturity level and its target and current maturity level against the nine elements. In 2018 the ABS assessed its overall maturity rating as ‘Integrated’ (see Figure 2.3).

Figure 2.3: ABS’ self-assessed overall risk management maturity levels 2015–2018

A chart that shows ABS self-assessed overall risk management maturity ratings. There are six levels: Fundamental; developed; systematic; integrated; advanced; and optimal. The ABS assessed its maturity rating at: systematic in 2015; integrated in 2016; sy

Source: ANAO analysis of Comcover Risk Management Benchmarking Survey data, 2015 to 2018.

2.7 The self-assessed rating of ‘Integrated’ does not align with the status of key activities in the ABS’s Risk Action Plan (see paragraph 2.15 and Table 2.2) and the ANAO’s findings. For example:

  • the current risk management framework does not comply with the Commonwealth Risk Management Policy and work on a revised framework is not complete (see Table 2.1 and Table 2.2);
  • the ABS has not actively managed its strategic risks (see paragraphs 2.17 to 2.19); and
  • there has been limited executive oversight of strategic risk (see paragraphs 2.21 to 2.24).

Reviewing the Risk Management Framework

November 2016 internal audit

2.8 In November 2016, an ABS internal audit reviewed the Risk Management Framework. It reported that risk management related initiatives undertaken since 2015 had identified weaknesses in the application of risk management across the ABS.

2.9 The internal audit found that risk management was conducted inconsistently across each risk and across each business unit. The root causes included a lack of overall strategy or direction for risk management, a perception that the risk appetite statement was unrealistic and the absence of checks in the governance structure. A total of 18 actions under five categories were proposed to enable the ABS to achieve a fit-for-purpose risk management framework. As a minimum, the audit recommended the ABS start the following activities:

  • appoint a dedicated, senior, technically competent expert resource;
  • understand the cost benefit relationship between risk and control; and
  • report on risk performance.

2.10 The role of Chief Risk Officer was allocated to the existing Chief Financial Officer in March 2017 and the position is supported by the Risk, Planning and Policy Branch. The Chief Finance Officer/Chief Risk Officer is a Senior Executive Service Band 2 and is responsible for the Finance, Risk and Planning Division. The ABS has revised the governance structure and incorporated risk responsibilities in the ABS Executive Board Terms of Reference and some committees. No other actions have been taken in response to the review.

July 2017 review

2.11 In July 2017, Deloitte Risk Advisory Pty Ltd was engaged to review the state of risk management and develop a risk management strategy and roadmap.9 The Deloitte review assessed the ABS’ level of maturity against the elements of the Commonwealth Risk Management Policy to be between ‘Fundamental’ and ‘Developed’, and its target level between ‘Integrated’ and ‘Advanced’ (see Figure 2.4). The ABS assessed its overall risk management maturity level as higher: ‘Systematic’ in 2017 and; ‘Integrated’ in 2018 (see Figure 2.3).

Figure 2.4: Assessed overall risk management maturity level

Source: Deloitte Risk Advisory Pty Ltd, ABS Risk Management Review — Risk Roadmap and Strategy 18 July 2017.

2.12 Key findings from this review included:

  • understanding of the Risk Management Framework, risk appetite and key risk management concepts varied across the organisation;
  • risk assessment processes were inconsistent; and
  • resources allocated to risk management were limited and/or shared with other responsibilities.

2.13 A comprehensive risk roadmap and strategy, a risk glossary and risk appetite statement were delivered as part of this contract, but have not been implemented by the ABS.

October 2017 consultancy

2.14 Following the Deloitte review, in October 2017 the ABS engaged Aerosafe Risk Management Pty Ltd (Aerosafe) to assist with work relating to strategic risk. Austender records show the contract value of $82,500 covers consultancy services for the period 1 September 2017 to 31 March 2018.10 By 28 June 2018, Aerosafe had submitted two invoices totalling $122, 650.11 On 9 July 2018, the ABS signed an agreement with Aerosafe for $122,650. As at 6 August 2018, the contract details recorded on Austender had not been updated to reflect the revised amount. As at 30 June 2018 Aerosafe had delivered:

  • one-on-one engagement sessions and a risk management workshop with ABS management12;
  • an environmental scan and Enterprise Strategic Risk Context; and
  • ABS strategic risks for 2018–20.
ABS Risk Action Plan

2.15 In July 2017 the ABS developed a Risk Action Plan to enable it to achieve its target maturity level. The plan includes 24 activities to be completed by the end of 2018. Table 2.2 shows the status of key activities included in the plan (see Appendix 4 for further detail of the ABS Risk Action Plan and progress against the activities).

Table 2.2: Key risk management activity status, as at 31 May 2018

Activity

Deadline

Status

Develop risk appetite statements and tolerances

March 2018

In progress

Create ABS strategic risk register and review procedure

June/July 2018

In progress

Create enterprise risk register and process for maintenance

July 2018

Not commenced

Roll out updated Risk Management policy, manual and governance

June/July 2018

On hold

Develop risk reporting mechanisms and tools

November 2018

Not commenced

Update risk management training materials

July/August 2018

In progress

     

Note: The Risk Action Plan incorporates three phases: Foundational; Implementation; and Optimisation. Key risk management activities were selected from the Foundational phase.

Source: ANAO analysis of ABS Risk Action Plan, May 2018.

2.16 The ABS has expended significant time and effort on reviews of its approach to risk management, but has not finalised its framework. The ABS should ensure that the framework is finalised, the Risk Action Plan is implemented and progress is monitored and reported to its executive.

Strategic risks

2.17 Since 2015, the ABS has defined three sets of strategic risks (see Table 2.3).

Table 2.3: ABS strategic risks 2015, 2016 and 2018

2015

2016

2018

Stakeholder engagement

Stakeholder engagement

(last evaluated August 2016)

Delivery of high quality statistical services and products

Statistical quality

Statistical quality

(last evaluated August 2016)

Relevant accessible statistics and information

Delivery of the Statistical Business Transformation Program

Delivery of the Statistical Business Transformation Program

(last evaluated February 2017)

Effective data security, privacy and confidentiality

Workforce capability

Organisational capability

(not evaluated)

Trust and support of the authorising environment

Legislative and corporate risks

Privacy and confidentiality obligations

(last evaluated February 2017)

A skilled, capable and flexible workforce with a high level of wellbeing

 

IT security

(last evaluated February 2017)

Effective transformation of the ABS

 

Relevance of statistics

(last evaluated August 2016)

 

     

Note: Strategic risk descriptions in this table have been summarised from ABS documentation.

Source: ANAO analysis of ABS documentation.

2.18 The Risk Guidelines, which remain current as at 26 June 2018, refer to five strategically important risks, listed in the first column of Table 2.3 above. In May 2016 the ABS articulated seven strategic risks, listed in the second column of Table 2.3 and developed treatment plans for six of these seven strategic risks.13 The treatment plans indicate the risks were to be evaluated in August 2016, February 2017 and August 2017. None of the risks were evaluated in accordance with the schedule. In March 2017 the ABS Audit Committee acknowledged that the 2016 strategic risks ‘are not the right strategic risks for the organisation to be focussing on’ and was advised that work was to commence to develop a new approach to identifying and managing strategic risks. On 1 June 2018, in its response to the ANAO’s audit findings, the ABS acknowledged that the 2016 risks were no longer current and were not being actively managed. In June 2018, a revised list of six strategic risks was endorsed, as listed in the third column of Table 2.3.

2.19 None of the risks referred to in Table 2.3 have been included in a risk register, or been subjected to the systematic application of management policies, procedures and practices referred to in the ABS risk management framework.

Is the enterprise-level risk management framework embedded into business processes and procedures?

Reviews of the ABS have identified that its risk management framework is embedded into some parts of the organisation. However, inconsistencies in risk management guidance and work practices, and limited executive level oversight of strategic risk indicates that the framework is not fully embedded into all business processes and procedures.

2.20 The internal audit of ABS risk management in November 2016 and the review of July 2017 made a number of findings that indicate that some areas within the ABS manage risk effectively, but the ABS Risk Management Framework is not embedded in processes and procedures. For example, the reviews found:

  • the ABS Risk Management Guidelines did not include the ABS’ approach to embedding risk management into existing business processes14;
  • inconsistent levels of understanding of the ABS Risk Management Policy and its objectives;
  • no clear linkage between planning, budgeting and reporting processes and risk assessments;
  • inconsistent or ad hoc risk management processes; and
  • limited ability to monitor enterprise risk management performance.

Executive oversight of strategic risk

2.21 ABS records indicate limited oversight of strategic risk. Prior to April 2017 the terms of reference for the senior executive board responsible for oversight of the ABS (the Executive Leadership Group) did not include any risk related responsibilities and the Group’s meeting agenda did not include consideration of risk. In April 2017 the ABS Executive Board replaced the Executive Leadership Group and was allocated the following risk related responsibilities:

  • monitoring of enterprise-wide risks;
  • risk planning and mitigation strategies; and
  • risk management skills, culture and capability development.

2.22 The ABS Executive Board’s fortnightly meeting records from April 2017 to April 2018 indicate:

  • there is no standing agenda item relating to risk;
  • there was no record of discussion or decisions relating to monitoring of enterprise-wide or strategic risks15;
  • there was no record of discussion or decisions relating to risk management skills, culture or capability development;
  • risk planning and mitigation strategies were reported once;
  • risk management processes were discussed twice; and
  • risks relating to the Program, which is classified as a strategic risk, appeared five times.16

2.23 The ABS informed the ANAO that: its strategic risks are driven by the specific risks identified in its programs and projects; that strategic risk management frames discussion in relation to the operations of those programs and projects; and that this approach vests the risk management process in the core business of program and project management.

2.24 The ANAO found evidence that Divisions report to the ABS Executive regularly and include information relating to risks identified within those Divisions, supporting the ABS’ statement that programs and projects address risk. However, the ABS could not demonstrate how it determined whether this information had any impact on the ABS strategic risks shown in Table 2.3.

2.25 In order to ensure that the risk framework is embedded across the entity and risks are effectively managed, the ABS should ensure that oversight of strategic risk occurs in accordance with its governance arrangements.

Recommendation no.1

2.26 The ABS:

  1. finalise its risk management framework and ensure that the revised framework complies with the Commonwealth Risk Management Policy and is embedded into its processes and procedures; and
  2. implement an effective process to manage strategic risks.

Australian Bureau of Statistics response: Agreed.

2.27 Risk management is baked deeply into the management of all ABS programs and projects. The ABS has finalised its 2018–19 strategic risks in June 2018. The ABS is working to align its risk management framework with the Commonwealth Risk Management Policy and will be implementing an effective process to manage the strategic risks. ABS has already enhanced its risk management of our main economic and population statistics.

Has the ABS identified and maintained an appropriate level of risk management capability?

The ABS has identified a target level of risk management capability and, in 2018, assessed that it meets that target level. However, key risk management activities relating to capability have been identified that have not yet been completed.

Comcover Risk Management Benchmarking Survey — capability maturity rating

2.28 The Comcover Risk Management Benchmarking Survey includes element 8 of the Commonwealth Risk Management Policy — maintaining risk management capability. The ABS’ results for element 8 are shown at Figure 2.5. The survey shows the ABS’ target maturity rating fluctuated between 2015 and 2018, while its self-assessed maturity rating has increased overall and now meets its target rating of between ‘Systematic’ and ‘Integrated’.

Figure 2.5: ABS maturity levels for maintaining risk management capability 2015―2018

A chart that shows the ABS self-assessed actual and target maturity ratings for maintaining risk management capability. The ABS assessed its actual rating was: lower than its target level in 2015; higher than its target level in 2016; lower than its targe

Source: Comcover Risk Management Benchmarking Survey, 2015 to 2018.

2.29 Recent reviews of risk management in the ABS (discussed in paragraphs 2.8 to 2.14) found the level of risk management capability needed improvement. The reviews found varying degrees of understanding of key risk management concepts, inconsistent practices, limited ability to monitor enterprise risk management and ad hoc reporting.

2.30 The ABS Risk Action Plan identifies a number of key enterprise-level risk management functions that had not been completed as at 31 May 2018 (see Table 2.2 and Appendix 4). The ABS’ self-assessed increase in risk management capability in 2018 does not align with the incomplete status of these activities.

3. Managing Statistical Business Transformation Program risks

Areas examined

This chapter considers whether the Australian Bureau of Statistics (ABS) effectively manages Statistical Business Transformation Program (Program) risks and maintains an appropriate level of risk management capability, governance and communication.

Conclusion

The ABS has established an appropriate risk management framework for the Statistical Business Transformation Program, which outlines the strategy and processes for managing risk within the Program. Program risks have been identified and assessed and treatments planned in accordance with the framework. The ABS has not always met the requirement to monitor the effectiveness of treatments and take action where responses do not meet expectations. The ABS has taken steps to improve risk management capability within the Program.

Areas for improvement

The ANAO made two recommendations aimed at managing the Program budget and monitoring Program risks.

3.1 Overall responsibility for the Statistical Business Transformation Program (Program) rests with the Senior Responsible Officer, who is the Deputy Australian Statistician (Corporate Services and Transformation Group). The Program’s structure is shown in Figure 3.1. Approximately 350 Australian Public Service staff and over 90 contracted personnel were employed in the Program as at April 2018.

Figure 3.1: Program organisational structure

A hierarchical diagram that shows four Program branches and the Program office overseen by the Statistical Business Transformation Division, which is overseen by the Corporate Services and Transformation Group, which is overseen by the Australian Statistician.

Source: ANAO analysis of ABS Organisational Chart.

3.2 Responsibilities of the Program Office are depicted in Figure 3.2.

Figure 3.2: Program Office functional breakdown

A diagram that shows six sections under the Program Office: Project Management Coaching; Configuration Management; Schedule Management; Risk and Issue Management; Program Performance; and Project Performance.

Source: ANAO analysis of ABS documentation.

3.3 The Program Office provides risk management support to the Program and projects through:

  • authoring the Risk and Issue Management Strategy and Process;
  • participating in the identification, assessment and control of risks and issues;
  • ensuring all Program risks and issues are logged, all fields complete and information updated on a regular basis;
  • co-ordinating monthly review and reporting of Program risks and issues;
  • providing risk and issue management support systems, tools and documents;
  • conducting regular reviews of project risks and issues to ensure consistency, active management and accurate reporting;
  • identifying best practice and opportunities for improvement; and
  • identifying cross project and sub-program related risks for consolidation and/or escalation.

3.4 The Program consists of 27 separate components (depicted in Appendix 2). Each component is managed as an individual project. Project managers are responsible for delivery of products specified in the relevant project management plan. They are also responsible for managing risk within their respective projects and for recording and reporting risks to the relevant Branch Managers.

Have sound governance arrangements been established to monitor the effectiveness of risk management arrangements for the Program?

Sound Program governance arrangements have been articulated in the Program governance plan. Risk is primarily the responsibility of the Program Delivery Board, which regularly addresses risk related matters.

Program governance

3.5 The Program governance arrangements are described in the Statistical Business Transformation Program Governance Plan. An overview of the arrangements in place to oversee the ongoing management of the Program are shown in Figure 3.3. The two main governance bodies for the Program are the Statistical Business Transformation Program Executive Board (Program Executive Board) and the Statistical Business Transformation Program Delivery Board (the Delivery Board).

Figure 3.3: Program governance, as at 30 March 2018

Note: Arrows represent reporting arrangements.

Source: ANAO analysis of ABS documentation.

3.6 The Program Executive Board is the sponsoring body for the Program with investment and strategic responsibilities, and supports the Program’s Senior Responsible Officer. The Program Executive Board meets every two months and is chaired by the Australian Statistician. Membership includes the three Deputy Australian Statisticians and three external members.17 The Program Executive Board’s terms of reference do not include responsibility for risks and issues. The Program Executive Board receives the Program Performance Report, which includes high level information about Program risks and issues.

3.7 The Delivery Board is the primary governance board and is responsible for: delivery, management, monitoring and review of the Program; defining the acceptable risk profile and risk thresholds for the Program and constituent projects; and supporting risks and issues management.18 The Delivery Board undertakes its responsibilities through monthly meetings chaired by the Program’s Senior Responsible Officer.

3.8 The Delivery Board receives a range of regular reports that include risk related information. The monthly Update on Program Risk and Issues report provides the most detailed information on current risks and issues and includes the following information for each risk and issue:

  • risk or issue owner;
  • risks or issues update;
  • risk status (stable, improving or worsening); and
  • treatments including:
    • treatment description;
    • person responsible for completing the treatment;
    • due date for completion; and
    • status (in progress, ongoing, completed).19

3.9 Prior to February 2017 risk related matters were raised intermittently at Delivery Board meetings. In February 2017 Program risk owners presented risk status updates to the Delivery Board. The Board agreed that the active discussion of risk at the meeting was a positive step. Between February 2017 and June 2018, the Delivery Board met 14 times20 and risk related topics were included on the agenda for 11 of those meetings. Discussions included:

  • general risk related matters;
  • risk management processes; and
  • specific Program risks, including two occasions when each risk and issue on the register was discussed.

3.10 The Delivery Board reports to the Australian Statistician through the Program Executive Board. Neither the Delivery Board nor the Program Executive Board report to the ABS Executive Board. The ABS has stated that formal reporting to the ABS Executive Board is not necessary as the members are also members of the Program Executive Board and both Boards are chaired by the Australian Statistician. Formalising the arrangements would help to maintain proper record keeping and ensure that decisions made regarding Program risks are also considered in the context of the ABS as a whole.

Program oversight

3.11 Two mechanisms operate externally to the Program to provide assurance to the ABS executive on the progress of the Program: engaging an assurer; and being subject to Gateway reviews.

3.12 In November 2015, the ABS engaged KPMG as an ‘Independent Assurance Partner’ (Assurer).21 The contract under which KPMG is engaged states that the role of the assurance partner is to provide: assurance advice to the Program Executive Board and the Senior Responsible Officer; and support and advice to the Senior Responsible Officer on Program delivery. There is a potential for conflict when an organisation is contracted to provide assurance and advice services — that organisation may be required to provide assurance about advice it has provided. The ABS should be aware of any potential conflicts and manage them appropriately.

3.13 The Assurer conducts reviews of specific aspects of the Program’s performance in accordance with a predetermined assurance map.22 Twenty-one reports were delivered between February 2017 and June 2018 covering various aspects of Program delivery.

3.14 The Department of Finance Gateway Review Team undertook Program Reviews in 2015 and 2016, and a Mid Stage Program Review in 2017.23 The Mid Stage Review found that the Program met the review requirements and assessed five of six focus areas as ‘Green’.24 Risk Management was found to be ‘Amber’25 and the subject of three of six recommendations. A further review has been scheduled for January 2019.

ABS Audit Committee

3.15 The Public Governance, Performance and Accountability Rule 2014 specifies that the function of an entity’s internal audit committee must include reviewing the appropriateness of the entity’s:

  • financial reporting;
  • performance reporting;
  • system of risk oversight and management; and
  • system of internal control.

3.16 The ABS Audit Committee Charter includes the requirement to satisfy itself that a sound approach has been followed in managing ABS’ major risks, including those associated with projects, program implementation and activities.

3.17 In October 2016 the Australian Statistician advised the Audit Committee that it did not need to undertake assurance or internal audit for the Program as ‘sufficient independent assurance and advice in relation to the [Program]’ was provided through:

  • the Program governance arrangements;
  • regular reports provided to the ABS Executive Board (then ABS Executive Leadership Group)26;
  • engagement of the Assurer; and
  • the Gateway Review process.

3.18 The Assurer and Program representatives regularly attend Audit Committee meetings, and provide:

  • Gateway Review reports;
  • progress updates on implementation of Gateway Review recommendations;
  • Program Performance Management Reports;
  • risk and issue reports; and
  • the Executive Summary of reports completed by the Assurer.

3.19 The Mid Stage Program Review completed by the Gateway Review team in November 2017 stated that the ABS Audit Committee’s engagement with the Program ‘represent[ed] better practice.’27 This statement refers only to the Program; it does not consider the Audit Committee’s role in oversighting enterprise-wide risks.

3.20 In order to fulfil its responsibilities under the Public Governance, Performance and Accountability Rule 2014, audit committees should take an active role in reviewing risk, assurance and operational frameworks to ensure that they support the achievement of the entity’s objectives.28

Is the ABS appropriately managing Program risks?

The ABS has developed an appropriate Program risk management framework and implementation is largely effective, but treatments are not always managed in accordance with the framework. The ABS has not updated the Program risk appetite statement since 2015.

The risk that the ABS will not have sufficient funds to fully implement the Program has not been managed effectively. The ABS has not quantified the scale of funding issues or revised the Program costs to reflect changing circumstances.

Risk appetite for the Program

3.21 A risk appetite statement for the Program was articulated in the Risk Management Framework Part A - The Risk Policy:

We have a relatively high risk appetite early in a project’s life to allow maximum creativity and innovation. The high risk appetite extends to the early phase of a project’s life, to ensure that any potential shortcomings are identified rapidly, before there is a significant impact on cost, time or dependencies. As such, we will test early and learn quickly to help ensure the overall success of the Program.

We will use appropriate program and project methodologies to deliver agreed scope on time and budget and we will choose systems and services that are reliable, fit for purpose, and financially sustainable.

We will actively manage the impact of change through effective engagement and communication across all relevant areas of the ABS and with key external stakeholders to ensure ownership and support for the business solutions implemented as part of the Transformation.29

3.22 Following the events of the 2016 Australian Census of Population and Housing (eCensus), in December 2016 the Program Executive Board reported it needed to ‘review its risk appetite given the changed environment.’30 The ABS informed the ANAO that the Program’s risk appetite did not require updating and remains current and accurate.

3.23 Although the Australian Census of Population and Housing was not intended to be included as part of the Program (see paragraph 1.8), it was recommended as part of the options for a changed delivery approach presented to the Program Executive Board on 27 June 2018 (see paragraph 3.38). This option was reported to the Minister as being endorsed by the Board and is currently being progressed. The risks associated with including Census in the Program have not yet been assessed, are not included in the Program risk register and are not currently reflected in the risk appetite.

Program risk and issue management framework

3.24 The ABS has developed a Program risk and issues management framework that broadly aligns with the ABS Risk Management Framework. The Program’s framework consists of two documents:

  • the Risk and Issue Management Strategy, which includes the Program’s approach to risk management, tools and techniques, an overview of the process, review and reporting arrangements and accountabilities; and
  • the Risk and Issue Management Process, which contains information about the risk and issue registers and includes a four step process, which is shown in Figure 3.4.31

Figure 3.4: Overview of the Program Risk and Issue Management Process

A diagram that shows four consecutive actions for risk management: identify threats and opportunities; assess the likelihood and consequences; plan responses to reduce risk; implement the responses, monitor their effectiveness, and take action where responses do not meet expectations. Communication is a continuous action that is applied throughout the process.

Source: ANAO analysis of ABS documentation.

Managing Program risks

3.25 Program risks and issues fall into two categories:

  • those relating to the delivery of Program benefits; and
  • those relating to delivery of defined project outputs.

3.26 As at 8 June 2018, the ABS was managing 13 Program risks and seven issues (see Appendix 5). Of the 13 risks, nine are rated high, three are medium and one is low. All issues are rated as high except for one rated as medium.

3.27 The ABS uses a range of project management methodologies including Prince232, Managing Successful Programmes33 and Agile.34 Program risks are managed using Managing Successful Programmes. In August 2016, a commercial software tool (Jira) was introduced to record and report project and Program risks and issues.

Implementing the Risk and Issue Management Process

3.28 The following activities are to be undertaken in the Program Risk and Issue Management Process:

  • Identify risk — risks and issues can be raised by anyone at any time, and are often identified through regular project and Program level meetings and workshops. This step involves the description of the risk and communicating with relevant stakeholders.
  • Assess risk — risk assessment is conducted using a four by four risk matrix to determine the risk rating.
  • Plan risk response — the risk or issue is registered, a risk owner is assigned and treatment strategies are developed to mitigate the risk or manage the issue.
  • Implement response — ensure that treatment strategies are actioned, their effectiveness monitored and corrective action taken where responses do not meet expectations.

3.29 The ANAO observed that the ABS conducts the first three steps of the risk management process in accordance with the Program Risk and Issue Management Process. In practice the ABS does not always evaluate the effectiveness of Program risk treatments once they have been applied.35

3.30 On 2 February 2017 the Delivery Board stated that risk descriptions and treatments needed to be improved. Risk descriptions were revised following this directive and are now more concise, but treatments remained the same. On 15 September 2017 the Delivery Board agreed that the measure of ‘ongoing’ was insufficient to measure the progress of treatments for the Program’s cyber security and integration risks (see Appendix 5) and that metrics should be included to track progress. The status for these two risks has not been revised and remained ‘ongoing’ in the 8 June 2018 Update on Program Risk and Issues report.

3.31 The use of terms such as ‘ongoing’ and ‘in progress’ as a risk treatment status is consistently used within Program risk reporting. The term does not provide sufficient information to determine whether the treatment is effective, as required by the Program Risk and Issue Management Process. One key example where the treatments applied to high risks has not been effective in reducing the risk rating is the Program affordability risk.

Program affordability risk

3.32 The risk of insufficient funding to complete the Program was first raised in January 2016, when the Program Delivery Board identified ‘Cannot afford to develop all essential capabilities (cost greater than budget)’ as one of 16 high Program risks.36 The risk, referred to as ‘affordability risk’, was reported as being included on the Program risk register in January 2016. Table 3.1 outlines the treatments that have been in place to address affordability risk up to June 2018.

Table 3.1: Affordability risk treatments, as at 8 June 2018

Treatment

Date applied

Status

Utilise product based planning to establish accurate project budgets

August 2016

Complete

Deliver value earlier in the Program and enable lower value work to be de-prioritised / undelivered

August 2016

Ongoing

Manage scope and schedule to mitigate affordability issues

August 2016

Ongoing

Consider seeking additional funding from elsewhere in the ABS if required

August 2016

Complete

Restructure [Program] organisation to reduce cost and increase velocity

August 2016

Complete

Regularly review project cost and budget throughout Program lifecycle

February 2017

Ongoing

Establish strong mechanisms for budget oversight and review in Program governance

March 2017

Ongoing

Establish Program financial contingency for addressing unanticipated requirements

March 2017

Complete for 2017–18a

Secure agreement with Department of Finance for additional funding and time to mitigate impact of the Australian Marriage Law Postal Survey

October 2017

Complete

Secure agreement with Department of Finance for additional funding and time to mitigate impact of the Australian Marriage Law Postal Survey and approval for re-phasing [Program] funding over financial years

October 2017

In progress

Approve Final Addendum 3 [to the Program Business Case] with Department of Finance and notify [Program] Executive Board that it is approved

March 2018

Not yet commenced

Seek additional funding from government if required

June 2018

Not yet commenced

Replace 50% of contractors (50) with ongoing or non-ongoing staff

June 2018

In progress

Reduce Disseminateb work, focusing on specified key deliverables or stop disseminate work

June 2018

In progress

Stop selected Process and Analyseb work where it makes sense to do so

June 2018

Pending ABS Executive Board decision

Reduce Executive and Program Office resourcing level

June 2018

In progress

     

Note a: Financial contingency of $2.6 million was allocated for 2017–18.

Note b: Disseminate and Process and Analyse are Program components and are depicted in Appendix 2.

Source: ABS, Update on Program Risks and Issues, 8 June 2018.

3.33 The ABS sought additional funding in October 2017, which was received as part of the Budget (see Table 1.2). On 6 April 2018 the ABS noted that affordability risk remained high and that the status of the risk was ‘worsening’.37

3.34 In May 2018 the Program Executive Board reported to the ABS Executive Board that an additional $30.9 million would be required in 2018–19 to progress the Program as planned. A number of savings options were proposed, with the preferred option expected to reduce the additional amount required to $10 million and including the following actions:

  • replace contractors with staff — estimated saving of $3.4 million;
  • reduce Disseminate work — estimated saving of $2.0 million;
  • reduce executive and Program office staff — estimated saving of $0.7 million; and
  • reduce the number of statistical collections onboarded38 in 2018–19 from approximately 70 to 11 — estimated saving of $14.2 million.

3.35 No decision was recorded by the ABS Executive Board, but three of the options were included as treatments for affordability risk (see Table 3.1).

3.36 On 13 June 2018, the Program presented a strategy to prioritise delivery of selected ‘pioneer’ statistical programs. The ABS Executive Board noted the following financial implications of actions recommended:

  • an additional $16.5 million required in 2018–19;
  • an unspecified additional amount required in 2019–20;
  • an estimated $7 million required in 2020–21; and
  • an estimated $1 million required in 2021–22.

3.37 On 18 June 2018, the ABS Executive Board endorsed an additional $10 million for the Program.

3.38 On 27 June 2018 a paper was presented to the Program Executive Broad highlighting Program delays and the need for a different delivery approach. The Program identified the following four initiatives ‘to improve likelihood of delivery’39:

  • prioritise Disseminate work;
  • consider changing the underlying Program model;
  • prioritise data acquisition capability; and
  • increase the number of statistical collections onboarded in 2018–1940;

3.39 The ABS advised the ANAO that the first three initiatives would not require further funds above the $10 million endorsed by the ABS Executive Board on 18 June 2018. The fourth initiative was estimated to cost an additional $6.5 million, which had not been endorsed.

3.40 No decision was recorded by the Program Executive Board, but advice to the Minister indicates that the following options were endorsed:

  • prioritise delivery of enhanced Disseminate capability;
  • review details of the Program technical design; and
  • increase investment to support the onboarding of statistical programs.

3.41 There does not appear to be a correlation between the May 2018 and June 2018 papers. It is also not clear which actions are being progressed with additional funding. For example, according to the June Update on Program Risk and Issues report, Disseminate work is being reduced, whereas the Minister was informed that enhanced Disseminate work was being prioritised.

3.42 The actions taken are designed to reduce financial pressure in 2018–19. The impact of the decisions taken on program affordability in future years is not assessed. The ABS has not quantified the estimated total cost to complete the Program. The options developed and treatments applied to the affordability risk do not address longer-term budget challenges, including the cost to complete any deferred capability and the impact on benefits realisation.

3.43 Budget and expenditure data provided to the ANAO indicates that the ABS is forecasting a total Program underspend. However, the forecast underspend assumes the Program will not exceed its budget in 2018–19 and 2019–20 and will be completed on schedule in 2020. It does not take into account the information provided to the ABS Executive Board and the Program Executive Board on cost pressures in 2018–19 or additional costs associated with treatments for other risks identified in the Program such as:

  • schedule — elevated to an issue in April 2018;
  • capability and capacity — elevated to an issue in October 2016; and
  • the cost of system maintenance.

3.44 In April 2016 the ABS Executive Leadership Group (replaced by the ABS Executive Board) was advised that the Program budget excluded maintenance costs. These costs were expected to be offset by savings across the life of the Program.

3.45 In October 2017, the cost of maintaining the systems implemented by the Program was discussed by the Program Delivery Board. The Program Office provided a paper proposing system maintenance costs be included as a Program issue. The paper specified that the costs associated with licensing and maintenance for the new capabilities delivered by the Program could potentially be higher than the legacy capabilities being replaced. As these costs had not been incorporated into the overall Program costs, there was a potential funding shortfall in the Program budget.

3.46 The Delivery Board decided that the proposed system maintenance costs issue was not a Program issue and requested it be included as an enterprise level risk on the ABS risk register. This has not occurred. Unfunded system maintenance continues to be referred to as a factor effecting the Program affordability risk in the Update on Program Risk and Issues report.

3.47 In summary, the affordability risk has been identified and registered, and a number of treatments have been applied. However, the ABS has not estimated the total Program cost. In addition:

  • different options to address the Program’s budget pressures have been presented to different Boards;
  • some of the options presented have been included in the Program risk register as treatments without a formal decision-making process and approval; and
  • the options developed and treatments applied to the affordability risk refer to the current financial year but do not address longer-term budget challenges and the impact of short-term solutions on the Program in the longer-term.

3.48 Without quantifying the scale of funding issues, it is not possible to identify and implement initiatives to effectively manage affordability risk and to meet the required Program outcomes.

Recommendation no.2

3.49 The ABS update the total Program cost estimate, incorporating all work yet to be completed in accordance with the revised Program schedule, and effectively manage the Program budget to ensure that the Program achieves the intended benefits and meets Program outcomes.

Australian Bureau of Statistics response: Agreed.

3.50 The ABS regularly updates Program costs to reflect the outcomes of recent discussions on Program delivery at the Executive Board. The ABS will continue to revise Program costs to reflect changing circumstances and will continue to effectively manage the program budget in line with available funding to ensure as far as possible the Program delivers agreed outcomes.

Recommendation no.3

3.51 The ABS monitor Program risk treatments and take action when treatments are not effective.

Australian Bureau of Statistics response: Agreed.

3.52 The Statistical Business Transformation Program has updated the reporting and management of risks and issues to more explicitly document the assessment of the effectiveness of risk treatments. The ABS will continue to monitor Program risk treatments and take action where treatments are not effective.

Has the ABS identified and maintained an appropriate level of risk management capability for the Program?

The ABS has identified a shortage of project and program management capability within the Program and taken steps to increase the level of skill in this area.

3.53 Project and program management qualifications include coverage of risk management as an integral part of managing complex projects and programs. In February 2016 the Program’s Assurer reported that 26 per cent of people working within the Program had a project management qualification and 14 per cent had a program management qualification. Based on this finding, the Assurer recommended increasing program management capability to support the Program.

3.54 The ABS developed an action plan to address the Assurer’s suggested actions. These actions included engaging experienced project managers and a program manager to support the Program management team. Actions taken as at June 2018 include:

  • project and program management coaching in place;
  • sub-program co-ordinators introduced to advise project managers;
  • coaches working to improve implementation of the Agile methodology;
  • 57 Program staff trained in Managing Successful Programs;
  • 89 Program staff trained in Prince2;
  • 81 SBTP staff trained in the Agile methodology; and
  • non-ongoing staff and contractors with specialist project management expertise recruited.

3.55 The increase in project and program management capability has the potential to increase the level of risk management capability within the Program. An assessment of risk management capability would identify whether the actions taken have been successful in increasing capability.

Are arrangements in place for communicating, consulting and reporting on risks with stakeholders?

The ABS has arrangements in place to communicate with stakeholders and report on Program risks. Internal reporting includes detailed information about the status of risks and issues. Communication to external stakeholders is more general in nature and focuses on broader ABS transformation and associated risks.

3.56 In April 2016 the Assurer found there would be benefit in enhancing the capability of the Program Office to improve predictive analysis and make reporting less reactive and more responsive. Since then, the introduction of the project management system Jira has contributed to improvements in Program reporting. The Program Office currently produces three reports that include risk related data (see Table 3.2).

Table 3.2: Statistical Business Transformation Program internal reports

Report name

Frequency

Report recipient

Project status reports — for each project

Fortnightlya

Program Delivery Board

Update on Program Risk and Issues report

Monthly

Program Delivery Board

Program performance report (previously Program status report)

Approximately every two months

Program Executive Board

     

Note a: Prior to November 2017, this report was produced monthly.

Source: ANAO analysis of ABS documentation.

3.57 The Update on Program Risk and Issues report provides the most detailed information on risks and issues. As noted in paragraph 3.8, the report includes detailed information about the status of current risks and issues. It provides a concise aggregation of the data held in Jira, but does not include a measurement of the effectiveness of treatments on Program risks and issues, or indicate whether the treatment had any impact on the rating or status.

3.58 In a brief to the Minister for Small Business on 1 September 2016, the ABS undertook to provide an update on the progress of the Program following each Program Executive Board meeting (every two months). Since that time, the ABS has provided updates to the relevant Minister41 on Program status through: a brief section in a fortnightly circular; periodic status reports; and face-to-face briefings by the Australian Statistician.

3.59 The ABS developed an External Engagement Strategy for ABS Transformation in October 2017.42 The strategy includes roles and responsibilities, engagement materials and a list of 37 external stakeholders.43 The objectives of the strategy are that stakeholders:

  • are aware of how the ABS Transformation will impact them;
  • have an opportunity to provide input to the ABS Transformation;
  • expectations are managed through delivery of aligned and coherent transformation messages from across all areas of the ABS; and
  • are satisfied the changes will produce the desired outcome.

3.60 The ABS has developed plans outlining regular engagement activities to be undertaken with external stakeholders, and the ABS presents at various forums where risk is included as a discussion point. These plans and presentations include an element of Program information.

Appendices

Appendix 1 Entity response

Appendix 2 Statistical Business Transformation Program components

 

Statisticians’ Workbench (SWB)

Statistical Workflow Management System (SWMS)

Acquire

Process and Analyse

Disseminate

Data Acquisition and Stakeholder Management

Specialist Tools

Time Series

New Website and Authoring

Register, Frame & Sample

Management

Statistical Account Balancing

Data Processing Environment

Microdata Dissemination & Access

 

Accounts Compilation and Conceptual Adjustment

Data Visualisation and Exploratory Tools

Dissemination, Data Services and Applications

Imputation Tools

Data Visualisation and Exploratory Tools

 

Rules Management Environment

Confidentiality

SuperSTAR

Output Estimation: Core Elements

Foundational Infrastructure

Integration Platform

Enterprise Data Management Environment (EDME)

Statistical Metadata Infrastructure

Metadata Registry and Repository

Metadata Business Object Library

Metadata Authoring Tool

ABS Information Modelling

Security, Identity & Access Management

             

Source: ABS, Statistical Business Transformation on a Page, 20 November 2017.

Appendix 3 ABS’ Risk Management Framework Compliance with the Commonwealth Risk Management Policy

Policy elements

ANAO assessment

Element 1: An entity must establish and maintain an entity specific risk management policy that:

a) defines the entity’s approach to the management of risk and how this approach supports its strategic plans and objectives.

Met

b) defines the entity’s risk appetite and risk tolerance

Met

c) contains an outline of key accountabilities and responsibilities for managing and implementing the entity’s risk management framework

Met

d) is endorsed by the entity’s Accountable Authority.

Met

Element 2: An entity must establish a risk management framework which includes:

a) the overarching risk management policy (Element 1)

Met

b) an overview of the entity’s approach to managing risk

Met

c) how the entity will report risks to both internal and external stakeholders

Not Met

The Risk Guidelines include a section on ‘Communicate and Consult’ and lists benefits of good communication and consultation.

The ABS has not established an enterprise-level risk reporting framework.

d) the attributes of the risk management culture that the entity seeks to develop, and the mechanisms employed to encourage this

Met

e) an overview of the entity’s approach to embedding risk management into its existing business processes

Partly met

The Risk Guidelines state that the ABS aims to integrate risk management into the way it does business, but does not specify how this will be achieved, or how it will be measured.

f) how the entity contributes to managing any shared or cross jurisdictional risks

Partly met

The Risk Guidelines include a section on ‘Managing Shared and Cross-Jurisdictional Risks’. It discusses dependencies and the importance and complexity of collaboration, but does not specify how the ABS contributes to managing shared or cross jurisdictional risks.

g) the approach for measuring risk management performance

Partly met

The Risk Guidelines include details of risk categories, two risk matrices and actions for managing risks by rating. The requirement to assess performance of the risk management actions undertaken is not included.

h) how the risk management framework and entity risk profile will be periodically reviewed and improved

Not met

The framework does not specify a review period.

i) the risk management framework must be endorsed by the entity’s Accountable Authority.

Met

Element 3: Within the risk management policy, the Accountable Authority of an entity must define the responsibility for managing risk by:

a) defining who is responsible and determining an entity’s appetite and tolerance for risk

Met

b) allocating responsibility for implementing the entity’s risk management framework

Met

c) defining entity roles and responsibilities in managing individual risks

Met

Element 4: Each entity must ensure that the systematic management of risks is embedded in key business processes.

 

Partly met

The Risk Guidelines state that the ABS aims to integrate risk management into the way it does business, but does not specify how this will be achieved, or how it will be measured.

Element 5: An entity’s risk management framework must support the development of a positive risk culture.

 

Partly met

The Risk Policy refers to risk culture under the headings ‘Strong Leadership’, ‘Collaborative and Cooperative Approaches’ and ‘Responsible Decision Making’, but does not specify how this will be achieved, or how it will be measured.

Element 6: Each entity must implement arrangements to communicate and consult about risk in a timely and effective manner to both internal and external stakeholders.

 

Not met

The Risk Guidelines include a section on ‘Communicate and Consult’ but does not specify how this will be achieved or how it will be measured.

Element 7: Each entity must implement arrangements to understand and contribute to the management of shared risks.

 

Partly met

The Risk Guidelines include a section on ‘Managing Shared and Cross-Jurisdictional Risks’. It discusses dependencies and the importance and complexity of collaboration, but does not specify how the ABS contributes to managing shared or cross jurisdictional risks.

Element 8: Each entity must maintain an appropriate level of capability to both implement the entity’s risk management framework and manage its risks.

 

Not met

External reviews indicate the level of understanding of risk management concepts and capability in risk management is below target.

Element 9: Each entity must review its risks, its risk management framework and the application of its risk management practices on a regular basis, and implement improvements arising out of such reviews.

 

Partly met

The framework does not specify a review period.

Enterprise level risks are not consistently reviewed, there is no standing agenda item for risk for the ABS Executive Board.

The ABS is currently reviewing its risk management framework and has drafted revised strategic risks.

   

Source: ANAO analysis of ABS documentation.

Appendix 4 ABS Risk Action Plan at 31 May 2018

Activity

Deadline

Status

Comment

1. Foundational activities

Interim update to Risk Management Policy.

End of May 2018

Complete

Not approved or released.

Update to the Risk Management Manual (formerly the Risk Guidelines).

November 2017

Complete

Will be amended to reflect new enterprise risk management framework designed by Aerosafe.

Soft launch of interim updates to Policy and Manual.

June/July 2018

On hold

Delayed pending consultation with Aerosafe and finalisation of risk management framework update.

Consultation working group temporarily replaced with ‘Statistical Risk Tiger Team’.

Mid-June 2018

Complete

 

Develop risk communication and engagement strategy (including external engagement).

Mid-March 2018

In progress

‘Soft launch’ expected June/July following Aerosafe consultation.

Take stock of risk management resources (e.g. templates, tools, and training material).

May 2018

Complete

Work to be done on updating and consolidating risk management training materials, including the update of e-learning modules.

Intranet to be updated.

Improve risk governance (as per interim update to Policy).

June 2018

In progress

 

Create ABS strategic risk register and review procedure.

June/July 2018

In progress

Pending Aerosafe work.

Create enterprise risk register and process for maintenance.

July 2018

Not commenced

Requires the Chief Risk Officer to:

  • establish a system for identifying enterprise risks; and
  • ensure there is an enterprise risk management plan addressing each strategic risk and identified systematic operational risks.

Develop Risk Appetite Statements and Tolerances

March 2018

Not commenced

Pending Aerosafe work.

Major update of Risk Management Policy.

 

On hold

Pending Aerosafe work and finalisation of revised risk management framework.

Major update of Risk Management Manual.

March 2018

Complete

Update may be required following Aerosafe consultation.

Update resources/tools as necessary to reflect updated Risk Management Policy and Manual.

From mid-December 2017

In progress

Template and training review process to be developed when revised risk management framework is confirmed.

a) Update risk management training materials.

b) Integrate circulation of the strategic risk profiles into the communication and education strategy.

July/August 2018

In progress

 

Develop risk management reporting and performance framework.

November 2018

In progress

 

Identify integration opportunities for risk management with risk related activities including: budget prioritisation; ABS Executive Board papers; strategic planning; and project management.

August 2017 onwards

No embedded processes at present.

 

2. Implementation Activities

Roll out the updated Risk Management Policy, Manual, and governance.

June/July 2018

On hold

Waiting for revised risk management framework to be finalised.

Implement communication and education strategy.

December 2018

In progress

Pending Aerosafe work.

3. Optimisation

Evaluation of risk management in the ABS.

Not set

Not commenced

Waiting for foundational and implementation activities to be completed.

Develop an assurance map.

Not set

Not commenced

Waiting for foundational and implementation activities to be completed.

Align risk and performance frameworks.

Not set

Not commenced

Waiting for foundational and implementation activities to be completed.

Complete an assessment of risk management enablers, including ICT.

Not set

Not commenced

Waiting for foundational and implementation activities to be completed.

       

Source: ANAO analysis of ABS Risk Action Plan, 31 May 2018.

Appendix 5 Statistical Business Transformation Program risks and issues — at 8 June 2018

Description

Rating (status)

Target residual rating

Issues

Program schedule

There is a risk, given the breadth and complexity of the Program, that important activities, milestones or dependencies are not identified and scheduled.

High (worsening)

N/A

Program affordability

Actual costs have exceeded allocated budget as a result of the complexity and scale of the Program and the need for high level estimation.

High (worsening)

N/A

Instrument creation

Work undertaken on pioneer collections have identified that the metadata authoring and instrument creation process is not as efficient as possible, and has not been effectively completed in testing.

High (improving)

N/A

Testing

Testing of Program Deliverables is not to schedule.

High (stable)

N/A

Extract, transform and bulk load tools

Tools to transfer metadata, data and rules are not yet available.

High (improving)

N/A

Capability and capacity

Inability to fully staff projects leads to delays in Program delivery and onboarding.

High (worsening)

N/A

Enterprise status codes

SBTP projects have paused pending Enterprise Status Codes (ESCs) and/or are making assumptions about ESCs.

Medium (improving)

N/A

Risks

Product quality

There is the risk that inadequate specification of product and capability quality, including acceptance criteria, will mean that stakeholder quality expectations are not met.

High (stable)

Medium

Complexity of integration

There is the risk that the complexity of the Program integration challenge exceeds the current capability (and to some extent capacity) and funding of the Program to be confident that integration will be successful.

High (worsening)

High

Change management

There is the risk given the size of the business change program planned that not all organisational aspects of the change are fully or correctly designed for and implemented.

High (stable)

Medium

Impact on statistical quality from transitioning to SBTP capabilities and infrastructure - economic and environment statistics

If migrating to Program infrastructure and capabilities is not managed and monitored effectively, then the quality of ABS economic and environment statistics may be impacted.

High (stable)

Medium

Impact on statistical quality from transitioning to SBTP capabilities and infrastructure - population and social statistics (excluding Census)

If migrating to Program infrastructure and capabilities is not managed and monitored effectively, then the quality of ABS population and social statistics may be impacted.

High (stable)

Medium

Future capability expertise

There is a risk that as projects close and staff move to new roles, there could be a lack of capability expertise in business areas.

High (improving)

Medium

Technical and capability debt

If Program activity is prioritised to meet the onboarding schedule, then additional costs may be incurred due to rework needed following adoption of expedient solutions rather than a more costly solution that aligns with enterprise architecture.

Medium (improving)

Medium

Foundational Infrastructure satisfying non-functional requirements

If the Enterprise Data Management Environment, Statistical Workflow Management System, Integration Platform, Metadata Registry and Repository or other components of foundation infrastructure are not able to meet performance, load, resilience or security requirements, this could result in failure for ABS to achieve business benefits or may lead to cost or time issues to remediate.

High (stable)

Low

Foundational Infrastructure Bottleneck

If the Foundation Infrastructure projects do not have sufficient capacity to support timely integration with all other projects, then velocity of the Program will decrease and may result in release and onboarding schedule not being achieved.

High (stable)

Medium

Transitional integration

There is risk that without adequate clarity of and planning for the transitional integration requirements of the new Program capabilities, integration gaps will be left in linkages between legacy systems and new capabilities required whilst delivering the business releases sequentially.

High (stable)

Medium

Cyber security

There is the risk the new technologies delivered by the Program may have vulnerabilities which may be susceptible to a cyber-security attack.

Medium (stable)

Medium

Managing external stakeholders

There is a risk that the expectations of key external ABS stakeholders (e.g. the Minister, ANAO, Finance, the public and media) are not adequately managed with respect to the Program deliverables and expectations, such that stakeholders are sufficiently engaged and satisfied with the Program and ABS.

Medium (stable)

Low

Embedding an Agile culture

There is a risk that facets of ABS culture may inhibit Program progress and working in a more ‘agile’ fashion.

Low (improving)

Low

     

Source: ABS, Statistical Business Transformation Program, Update on Program Risks and Issues, 8 June 2018.

Footnotes

1 The budget was reduced from the original proposal following removal of a number of activities included in the original business case (such as training, stakeholder engagement, program management, reengineering and some system capability options). The ABS was initially expected to contribute $33.5 million from within ABS annual appropriations.

2 The Census is managed through a separate, third party arrangement using high speed, high capacity data warehousing. The ABS is considering options to adopt some components of the Program for use in delivering the 2021 Census.

3 ABS Risk Management Framework Part B – The Risk Guidelines, 2015, p. 3.

4 ibid.

5 eCensus is the name given to the ABS’ system for collecting census information from the Australian public electronically through an online interface on the ABS website.

6 MacGibbon, Alastair & Australia. Office of the Cyber Security Special Adviser 2016, Review of the events surrounding the 2016 eCensus : improving institutional cyber security culture and practices across the Australian government, [Canberra] Department of the Prime Minister and Cabinet.

7 David W Kalisch, ‘Census 2016: Lessons Learned – Improving Cyber Security Culture and Practice’, speech to the Institute of Public Administration (ACT), 13 December 2016.

8 Comcover, part of the Department of Finance, is the Australian Government’s self-managed insurance fund and provides risk management services to Australian Government entities. The survey is mandatory for Comcover fund members, including the ABS.

9 The contract was for the period 1 February 2017 to 30 June 2017.

10 AusTender provides centralised publication of Australian Government business opportunities, annual procurement plans and contracts awarded.

11 The invoices were for work undertaken in support of the Australian Marriage Law Postal Survey ($40,260) and risk management support ($82,390). Aeorsafe advised that 20.5 hours of work was yet to be invoiced.

12 One-on-one sessions were conducted with the seven ABS Executive Board members, and the risk management workshop was held with all ABS senior executives.

13 No treatment plan was created for the Organisational capability risk.

14 The ANAO found that the ABS Risk Management Framework partly met the Commonwealth Risk Management Policy requirement to provide an overview of the entity’s approach to embedding risk management into its existing business processes (see Appendix 3).

15 The Board has received papers relating to the enterprise risk management framework update and the proposed 2018 ABS strategic risks, but no reports on the status of enterprise-wide or strategic risks.

16 On 4 June 2018, the ABS Executive Board agreed that strategic risks should be reviewed by the Board on a regular basis to enable detailed review of high risk areas.

17 The external members are: David Whiteing (Chief Information Officer, Commonwealth Bank), David Borthwick (Senior Consultant) and Peter Harper (Senior Consultant and a previous Deputy Australian Statistician).

18 Delivery Board members include Program executives, representatives from business areas within the ABS and a representative of the Chief Financial Officer.

19 The report does not include information on whether the treatments applied to the risk or issue have been effective in reducing the rating.

20 Monthly Delivery Board meetings were cancelled in February, May, June and July 2018.

21 Order for Services, Statistical Business Transformation Program Independent Assurance Partner, 11 November 2015.

22 The assurance map is developed by KPMG and approved by the Program Executive Board.

23 The Gateway Review process involves a series of reviews conducted at critical points across the Program implementation lifecycle. The purpose is to provide the Senior Responsible Officer with assurance and advice to improve delivery and implementation, as well as early identification of areas requiring corrective action.

24 Green is defined as: no major outstanding issues that at this stage appear to threaten delivery significantly.

25 Amber is defined as: there are issues in this key focus area that require timely management attention.

26 The ABS Executive Board does not receive regular Program reports.

27 Department of Finance, Gateway Review Report Mid Stage Program Review, December 2017, p. 9.

28 In May 2018 the Department of Finance issued A guide for non-corporate Commonwealth entities on the role of audit committees, Resource Management Guide No. 202. It provides guidance for accountable authorities to establish a well-functioning audit committee and helps to frame the review of the committee. One example, which relates to an audit committee’s oversight of programs, states that the audit committee could: satisfy itself that an appropriate approach has been followed in managing the entity’s key risks—including those associated with individual projects and program implementation and activities.

29 ABS, Risk Management Framework Part A – The Risk Policy, October 2015, p. 5.

30 Statistical Business Transformation Program Executive Program Board Minutes, 2 December 2016.

31 The terminology used by the Program differs from that in the ABS risk management framework (see Figure 2.1 for an overview of the ABS risk management process).

32 PRINCE2 (an acronym for PRojects IN Controlled Environments) is a commercially available process-based project management methodology.

33 Managing Successful Programmes provides a structured framework for planning, controlling and implementing major change programs

34 Agile methodology uses an iterative, incremental approach to project management and software design and delivery.

35 The need to monitor the effectiveness of treatments was raised by the Program Delivery Board in September 2017.

36 Since January 2016, the risk name, identification number and description of the risk relating Program affordability have been changed multiple times.

37 The reasons for escalation included the phasing of funds in future years due to schedule delays, delays to expected benefits and unfunded system maintenance.

38 Onboarding is the point at which the ABS commences using the capabilities delivered by the Program.

39 Paper, Changed approach to Program Delivery, presented to the Statistical Business Transformation Executive Program Board, 27 June 2018.

40 The revised number of collections to be onboarded was not specified.

41 Relevant Ministers over the period are: Minister for Small Business; Minister for Small and Family Business, the Workplace and Deregulation; and the Assistant Minister to the Treasurer.

42 While the focus of the document is on ABS-wide transformation, the Program is the largest and most complex element of the transformation.

43 Stakeholders include Australian, State and Territory government entities, tertiary and research organisations, statistical forums and the Australian public.