The audit objective was to assess the effectiveness of the Australian Taxation Office's processes for estimating and monitoring the costs, savings and benefits associated with the Reinventing the ATO program.

Summary and recommendations

Background

1. Reinventing the ATO is a broad transformational change program focused on achieving the Australian Taxation Office’s (ATO) vision of being a contemporary service oriented organisation. The program was initiated partly in response to the Australian Public Service Commission’s capability review in 2013, which outlined the challenge for the ATO to transform its existing processes, systems, culture and workforce to be more agile, responsive, efficient and effective. At a high level, implementation of the program was expected to better position the ATO to be more contemporary, innovate with technology and meet taxpayer expectations. While productivity benefits and operational savings are expected from the program, they were not a key driver for its implementation.

2. The Reinventing the ATO program formally commenced in 2015 with the release of a ‘blueprint’ that outlined experience shifts for key stakeholders, such as staff and taxpayers, as a result of implementation of the program. The program consists of behavioural and cultural elements, locally managed change and continuous improvement initiatives, as well as six strategic programs that oversee 100 projects. These projects are required to apply the ATO’s corporate project management framework, which was revised in July 2016 to provide a greater focus on the value proposition of projects, including costs and savings.

3. The ATO is not managing the entire Reinventing the ATO program using a formal program management methodology, however, governance arrangements have been put in place to support the implementation of the Reinventing the ATO projects, including a program office and strategic program governance bodies.

Audit objective and criteria

4. The audit objective was to assess the effectiveness of the ATO’s processes for estimating and monitoring the costs, savings and benefits associated with the Reinventing the ATO program. The audit criteria were that:

  • sound processes were in place for estimating the potential costs, savings and benefits associated with the Reinventing the ATO program; and
  • actual costs, savings and benefits associated with the Reinventing the ATO program are measured and monitored.

Conclusion

5. The ATO has sound systems and guidance for estimating and monitoring the costs, savings and benefits associated with Reinventing the ATO projects but the effectiveness of these processes has been compromised by low levels of conformance. As a result, the costs, savings and benefits from these projects cannot be calculated. The ATO never intended to calculate these measures for the entire Reinventing the ATO program as it included many locally managed and cultural change initiatives. The ATO needs to ensure greater conformance to processes for estimating and monitoring project costs, savings and benefits, to provide transparency about the net benefits of programs and support decisions about the commencement, continuation, resourcing and direction of projects.

6. The ATO measures benefits from projects through a Connected Benefits Management System that links project and program outcomes to corporate benefits categories and ATO corporate impact areas. There was a general improvement across the ATO’s corporate benefits categories from 2013–14 to 2015–16, particularly relating to the corporate impact areas of willing participation and revenue. Further, the ATO advised of a number of positive business changes, including improved employee engagement, as a result of the Reinventing the ATO program. However, there would have been a higher level of assurance of the benefits from the Reinventing the ATO program if the ATO had identified performance indicators to measure the impact of the program or established a baseline to systematically measure anticipated benefits.

7. The ATO has sound project management processes in place to support the estimation of costs associated with Reinventing the ATO projects but has not always had sound processes for estimating potential savings from the projects. Despite the availability of a cost estimation tool and a requirement to estimate costs in key pre-approval documentation, costs were not consistently recorded in business cases and project plans. Potential savings from the projects were rarely included in this documentation. Detailed processes have been in place to support the estimation of benefits associated with the Reinventing the ATO program, although these processes have often not been applied to projects.

8. Costs and savings associated with the Reinventing the ATO program and most of its projects have not been tracked. However, the ATO recently introduced internal financial benefits reporting that provides a framework for measuring and monitoring savings from Reinventing the ATO projects going forward. The ATO’s benefits measurement approach has been strengthened since the commencement of the Reinventing the ATO program to enhance the profile of benefits and their alignment with broader ATO corporate impact areas when considering the value proposition of potential projects. Nonetheless, a lack of completeness in monitoring and reporting on the achievement of Reinventing the ATO projects, and the program more broadly, has limited transparency about the scale and nature of benefits achieved.

Supporting findings

Estimating costs, savings and benefits associated with Reinventing the ATO projects

9. The ATO has sound processes to support project managers to estimate the costs of Reinventing the ATO projects, including providing a cost estimation tool and having estimated costs assured by the ATO’s Finance team. However, there was not widespread adherence to these processes. Of the 100 Reinventing the ATO projects, 62 had applied the ATO costing tool, of which 34 had costs assured by ATO Finance. Cost information was also infrequently recorded in project documentation such as business cases and project plans—25 Reinventing the ATO projects had final project plans that included estimated project costs.

10. Savings estimates were infrequently included in Reinventing the ATO project pre-approval documentation as required by the ATO’s project management procedures. In early 2017, the ATO implemented a verification process to confirm expected savings from projects, which should improve the accuracy of savings estimates and frequency of inclusion in project management documentation.

11. While the ATO’s guidance has consistently required that project outcomes and benefits are identified and recorded in key project management documentation, conformance with these requirements by Reinventing the ATO projects has been low. Only 56 of the 100 Reinventing the ATO projects outlined expected benefits in project pre-approval documentation, including non-financial benefits and productivity improvements. Under the revised project management approach, the ATO’s benefits management processes have been strengthened to require that project outcomes align with broader ATO organisational priorities.

Measuring and monitoring costs, savings and benefits associated with Reinventing the ATO projects

12. The ATO is unable to measure and monitor the total costs of implementing Reinventing the ATO projects because of low levels of conformance with requirements to track costs—only eight projects included actual costs in status reports and 13 projects included actual costs in closure reports. The magnitude of costs of Reinventing the ATO projects warrants greater attention to measurement and monitoring—as for the 67 Reinventing the ATO projects where data was available, costs were estimated at $300 million from 2013–14 to 2018–19.

13. The ATO has not been tracking the monetary savings associated with the Reinventing the ATO program. However, in April 2017 it implemented internal reporting on financial benefits across the office, including Reinventing the ATO projects. As the reporting process involves the verification of estimated and realised financial benefits, this should better position the ATO to consider realised savings when making operational decisions, such as reallocating resources due to productivity gains.

14. The Reinventing the ATO program has provided a number of benefits, as indicated by the large number of outcomes listed as achieved for individual projects. However, there is a lack of clarity about the results of Reinventing the ATO projects as a consequence of the:

  • lack of conformance with the ATO’s processes for monitoring and reporting on the achievement of project outcomes—of 57 closed, cancelled or transferred to business-as-usual, 21 had closure reports that indicated whether project outcomes had been achieved; and
  • implementation of the Connected Benefits Management System after the commencement of the program and many projects, and the ATO not accurately identifying the contribution made by Reinventing the ATO projects to corporate priorities.

15. The ATO has identified and discontinued projects as a result of concerns relating to their relevance and progress. Nevertheless, there is scope for the ATO to improve the: frequency of program status reporting to governance bodies; quality of information provided in relation to projects’ status; and use of governance gates.

Recommendations

Recommendation no.1

Paragraph 4.10

The Australian Taxation Office mandates and monitors the recording and reporting of actual project costs for all corporate projects.

Australian Taxation Office response: Partially agreed.

Recommendation no.2

Paragraph 4.50

The Australian Taxation Office enforces the mandating of status reports and governance gate assurance activities to support assessment of the ongoing viability of projects including delivery of expected benefits.

Australian Taxation Office response: Partially agreed.

Summary of entity response

16. The summary response to the report from the ATO is provided below, with the covering letter included in Appendix 1.

The ATO acknowledges the ANAO review and considers the report supportive of our overall approach to estimating and monitoring the costs, savings and benefits associated with projects.

The review recognises the overall intent of the Reinventing the ATO program was to transform our internal culture, providing a stronger connection to the community and an openness and willingness to change in order to maximise willing participation in the tax and superannuation systems. Although some elements were delivered through formally recognised programs of work and projects, a large proportion of the Reinvention Program was driven through localised action in teams, branches and business lines.

Our approaches to strategic planning, investment management, project delivery and change management have evolved and matured since the establishment and of the Reinventing the ATO program in 2015. The review acknowledged that the ATO has developed sound systems and guidance to support project management including estimating and monitoring costs, savings and benefits.

The review also identified that there is scope for the ATO to continue to increase consistency of application as part of maturing our frameworks. The ATO recognises this opportunity and is committed to continuous improvement in relation to project management, including recognising the refinement of frameworks and practices as an area of focus in our corporate plan for 2017–18.

Key learnings for all Australian Government entities

Below is a summary of key learnings identified in this audit report that may be considered by other government entities when managing the costs, savings and benefits associated with programs and projects.

Managing costs, savings and benefits

Measuring program and project performance

1. Background

Introduction

1.1 Responsibilities of the Australian Taxation Office (ATO) include:

  • collecting taxation revenue, such as personal income and company taxes;
  • administering the goods and services tax on behalf of the Australian states and territories;
  • administering a range of programs that result in transfers and benefits back to the community;
  • administering major parts of Australia’s superannuation system; and
  • administering the Australian Business Register.1

1.2 These responsibilities require interaction with a wide range of stakeholders including individual taxpayers (10.9 million), small businesses (3.8 million) and super funds (597 000). Improving stakeholders’ experiences in dealing with the ATO was one of the key drivers for introducing the Reinventing the ATO program in 2015.

Reinventing the ATO program

1.3 The Reinventing the ATO program is a broad transformational change program focused on achieving the ATO’s vision of being a contemporary service oriented organisation.2 It is aimed at improving taxpayer and staff experiences as well as transforming the ATO’s culture to be more service oriented. At a high level, implementation of the program was expected to better position the ATO to: be more contemporary; innovate with technology; and meet taxpayer expectations. While productivity benefits and operational savings were expected from the Reinventing the ATO program3, they were not a key driver for its implementation.

1.4 The program was initiated partly in response to the Australian Public Service Commission’s capability review in 2013, which outlined the challenge for the ATO to transform its existing processes, systems, culture and workforce to be more agile, responsive, efficient and effective. Initiation of the program was also due in part to an internal review of the ATO’s cultural traits conducted in early 2014.

1.5 The program applies to all aspects of the ATO’s operations, including infrastructure, tools, services and capability. It is expected to create a different internal culture resulting in a stronger connection to the community and an openness and willingness to change in order to maximise the community’s willing participation in the tax and super systems. The program is also expected to generate new products and services aimed at improving the staff and taxpayer experience.4

1.6 The precursor to the program was the ATO’s 2020 Vision and Mission, developed in July 2013. The Reinventing the ATO program formally commenced in 2015 with the release of the Reinventing the ATO Blueprint. The ‘blueprint’ was co-designed with taxpayers and staff and describes their expected experience shifts as a result of the implementation of the program.

1.7 Although not a ‘program’ in the technical sense, Reinventing the ATO encompassed both formal corporate projects, a number of behavioural and cultural elements, and more locally managed change and continuous improvement initiatives driven through localised action in teams, branches and business lines.

1.8 The Reinventing the ATO program included the establishment of six strategic programs. Existing and new projects have been assigned to one of the six strategic programs, and there are 100 Reinventing the ATO projects (see Appendix 2).5 These projects are required to be managed in accordance with the ATO’s project governance methodology, which requires clear articulation and approval of deliverables, milestones, timeframes, cost and outcomes. The project governance methodology is not applied to the Reinventing the ATO program as a whole. As such, aggregated program costs, savings and performance indicators have not been established.

1.9 The ATO Executive is ultimately responsible for implementation of the Reinventing the ATO program as it sets the strategic direction and priorities for change in the ATO. As illustrated in Figure 1.1, the governance arrangements for the program also included:

  • an Integration Forum—responsible for ensuring activities align with the Reinventing the ATO program and that an appropriate level of resources and expertise are allocated to the six strategic programs. The Integration Forum is responsible for providing recommendations, guidance and advice to the ATO Executive in relation to the program as well as providing strategic advice and guidance to the six strategic program governance bodies;
  • a Resource Forum—responsible for making recommendations to the ATO Executive on strategic investments, providing strategic oversight of the use of ATO resources and facilitating the allocation and assignment of resources to ATO business outcomes;
  • a Reinventing the ATO program office—established to support the delivery of the Reinventing the ATO Blueprint through developing tailored governance arrangements and providing assurance activities to support the strategic programs;
  • a Value Management Office—established in 2016, with a central function to provide consultancy services and support to project and program teams for value management, including benefits planning, monitoring and realisation; and
  • strategic program governance bodies—responsible for directing and managing implementation of the strategic programs.

Figure 1.1: Reinventing the ATO governance arrangements

Source: ANAO analysis of ATO information.

1.10 The ATO advised that to effectively enable transformation, it was important to strike the right balance with governance, ensuring it was commensurate to the scale and scope of each element and not imposing unnecessary red tape for localised change. Evolving its approaches to strategic planning, investment management, project delivery and change management were also part of the overall cultural change being sought.

Internal audits in relation to the Reinventing the ATO program

1.11 The ATO has undertaken a number of internal audits in relation to the Reinventing the ATO program, as shown in Table 1.1. Notwithstanding sound design of the new benefits management framework6, the audits found scope to improve major elements of the program’s administration, including monitoring of projects’ performance and costs.

Table 1.1: ATO internal audits in relation to the Reinventing the ATO program

Name of review

Overall rating

High-level findings

Reinventing the ATO (October 2016)a

Room for improvement

  • Programs identified top level risks that could impact on their strategic intent. However, there is limited evidence of active risk management to identify new risks, re-assess identified risks and provide updates on actions and additional mitigation strategies.
  • There were clear road maps outlining program intent, deliverables and timing. However, some program governance bodies did not have sound processes in place to track project progress, schedule and performance against measures.
  • Project costs and responsibility for monitoring them are unclear although information technology (IT) related costs were tracked and could be made available when required.
  • There is an absence of systematic program and project assurance activities and quality management is not being used as an enabling tool to manage benefits realisation.

Benefits Management Framework (October 2016)

Satisfactory

  • The design of the ATO’s Connected Benefits Management System, and the controls in the new project management method, should assist to improve the maturity of the ATO’s benefits management and realisation.
  • Effort is required to embed the new benefits management practices; particularly in relation to governance and harvesting of benefits.

Rollout of the Culture and Leadership Strategies (August 2016)b

Room for improvement

  • Project-based practices have been used to plan and prioritise activities.
  • Culture priorities and key initiatives are identified as part of annual planning but there is an absence of monitoring and reporting on progress.
  • A complete set of success measures has not been identified. Appropriate measures will be necessary for measuring and reporting the success of strategies from 2016–17.

Corporate Function Review Outcomes (June 2016)c

Satisfactory

  • There is evidence of benefits realisation for the Smarter Data program.
  • While there are outstanding deliverables for the Smarter Data program, including finalising a benefits management plan, there are adequate processes in place to support completion of these deliverables.
     

Note a: These findings were made based on a sample of four of the strategic programs.

Note b: Aspects of the Optimise Workforce Capability and Culture program were reviewed as part of this audit. Findings were not limited to the Optimise Workforce Capability and Culture program, and therefore, the high-level findings are not specific to the program.

Note c: Aspects of the Smarter Data program were reviewed as part of this audit. The high-level findings outlined in the table are only those relevant to the Smarter Data program.

Source: ANAO analysis of ATO information.

1.12 The ATO has outlined a number of positive results from the Reinventing the ATO program, which are discussed in Chapter 2.

Audit approach

1.13 The audit objective was to assess the effectiveness of the ATO’s processes for estimating and monitoring the costs, savings and benefits associated with the Reinventing the ATO program.

1.14 To form a conclusion against this objective, the Australian National Audit Office (ANAO) adopted two high-level criteria:

  • sound processes were in place for estimating the potential costs, savings and benefits associated with the Reinventing the ATO program; and
  • actual costs, savings and benefits associated with the Reinventing the ATO program are measured and monitored.

1.15 The audit focused on the costs, savings and benefits of the 100 Reinventing the ATO projects that contributed to the Reinventing the ATO program, but also examined the benefits delivered by the program as a whole. The audit did not examine the:

  • project governance or management arrangements applied to individual projects, the six strategic programs or the Reinventing the ATO program beyond costs, saving and benefits—for example, risk management, communication or governance arrangements;
  • selection, prioritisation and delivery of individual projects; or
  • locally managed change and continuous improvement initiatives, including those affecting behaviours and cultures in the ATO.

Audit methodology

1.16 The audit methodology included reviewing: relevant project and benefits management templates and guidelines; Reinventing the ATO project documentation and performance reporting; and contributions received through the ANAO’s website.7 The methodology also included interviewing key ATO staff.

1.17 The audit was conducted in accordance with the ANAO’s auditing standards at a cost to the ANAO of approximately $270 000.

1.18 The team members for this audit were Kylie Jackson, Renee Hall and Andrew Morris.

2. Benefits of the Reinventing the ATO program

Areas examined

The ANAO examined the benefits delivered by the Reinventing the ATO program.

Conclusion

The ATO measures benefits from projects through a Connected Benefits Management System that links project and program outcomes to corporate benefits categories and ATO corporate impact areas. There was a general improvement across the ATO’s corporate benefits categories from 2013–14 to 2015–16, particularly relating to the corporate impact areas of willing participation and revenue. Further, the ATO advised of a number of positive business changes, including improved employee engagement, as a result of the Reinventing the ATO program. However, there would have been a higher level of assurance of the benefits from the Reinventing the ATO program if the ATO had identified performance indicators to measure the impact of the program or established a baseline to systematically measure anticipated benefits.

Introduction

2.1 As discussed in Chapter 1, Reinventing the ATO was a broad transformational program focused on improving the client and staff experience. Benefits expected from the program were articulated at a high-level. For example:

  • implementation of the program was expected to better position the ATO to be more contemporary, innovate with technology and meet taxpayer expectations; and
  • the ‘blueprint’ outlined expected experience shifts for key stakeholders as a result of implementing the Reinventing the ATO program.

The blueprint also stated that ‘ultimately, the true measure of success of our reinvention will be client satisfaction and participation in the tax and super systems’.8

2.2 Some elements of the Reinventing the ATO program were delivered through formally recognised programs of work and projects, with other elements of the program delivered through localised action in teams, branches and business lines.

2.3 The Reinventing the ATO program is supported by a suite of articulated outcomes that include both financial and non-financial benefits arising from Reinventing the ATO projects. While outcomes from these projects have been monitored, the success of the Reinventing the ATO program needs to consider other elements including the cultural and experience changes from less formal and continuous improvement initiatives. However, the ATO has not clearly captured the benefits from these other elements, as it intentionally has not managed or reported on the entire Reinventing the ATO program technically as a ‘program’. In this light, the ATO has not identified performance indicators to measure the benefits delivered by the program or established baselines to measure improvements in key benefit areas.

2.4 To assess the extent of benefits from the Reinventing the ATO program, the ANAO examined the benefits reported at the project level (through the Connected Benefits Management System and ATO benefits performance measures), and more broadly for the program by examining recent ATO annual reports and advice provided by the ATO.

High-level benefits from projects

Connected Benefits Management System

2.5 On 1 July 2016, the ATO introduced the Connected Benefits Management System with the revised corporate project management method. The Connected Benefits Management System links project and program outcomes to defined corporate benefits categories and ATO corporate impact areas. It is intended to measure enterprise-wide investment, not only the impact of the Reinventing the ATO program. Figure 2.1 illustrates the ATO’s benefits categories and their relationship to broader ATO impact areas.

Figure 2.1: Benefits categories in the Connected Benefits Management System

Source: ATO.

2.6 In 2015–16 the ATO had four corporate impact areas: integrity; willing participation; revenue; and productivity, as illustrated by the inner circle of Figure 2.1. The middle circle illustrates how the ATO intends to achieve its corporate impact areas and the outer circle defines the associated benefits categories.9

2.7 There are 21 benefits categories in the ATO’s Connected Benefits Management System that can be considered in the assessment of the impact of programs of work on ATO corporate priorities. The ATO has developed benefits measurement methods for 18 of the 21 categories, which leverage off existing tools and processes, such as surveys, where possible.10 As at August 2017, measurement methods had not been defined for the cost per transaction, total revenue effects and tax assured benefits categories.

2.8 The ATO is tracking and reporting on the contribution from individual projects by monitoring the project outcomes achieved towards these 21 benefits categories. Monitoring is undertaken at an enterprise-wide level, including Reinventing the ATO projects, and the exclusive contribution of the Reinventing the ATO program towards these benefits categories is not monitored by the ATO. Figure 2.2 outlines the main outcomes contributing to ATO benefits performance measures.

Figure 2.2: Outcomes contributing to ATO benefits performance measures, June 2017

Note: The corporate impact of ‘perception that ATO listens and responds to feedback’ is a measure from the ATO 2015–19 corporate plan and not included in the Connected Benefits Management System wheel (refer Figure 2.1). The wheel is aligned to measures from the 2016–20 corporate plan.

Source: ATO portfolio benefits reporting.

2.9 As illustrated in Figure 2.2, the benefits categories that have the largest number of contributing project outcomes are11:

  • perception of ease to access services and information (56 project outcomes);
  • community satisfaction with ATO performance (44 project outcomes);
  • level of employee engagement (39 project outcomes); and
  • cost per transaction (26 project outcomes).

Benefits from the program more broadly

2.10 The ATO reports publicly on benefits categories in its performance statements as part of its corporate planning and annual reporting. Since 2013–14, the ATO has been increasing the number of measures reported in its annual reports—from 13 in 2013–14 to 15 in 2014–15 and 18 in 2015–16.

2.11 Table 2.1 outlines the ATO’s performance against the specific benefits categories, as reported by the ATO in its annual reports from 2013–14 to 2015–16. The ANAO assessed12 that performance of seven benefits categories was positive13; four categories had negative performance results; and another four remained stable.14 As noted in paragraph 2.8, these results cannot be wholly attributed to the Reinventing the ATO program and the ATO advised that the program’s exclusive contribution towards these categories cannot be reliably measured.

Table 2.1: Performance of benefits categories

Benefits categories

Reporting period

Performance

Integrity

Community satisfaction with ATO performance

2014–16

Negative

Perceptions of fairness in disputes

2014–16

Positive

Level of employee engagement

2014–16

Positive

Willing participation

Perception of ease to access services and information

2013–16

Positive

Percentage of inbound transactions received digitally

2015–16

Baseline set

Adjusted average cost of managing tax affairs

2013–16

Positive

Percentage of companies and individuals registered in the system

2013–16

Individuals – stable

Companies – new baseline seta

Percentage of activity statements and income tax returns lodged on time

2013–16

Stableb

Percentage of liabilities paid on time by value

2013–16

Positive

Adjusted employer superannuation contributions as a percentage of adjusted salary and wages

2013–16

Positive

Tax gap as a percentage of revenue

2013–16

Negativec

Revenue

Tax assured

Not reported

Under development

Cash collected from direct compliance activities

2013–16

Stable

Total revenue effects

Not reported

Under development

Ratio of collectible debt to net tax collection

2013–16

Positive

Proportion of revenue collected compared with forecast

2013–16

Negative

Productivity

Cost to collect $100

2013–16

Stable

ATO manages its operating budget to balance

2013–16

Negative

Tax administration expenditure as percentage of Gross Domestic Product (GDP)

2015–16

Baseline set

Non-tax expenditure as percentage of total expenditure

2015–16

Baseline set

Cost per transaction

Not reported

Under development

     

Note a: This result was considered by the ANAO as stable overall. Results for companies were not comparable to the prior year as a change in methodology occurred in 2015–16.

Note b: Overall result is stable. Percentage of activity statements lodged on time is declining and percentage of tax returns lodged on time is increasing.

Note c: Against this measure, the ATO only reports on the goods and services tax gap in its performance statement (which was negative as reflected in this table). However, in its annual report the ATO also includes reporting on the proportion of tax gap for the luxury car tax (declining), wine equalisation tax (not reported in 2015–16), Pay-As-You-Go withholding (not reported in 2015–16) and fuel tax credits (improving). The methodology for the measure on the tax gap as a percentage of revenue is changing and the ATO intends to report a new baseline in 2016–17.

Source: ANAO analysis of ATO information.

2.12 In addition to the improvements in the seven benefits categories outlined in Table 2.1, the ATO advised that there have been a number of positive changes in the ATO’s business results and performance stemming from, and as a direct outcome of, the Reinventing the ATO program. Examples of positive changes included improved employee engagement, changing lodgment behaviour by taxpayers and dispute resolution approaches. These positive changes occurred in the context of a decrease in the ATO’s workforce of approximately 3500 people due to a redundancy program undertaken in 2014–15.

3. Estimating costs, savings and benefits associated with Reinventing the ATO projects

Areas examined

The ANAO examined whether the ATO has sound processes in place for estimating the costs, savings and benefits associated with Reinventing the ATO projects.

Conclusion

The ATO has sound project management processes in place to support the estimation of costs associated with Reinventing the ATO projects but has not always had sound processes for estimating potential savings from the projects. Despite the availability of a cost estimation tool and a requirement to estimate costs in key pre-approval documentation, costs were not consistently recorded in business cases and project plans. Potential savings from the projects were rarely included in this documentation. Detailed processes have been in place to support the estimation of benefits associated with the Reinventing the ATO program, although these processes have often not been applied to projects.

Areas for improvement

The ANAO made four suggestions aimed at: including estimated project costs and savings in pre-approval documentation for any future Reinventing the ATO projects (paragraph 3.9 and paragraph 3.15); outlining expected outcomes and benefits in pre-approval documentation for future Reinventing the ATO projects (paragraph 3.23); and improving the transparency of outcomes mapping to better identify the expected contribution of existing and planned projects to program outcomes (paragraph 3.27).

Was there a sound basis for estimating the costs associated with Reinventing the ATO projects?

The ATO has sound processes to support project managers to estimate the costs of Reinventing the ATO projects, including providing a cost estimation tool and having estimated costs assured by the ATO’s Finance team. However, there was not widespread adherence to these processes. Of the 100 Reinventing the ATO projects, 62 had applied the ATO costing tool, of which 34 had costs assured by ATO Finance. Cost information was also infrequently recorded in project documentation such as business cases and project plans—25 Reinventing the ATO projects had final project plans that included estimated project costs.

3.1 While the ATO is not applying a program management methodology to the Reinventing the ATO program as a whole, all projects within the scope of the Reinventing the ATO program are required to follow the corporate project management methodology. Most of the projects categorised as Reinventing the ATO projects were already in place prior to the initiation of the Reinventing the ATO program.15 In January 2015, the six strategic Reinventing the ATO programs were established, and existing projects were subsequently allocated to these programs. This process identified integration opportunities with some projects being stopped or amalgamated.

3.2 The ATO had an existing corporate project management approach in place to support the implementation of these projects and any new Reinventing the ATO projects. In July 2016, the ATO revised its project management framework to better support the new portfolio management practices that were introduced as part of the Reinventing the ATO program. The introduction of the framework was also designed to address issues from a project management certificate of assurance exercise undertaken in 2013 as well as an external assessment of the ATO’s project management maturity in 2015.16

3.3 The revised project management approach was expected to provide a greater focus on the value proposition of projects as well as tailor governance and project management methods to individual projects according to project complexity and risk. This is different from the previous project management approach where projects were classified as one of three tiers17 and then were expected to be managed in line with the ATO’s corporate project management approach relevant to that tier.

3.4 Both approaches included a suite of project management documentation as well as review points throughout projects’ lifecycles. Under the previous project management approach, governance gates were required for Tier 2 projects and were intended to be the points where project sponsors18 determined whether projects should be discontinued or continued with or without modification.19 The governance gates remained as progress decision points under the revised project management approach, however, rather than project sponsors being responsible for these decisions, governance bodies became responsible.20 Figure 3.1 illustrates the purpose of the different governance gates under the revised project management approach, as well as the required project documentation.

Figure 3.1: Governance gates and required documentation under the ATO’s revised project management approach

Source: ANAO analysis of ATO information.

3.5 To support project managers to complete the required project documentation, the ATO provided templates on its intranet. Further, to assist with estimating the costs associated with projects, the ATO also provided a project resource estimation tool on its intranet. The tool estimates the cost associated with project staff and captures expected supplier costs, such as contractors, software, legal, travel and accommodation. The Enterprise Portfolio Office21 was then responsible for having completed cost estimates assured by ATO Finance. Of the 100 Reinventing the ATO projects, 62 had a completed costing tool; of which 34 had been assured by ATO Finance.

3.6 The ATO’s project management frameworks identified the documentation that should have included estimated cost information, as illustrated in Table 3.1. Actual cost information requirements are outlined in Chapter 4.

Table 3.1: Requirement to outline costs in project management documentation

 

Previous framework

Current framework

 

Tier 2

Tier 3

 

Concept brief/project proposala

Indicative overall cost

Indicative overall cost

No cost estimate required

Business/value caseb

Estimated total project cost

N/A

Estimated total project cost

Project plan/outlinec

Detailed project cost to be included

Estimated total project cost

Estimated total project cost

Status report

Estimated total project costs

Estimated total project costs

Cost (year-to-date, full year and whole project)

Closure report

Total approved project cost

Total approved project cost

Cost variances

       

Note a: Under the revised project management approach, rather than a concept brief, a project proposal is required.

Note b: Under the revised project management approach, rather than a business case, a value case is required.

Note c: Both project management approaches require project plans. However, under the previous project management approach, Tier 2 projects were required to complete project plans while Tier 3 projects were required to complete project outlines.

Source: ANAO analysis of ATO information.

3.7 The ATO advised that there were more than 200 projects in place prior to the initiation of the Reinventing the ATO program, however the re-scoping and amalgamation of projects, as well as the program’s dynamic nature made the number of projects difficult to quantify. At the time of the audit the ATO provided 100 projects in the Reinventing the ATO program for review.22 Sixty-seven projects were initiated under the previous tier-based project management; six were initiated under the revised project management approach; 12 have applied another project management approach (for example, tailoring templates23); and for 19 projects, the methodology applied is unclear.

3.8 Indicative costs were not routinely recorded in project pre-approval documentation as illustrated in Table 3.2. Twenty-five of the 100 Reinventing the ATO projects had final project plans that included estimated project costs.

Table 3.2: Completion of required documentation for Reinventing the ATO projects

 

No. of projects that were not required to complete document

No. of projects without document

No. of projects with final or draft document with no or incomplete estimated cost information

No. of projects with final document with estimated cost information

Total

Concept brief/project proposal

0

59

39a

6b

104

Business/value case

17c

41

25

17

100

Project plan/outline

3c

39

33

25

100

Status report

0

49

41d

10e

100

Closure report

44f

15

29g

12

100

           

Note a: Thirteen projects had draft or incomplete scoping documents. Three of these projects were initiated under the revised project management approach and were not required to include cost information. Thirteen of these projects had prepared a concept brief or project proposal, the remaining thirteen had another form of scoping document such as a design document, green paper, policy project briefing, scoping document or workshop.

Note b: Four of these were concept briefs; one was a project proposal; and one was a scoping document.

Note c: Three projects were cancelled or halted prior to reaching the value case preparation stage. These projects were: Improving the business experience, Sub accounting period application and Accounting program of work Phase 2. Therefore, these projects were not required to deliver a business/value case or project plan/outline.

Note d: Twenty-two of these projects had traffic light reporting in relation to budget position. Five had traffic light reporting for budget position for some months and no mention of budget position in other months. Two had traffic light reporting and reported the estimated project cost in less than half of its status reports. A further 12 projects had status reporting that did not reference budget position.

Note e: One of these projects reported its budget for more than half of its status reports but not for all.

Note f: Thirty-six of those projects were in progress at the time of the audit and therefore, did not require closure reports; and seven projects were halted but not cancelled at the time of the audit. Another project was advised by the project office that it did not require a closure report.

Note g: Ten of these 29 projects indicated that a budget was not reported as the project had been undertaken from business-as-usual resources or was now being undertaken as part of business-as-usual resources and three projects indicated that no budget had ever been prepared.

Source: ANAO analysis of ATO information.

3.9 Project costing guidance indicated that projects were to be approved based on their value proposition and/or alignment with ATO priorities. The guidance also indicated that project sponsors would review the project at least monthly to assess its ongoing viability. In particular, sponsors might consider the actual costs to date as well as the projected versus budgeted costs of the project. The absence of cost information in key project documentation did not support value for money assessments during the project scoping, approval or delivery phases. Expected cost estimates should be included in the pre-approval documentation for any future Reinventing the ATO projects.

3.10 The ATO is not managing the Reinventing the ATO program by applying a program management methodology and consequently, did not quantify the cost versus the benefit of the Reinventing the ATO program at an aggregate level. While costs and benefits were expected to have been quantified at a project level, costs were not consistently captured and there is no total Reinventing the ATO program cost estimate.

Was there a sound basis for estimating the savings associated with Reinventing the ATO projects?

Savings estimates were infrequently included in Reinventing the ATO project pre-approval documentation as required by the ATO’s project management procedures. In early 2017, the ATO implemented a verification process to confirm expected savings from projects, which should improve the accuracy of savings estimates and frequency of inclusion in project management documentation.

3.11 The ATO advised that it does not have a savings methodology, however, in early 2017 it implemented a verification process where ATO Finance confirm the value of expected savings indicated in project pre-approval documentation (refer paragraph 4.16).

3.12 Savings estimates associated with individual Reinventing the ATO projects should have been outlined in business cases and project management plans. Savings were to be identified as potential benefits in the business case and project plan templates for Tier 2 projects but not in the project plan template for Tier 3 projects. A financial benefits table was included in the revised business case template but there was no mention of savings in the revised project plan template. Instead, the template indicated that the project benefits management plan should be attached.

3.13 As illustrated in Table 3.3, a small number of Reinventing the ATO projects had identified and estimated savings associated with their projects (16 in total).24 Twelve projects had estimated all expected savings components in either a business case or project plan and six of those had included the calculations to support the savings estimates. The ANAO assessed the six projects that had included calculations and found that three of those estimates appeared to be of a sound basis.25 The ANAO could not assess the remaining three as the supporting calculations were of insufficient detail for two projects and the third was based on an external provider’s estimate.

Table 3.3: Number of projects that identified and estimated savings

 

Business/value case

Project plan/outline

No. of projects that did not require document

17

3

No. of projects without document

41

39

No. of projects that did not identify any monetary savings

15a

36b

No. of projects that indicated that savings were not expected

1

1

No. of projects that identified monetary savings but did not estimate them

15

14

No. of projects that wholly or partly identified and estimated monetary savingsc

11

7

Total

100

100

     

Note a: Two of these projects did not identify any benefits.

Note b: Six of these projects did not identify any benefits.

Note c: The projects identified in each category are not mutually exclusive and 16 projects in total identified savings in either a value/business case and/or project plan/outline.

Source: ANAO analysis of ATO information.

3.14 Table 3.4 outlines the total estimated monetary savings outlined by Reinventing the ATO projects. As some projects included options, there are ranges for the estimated savings.

Table 3.4: Estimated monetary savings associated with Reinventing the ATO projects

Estimated monetary savings in first five years

Estimated monetary savings per annum after the first five years

Low

High

Low

High

$419.5 million

$598.9 million

$57.4 million

$81.6 million

       

Note: These figures are based on the estimated savings included in the project planning documentation of 16 Reinventing the ATO projects. The savings estimates do not relate to a specific time period. Eight projects estimated savings over a number of years rather than on a per annum basis; a five-year period was included to accommodate those projects as five years was the longest estimated period. Annual estimates have been recalculated accordingly. Ranges are reflected as two projects included a range of estimated savings. For projects that did not include a range, the total value of savings is included in both the ‘low’ and ‘high’ estimate.

Source: ANAO analysis of ATO information.

3.15 Including savings estimates in value cases and project plans informs value for money assessments. As outlined in paragraph 3.3, the revised project management approach is expected to deliver a greater emphasis on the value proposition of proposed projects. To support informed decision-making of this nature, the ATO needs to improve the consistency with which project cost and savings information is outlined in key project documentation.

3.16 There is no total savings estimate for the Reinventing the ATO program. While the ATO identified productivity offsets associated with the Reinventing the ATO program to support the remuneration increases in its proposed 2015 Enterprise Agreement26, it advised the ANAO that these offsets were ‘intangible’ savings that were expected to be delivered from indirect benefits associated with the program. Examples of these benefits included: addressing ‘irritants’ associated with the ATO’s case management system; simplifying internal interactions as well as those with taxpayers; and engaging with taxpayers through different communication modes, such as social media.

Was there a sound basis for estimating the benefits expected from Reinventing the ATO projects?

While the ATO’s guidance has consistently required that project outcomes and benefits are identified and recorded in key project management documentation, conformance with these requirements by Reinventing the ATO projects has been low. Only 56 of the 100 Reinventing the ATO projects outlined expected benefits in project pre-approval documentation, including non-financial benefits and productivity improvements. Under the revised project management approach, the ATO’s benefits management processes have been strengthened to require that project outcomes align with broader ATO organisational priorities.

3.17 Successive ATO project management frameworks have required that benefits and outcomes be identified and measured for projects. According to these frameworks, outcomes are a change in state; they are the results or impacts that follow from the change the project is making. Outcomes are expected to lead to benefits. Benefits are measurable changes that contribute to the ATO’s corporate objectives, for example increased taxation revenue or increased stakeholder confidence.

Outcomes

3.18 Under the previous project management approach, Tier 2 and 3 projects were required to identify outcomes in the following project management documents: outcomes map; project plan/outline; and benefits management plan.27 Tier 2 projects were also required to outline outcomes in the business case.28 The ATO’s project management guidance indicated that outcomes were to be described in terms of the difference between the current and future state, and the problem that would be addressed. Outcomes were to be categorised according to the group to benefit—community (individuals, small business and tax agents), Government and the ATO.

3.19 The ATO recognised shortcomings in relation to managing outcomes under the previous project management approach, including that project outcome success measures needed improving, and outcomes owners were not always identified. As previously mentioned, a revised project management approach was introduced in July 2016. The revised approach aimed to improve conformance with project management requirements, improve accountability for outcomes and enhance the ATO’s ability to monitor and realise corporate benefits.

3.20 The revised project management guidance indicates that outcomes should be defined for all projects in a value case.29 The value case template focuses projects on linking key project outcomes to corporate priorities and defined corporate benefits categories. In addition to the value case, the revised project management approach requires an outcomes map and benefits management plan or entry in the corporate benefits register.30 The ATO’s guidance states that the level of benefits management planning will vary depending on a project’s assessed level of risk, complexity, change and business impact.31

3.21 Around one-third of the Reinventing the ATO projects had documented intended outcomes in a business or value case, while nearly half had included outcomes in a project plan or outline as illustrated in Table 3.5.

Table 3.5: Number of Reinventing the ATO projects with intended outcomes in planning documentation

 

Business value case

Project outcomes map

Project plan outline

Benefit management plan/register

No. of projects that did not require document

17

N/Aa

3

0

No. of projects with no document

41

67

39

84

No. of projects that did not document outcomes

10

n/a

10

1

No. of projects that documented outcomes

32

33

48

15

Total

100

100

100

100

         

Note a: The ANAO was unable to reliably determine from project management documentation those Tier 2 and 3 projects that were subject to the Department of Finance Gateway review process or an ATO IT review process, or were discretionary projects where the IT component was estimated at greater than $500 000.

Source: ANAO analysis of ATO project documentation.

3.22 Seven projects in the Reinventing the ATO program had all of the required documents for the initial stages of the project. Of those seven projects, five had recorded intended project outcomes in all the required documents.

3.23 Conformance with the ATO’s project management requirements to identify and document expected outcomes and benefits has been low among Reinventing the ATO projects. Defined project outcomes and benefits support informed decision-making as to the value proposition of projects as well as assist projects to remain on track during implementation. Future Reinventing the ATO projects should outline expected outcomes and benefits in pre-approval project documentation.

Outcomes mapping

3.24 The ATO’s outcomes mapping process was strengthened with the introduction of the Connected Benefits Management System and revised project management approach in 2016, with the requirement that outcomes are directly linked to defined corporate benefits categories. This enhancement is expected to better support a project’s value proposition. Project outcomes maps are intended to show the relationship between project outcomes and benefits. Currently the ATO prescribes three types of outcomes:

  • capability outcome: when the business can do something that it could not previously do;
  • change outcome: when the business exhibits a new behaviour or way of doing something; and
  • strategic outcome: a benefit that will arise from doing a program of work.

3.25 The Value Management Office facilitated outcomes mapping for the six strategic programs and the Reinventing the ATO program from 2014 to 2016. The strategic programs mapping process assisted the Value Management Office to identify projects that would also benefit from undertaking the mapping process. Project outcomes would contribute to the Reinventing the ATO program outcomes through the six strategic programs as illustrated in Figure 3.2.

Figure 3.2: Expected alignment between project outcomes and ATO corporate priorities

Source: ANAO.

3.26 Thirty-three projects had completed outcomes maps. The ANAO analysed the alignment between the outcomes maps of these 33 projects, the six strategic programs and the Reinventing the ATO program. The ANAO also examined the extent to which outcomes maps linked to the ATO corporate impact areas. As illustrated in Table 3.6, there was a high-level of conformance in identifying programs’ outcomes and corporate priorities on the six strategic programs’ and the Reinventing the ATO program outcomes maps. However, individual Reinventing the ATO projects’ outcomes were not discernible on the six strategic programs’ or the Reinventing the ATO outcomes maps. Further, there was a lower level of conformance among the 33 projects with identifying relevant links on their respective outcomes maps as demonstrated by:

  • five projects (15.2 per cent) identified their relevant strategic program outcomes;
  • 24 projects (72.7 per cent) identified relevant Reinventing the ATO outcomes; and
  • 23 projects (69.7 per cent) identified ATO corporate priorities.

Two of the 33 projects did not identify their own project outcomes on projects maps.

Table 3.6: Project outcomes mapping information

 

Project outcomes mapsa

Six strategic program outcomes maps

Reinventing the ATO outcomes map

ATO corporate impact areasb

23

Reinventing the ATO program outcomes

24

Strategic program outcomes

5

Project outcomes

31c

Note a: From a total of 33 projects.

Note b: Refer to the inner circle of Figure 2.1 for the ATO corporate impact areas.

Note c: Ten of these projects had outcomes that clearly aligned from their outcomes map to their business case.

Source: ANAO analysis of ATO information.

3.27 Greater transparency of the expected contribution by existing and planned projects to program outcomes would enable the ATO to better prioritise projects that would deliver the greatest value as well as assist it to identify where there are gaps in efforts to achieve intended program outcomes.

Benefits

3.28 Identifying and recording expected project benefits in key project documentation has been required under successive project management frameworks, as illustrated in Table 3.7. Guidance for Tier 2 projects under the previous project management framework included that information on benefits in addition to costs and options assists to present value for money propositions and confirm the viability of projects.

Table 3.7: Requirements to outline benefits in project management documentation

 

Previous framework

Current framework

 

Tier 2

Tier 3

 

Business/value case

Benefits as well as the means to measure them to be identified.

N/A

Corporate indicators and benefit categories are to be identified for each planned outcome.

Project plan/outline

Benefits to be identified and an update on the intended approach to measure them to be included.

Benefits to be listed.

Benefits management plan to be attached to project plan.a

       

Note a: For simple projects, rather than developing a benefits management plan, benefits are listed in the project log.

Source: ANAO analysis of ATO information.

3.29 The ATO’s Tier 2 project management guidance encouraged the consideration of benefits according to the categories of:

  • financial: benefits that can be measured in monetary terms, for example, an increase in taxation revenue;
  • non-financial: qualitative benefits such as improved access to systems or greater confidence in the taxation system; and
  • savings: quantifiable reductions in the ATO’s running costs.

3.30 Under the revised project management framework, project outcomes are linked directly to existing corporate indicators and benefits categories.32 Further, internal financial benefits are identified as harvestable benefits, reinvested benefits or notional savings.

3.31 As illustrated in Table 3.8, around one-third of the Reinventing the ATO projects had expected benefits outlined in value cases and half of the projects had benefits outlined in project plans. In total, 56 of the 100 projects included expected benefits in at least one pre-approval document.

Table 3.8: Conformance with documenting benefits in project management pre-approval documentation

 

Value/business case

Project plan/outline

No. of projects that did not require document

17

3

No. of projects that did not have document

41

39

No. of projects with documentation that did not identify benefits

7

7

Total no. of projects that identified benefits

35

51

  • no. of projects that identified financial benefits

21

26

  • no. of projects that identified non-financial benefits

35

50

  • no. of projects that identified savings

26

21

     

Source: ANAO analysis of ATO information.

3.32 To determine whether benefits have been achieved, benefit measurement methods need to be identified and applied. For the Reinventing the ATO program, of those projects that identified benefits:

  • 15 projects identified measurement methods in value cases for some or all of the identified benefits and 20 did not; and
  • 20 projects identified measurement methods for some or all of the identified benefits in project plans and 31 did not.33

3.33 Benefits and their measurement methods have not been commonly identified and recorded. Consequently, projects’ value proposition and viability may not have been discernible from project management documentation.

3.34 A high level of conformance with the revised project management framework is required to enable the ATO to initiate projects with strong value propositions that contribute to its organisational priorities.

3.35 The ANAO reviewed the documentation for all six projects that were initiated after the introduction of the Connected Benefits Management System and revised project management framework.

3.36 Table 3.9 illustrates that half the projects that were initiated under the revised project management framework have been cancelled, with two of the projects cancelled prior to the development of a value case. For the three projects that remained in flight as at August 2017, two had developed value cases but not an outcomes map. The remaining project had been exempted from developing a value case and outcomes map, and was approved to proceed to developing a project plan.

Table 3.9: Projects in the Reinventing the ATO program initiated under the revised project management framework

 

Value case

Outcomes map

Project plan

Benefits management plan

Document links to Reinventing the ATO

Project status

Project 1

Exempted

Exempted

In flight

Project 2

Not required

Not required

Not required

Not required

Not required

Cancelleda

Project 3

b

In flight

Project 4

Not required

Not required

Not required

Not required

Not required

Cancelleda

Project 5

c

Cancelledd

Project 6

b

In flight

             

Note a: These projects were cancelled as they did not align to organisational priorities.

Note b: These projects identified their key outcomes as Reinventing the ATO program outcomes in their value cases.

Note c: Project contributions to the Reinventing the ATO program were in a briefing update and not the standard project documentation.

Note d: Analysis undertaken by the ATO indicated a lack of benefit from the project and it was consequently cancelled.

Source: ANAO analysis of ATO information.

4. Measuring and monitoring costs, savings and benefits associated with Reinventing the ATO projects

Areas examined

This chapter examines the ATO’s measurement and monitoring of costs, savings and benefits associated with Reinventing the ATO projects.

Conclusion

Costs and savings associated with the Reinventing the ATO program and most of its projects have not been tracked. However, the ATO recently introduced internal financial benefits reporting that provides a framework for measuring and monitoring savings from Reinventing the ATO projects going forward. The ATO’s benefits measurement approach has been strengthened since the commencement of the Reinventing the ATO program to enhance the profile of benefits and their alignment with broader ATO corporate impact areas when considering the value proposition of potential projects. Nonetheless, a lack of completeness in monitoring and reporting on the achievement of Reinventing the ATO projects, and the program more broadly, has limited transparency about the scale and nature of benefits achieved.

Areas for improvement

The ANAO made two recommendations aimed at the ATO: increasing the number of Reinventing the ATO projects that record and report on actual project costs (paragraph 4.10); and improving its application of project management mechanisms to assess the ongoing viability of projects (paragraph 4.50).

Is the ATO measuring and monitoring the costs of implementing the Reinventing the ATO projects?

The ATO is unable to measure and monitor the total costs of implementing Reinventing the ATO projects because of low levels of conformance with requirements to track costs—only eight projects included actual costs in status reports and 13 projects included actual costs in closure reports. The magnitude of costs of Reinventing the ATO projects warrants greater attention to measurement and monitoring—as for the 67 Reinventing the ATO projects where data was available, costs were estimated at $300 million from 2013–14 to 2018–19.

4.1 Project managers are required to track actual project costs using ‘mandated’ status reports.34 The ATO’s costing guidance states that regularly tracking actual costs along with the progress of project deliverables assists with keeping the project on track and maintaining actual expenditure within budget. There is no single process or system for tracking costs for all projects, rather the ATO has a differentiated approach. Project managers and sponsors are responsible for selecting a cost tracking method based on the complexity and scale of their project.

4.2 ATO project management guidance outlines the options available to project managers to track costs, which include:

  • Establishing a project cost centre. This approach is promoted where it is considered necessary to capture labour and supply costs associated with the project. Benefits of this approach include being able to access cost information through the ATO’s financial reporting systems. However, as there are high administrative costs associated with establishing and monitoring a cost centre, some business lines have conditions around the establishment of cost centres, including that they cannot be established where estimated project costs are less than $2 million per annum.
  • Tracking effort through the ATO Unit Costs Analysis system. This approach only captures ATO staff costs—it does not capture supplier costs. The costing guidance indicates that this approach to tracking costs is suitable for low risk, small to medium sized projects with minimal supplier costs.
  • Recording costs in the Enterprise Project Management system. Projects undertaken by the ATO’s Enterprise Solution and Technology business line use this alternative cost recording system.
  • Manually recording actual cost information. The ATO’s costing tool can also be used to capture actual cost information. Applying this approach, project managers manually record staff effort and supplier costs in the costing tool. While this approach enables all project costs to be captured and can provide financial reporting that illustrates actual cost compared with budgeted cost, it is resource-intensive.

4.3 The ATO’s costing guidance indicates that the cost tracking approach to be adopted for the project should be documented in ‘relevant project documentation’, such as project plans and business cases.35 As illustrated in Table 4.1, cost tracking methods were not frequently identified in pre-approval documentation for Reinventing the ATO projects.

Table 4.1: Number of Reinventing the ATO projects where the cost tracking method was identified in project pre-approval documentation

 

Concept brief/project proposal

Business/value case

Project plan/outline

No. of projects that did not require document

0

17

3

No. of projects without document

59

41

39

No. of projects with draft or final document that did not specify a cost tracking method

45

31

41

No. of projects with draft or final document that specified a cost tracking methoda

Nil

11

17

Total

104b

100

100

       

Note a: The projects identified in each category are not mutually exclusive and 25 projects identified a cost tracking method in either a value/business case and/or project plan/outline.

Note b: These figures add to 104 as six projects that were initially scoped were later merged into two projects.

Source: ANAO analysis of ATO information.

4.4 In total, 25 of the 100 Reinventing the ATO projects identified the cost tracking method in project pre-approval documentation. Where the tracking method was identified in project plans, conformance with the ATO’s guidance (paragraph 4.2) varied, as illustrated in Table 4.2. Of the 17 projects that identified a cost tracking method in a project plan, manual tracking was the most common method identified.

Table 4.2: Cost tracking methods identified in project plans

Costing model

No. of times identified in project plans

Combination: cost centre/project costing model/IT costing model

1

ATO Unit Costs Analysis system

1

Enterprise Solution and Technology costing modela

2

Manually track project costs

8

Project scheduling tool

2

Work breakdown structure

1

Spreadsheet

2

Total

17

   

Note a: Two projects identified tracking methods for IT-related costs only.

Source: ANAO analysis of ATO information.

4.5 Actual project costs are required to be included in monthly status reports and closure reports, as illustrated in Table 4.3.

Table 4.3: Requirements to include actual cost information in project documentation

 

Previous framework

Current framework

 

Tier 2

Tier 3

 

Status report

Actual projects costs to date

Actual projects costs to date

Actual/forecast costs (year-to-date, full year and whole project)

Closure report

Total actual costs

Total actual costs

Budget variances

       

Source: ANAO analysis of ATO information.

4.6 As illustrated in Table 4.4, eight projects reported actual project costs in their status reports. Of the 43 projects that did prepare status reporting but did not report the actual cost of the project: 24 applied traffic light reporting to represent their budget position; two included traffic light reporting and reported the actual project cost in less than half of their status reports; five included either traffic light reporting on budget position or no budget reporting; and 12 did not include any budget information in status reports. As also illustrated in Table 4.4, 43 of the 56 Reinventing the ATO projects that should have had closure reports either did not prepare them or if a report was prepared, it did not include the actual cost of the project.36

Table 4.4: Number of Reinventing the ATO projects where cost information was included in status or closure reports

 

Status report

Closure report

No. of projects that did not require document

0

44

No. of projects without document

49

15

No. of projects with draft or final document that did not report actual project costs

43a

28b

No. of projects with draft or final document that reported actual project costs

8c

13

Total

100

100

     

Note a: Two of these projects did include the actual project cost but in less than half of their project status reports.

Note b: Six of these projects indicated that no cost was recorded as the project had been undertaken from business-as-usual resources or was now being undertaken as part of business-as-usual resources. One other project indicated that the actual cost of the project had not been captured.

Note c: Two of these projects did not report the actual projects costs in all status reports but reported it in more than half of their reports.

Source: ANAO analysis of ATO information.

4.7 The inconsistency with which project status reports are being prepared, including both the frequency and content of the reporting, does not enable the ATO to effectively measure or monitor the costs associated with the Reinventing the ATO program.37 This finding is supported by an ATO internal audit of the Reinventing the ATO program (October 2016) that found the costs of the Reinventing the ATO initiatives were unclear as was the identity of those responsible for monitoring costs. The ATO further advised that costs are not being tracked at the Reinventing the ATO program level.

4.8 A review of project management documentation38 found that cost estimates were available for 67 projects totalling $300.6 million for the period from 2013–14 to 2018–19. A review of project overviews and closure reports found that 17 of the 67 projects had actual costs recorded totalling $85.0 million.39 Total actual costs were $9.2 million less than the estimated costs for those 17 projects.40 The total estimated cost for those projects with no actual reported cost is $206.4 million (50 projects).

4.9 The ATO’s costing guidance indicates that regularly tracking actual project costs can help project managers to keep their projects on track, and variances between project budgets and actual costs can be useful indicators of project progress. The guidance further states that accurately recording actual cost can assist other project managers to better estimate costs for future projects. In addition to the benefits mentioned in the ATO costing guidance, this information can inform decision-making in relation to cessation or continuation of a project.

Recommendation no.1

4.10 The Australian Taxation Office mandates and monitors the recording and reporting of actual project costs for all corporate projects.

Australian Taxation Office response: Partially agreed.

4.11 We have commenced work to improve the recording and reporting of actual project costs where considered appropriate, including:

  • increased governance support to projects, and
  • improving our tooling and guidance.

4.12 The ATO will ensure all projects have recording and reporting. The level of investment in the recording and reporting will be considered using a risk-based approach to ensure the balance of recording, reporting and monitoring activities is appropriate relative to the risk, materiality and cost of the project.

Is the ATO measuring and monitoring the savings associated with implementing the Reinventing the ATO projects?

The ATO has not been tracking the monetary savings associated with the Reinventing the ATO program. However, in April 2017 it implemented internal reporting on financial benefits across the office, including Reinventing the ATO projects. As the reporting process involves the verification of estimated and realised financial benefits, this should better position the ATO to consider realised savings when making operational decisions, such as reallocating resources due to productivity gains.

4.13 The ATO advised that savings are not being tracked at the Reinventing the ATO program level.41 Further, there is no guidance to instruct project managers to track or record savings in project management documentation under the old project management approach. The closure report template for the revised project management approach includes a section to report benefits but does not specifically indicate that savings or financial benefits should be reported. Consequently, of the 41 projects with closure reports and the 51 projects with status reports, none reported savings.

4.14 To improve the categorisation and monitoring of project outcomes and benefits, including savings and other financial benefits, the ATO introduced the Connected Benefits Management System in July 2016. Financial benefits were categorised under the system as external or internal and financial or non-financial. Internal financial benefits are further categorised as:

  • harvestable financial benefits that are tangible savings, such as funding or staff who can otherwise be redeployed; and
  • non-harvestable financial benefits that are intangible productivity savings.

4.15 In April 2017, the ATO commenced formally reporting on internal financial benefits to its Resource Forum.42 In this report, the ATO revised its definition for harvestable benefits to ‘producing a direct budget saving’ and revised the financial benefit categories to:

  • notional saving—small productivity savings or cost avoidance measures that cannot be reinvested; and
  • reinvested benefit—larger productivity benefits that can be reinvested (such as a full-time equivalent position).

4.16 As indicated in Table 4.5, no projects identified harvestable savings. The projects outlined in Table 4.5 were at different stages in the project lifecycle with some being planned, others in the delivery phase and some that have been closed. For those projects being planned, the estimated financial benefits are being assured by ATO Finance.

Table 4.5: Estimated and verified financial benefits associated with Reinventing the ATO projects in Resource Forum reports, April to June 2017

 

Notional savings

Reinvested benefits

 

No. of projectsa

Total estimated value

Total verified value

No. of projectsa

Total estimated value

Total verified value

April 2017

8

$44.6 m

$45 000

1

$1.2 m

Nil

May 2017

6

$38.5 m

$45 000

3

Nil

Nil

June 2017

6

$44.0 m

$45 000

1

$1.2 m

Nil

             

Note a: Projects not part of Reinventing the ATO were included in the reports but excluded from the ANAO’s analysis. The Reinventing the ATO projects included in the table include those projects that are currently being planned as well as closed projects.

Source: ANAO analysis of ATO information.

4.17 ATO Finance is also responsible for verifying realised financial benefits. However, the Resource Forum reports acknowledged that ATO Finance is unable to verify whether financial benefits have been achieved for two of the five closed Reinventing the ATO projects reported, as baselines had not been established to enable benefits to be measured.43 Further, of the remaining three projects that were reported:

  • one had verified notional savings of $45 000. While the project had identified monetary savings in project documentation, it had not estimated the value of potential savings;
  • one had estimated notional savings of $5.5 million that are yet to be verified by ATO Finance. This project had also identified but not estimated monetary savings in its project planning documentation; and
  • one did not realise any financial benefits and had identified but not estimated savings in its value case.

4.18 The reporting methodology for unverified financial benefits has improved during the three-month reporting period. For example, an unverified estimate of notional savings was included in the April 2017 report, but was removed from the May 2017 report on the basis that it could not be confirmed.

4.19 While the ATO has not been measuring or monitoring the monetary savings associated with the Reinventing the ATO program or its projects, its new internal financial benefit reporting process should assist it to capture some of the financial benefits associated with the program going forward. Further, the ATO’s approach to verifying estimated and realised financial benefits has the potential to provide greater rigour in the reporting process and enable the ATO to reinvest realised benefits in a practical way such as returning savings to the consolidated revenue fund.

Does the ATO measure and monitor the actual benefits associated with the Reinventing the ATO projects?

The Reinventing the ATO program has provided a number of benefits, as indicated by the large number of outcomes listed as achieved for individual projects. However, there is a lack of clarity about the results of Reinventing the ATO projects as a consequence of the:

  • lack of conformance with the ATO’s processes for monitoring and reporting on the achievement of project outcomes—of 57 projects closed, cancelled or transferred to business-as-usual, 21 had closure reports that indicated whether project outcomes had been achieved; and
  • implementation of the Connected Benefits Management System after the commencement of the program and many projects, and the ATO not accurately identifying the contribution made by Reinventing the ATO projects to corporate priorities.

Reporting on project outcomes

4.20 The ATO’s project management guidance, under the previous and revised approaches, requires that a closure report is completed to record the formal termination of a project. It is expected that the closure report aligns with planning documentation, as illustrated in Table 4.6.

Table 4.6: Requirement for reporting on outcomes

 

Tier 2

Tier 3

Revised approach

Closure report

Include a description of what the project set out to achieve as per the business case and the project plan.

Include a description of what the project set out to achieve as per the project outline.

Include the project’s key outcomes, their relevant benefits categories and contributions as defined in the value case and project plan.

       

Source: ATO project management guidance.

4.21 Under the previous project management approach, the closure report included a series of questions for the project sponsor to answer including whether the agreed project outcomes and intent had been achieved. An explanation was required if the project had not achieved its expected outcomes and intent. Under the revised approach, the closure report template focuses on summarising any benefits realised at the time of project closure in relation to the project outcomes. The achievement of project outcomes under the revised approach is captured in the post implementation review, when most expected benefits have been realised and the intention is to summarise the overall performance of the project.

4.22 Of the 57 projects closed, cancelled or transferred to business-as-usual in the Reinventing the ATO program, there were 41 projects with closure reports. Twenty-five of those projects had recorded intended outcomes in the appropriate planning documentation and 75 of 132 project outcomes (57 per cent) had been reported as achieved in closure reports, as illustrated in Table 4.7. The other 16 projects with closure reports in the Reinventing the ATO program had not clearly recorded the intended outcomes in planning documentation and for 11 of those projects it could not be determined if project outcomes were achieved from the closure reports.

Table 4.7: Outcomes reported as achieved at project closure

 

Total

Range of outcomes per project

No. of projectsa

Intended outcomes recorded in planning documentationb

132

1 to 17

25

  • outcomes reported as achieved

75c

1 to 13

16

  • outcomes reported as not achieved

11

1 to 4

5

  • outcomes achievement unknown

46

1 to 12

11

       

Note a: The number of projects with outcomes reported as achieved, not achieved or where the result is unknown, do not total to 25 as these categories are not mutually exclusive. For example, one project may have reported a number of outcomes as achieved and others as not achieved.

Note b: Intended outcomes were recorded in the business/value case, project plan/outline, outcomes map or benefits management plan.

Note c: Four outcomes were partly achieved.

Source: ANAO analysis of ATO information.

4.23 Twenty-one projects had started under the previous project management approach and closed using the revised closure report template. The transition of project management methodology during the course of a project introduced difficulties in determining whether intended outcomes were achieved, such as:

  • discrepancies between the description of outcomes in planning and closing documentation; and
  • the revised closure report focused on the realisation of benefits.
Post implementation review

4.24 Post implementation reviews are undertaken after a project has closed to evaluate whether outcomes have been achieved and determine if benefits have been realised.44 The previous ATO project management guidance required projects to indicate in the closure report if a post implementation review was to be undertaken.45

4.25 Under the revised approach, the closure report does not indicate if a post implementation review is required. However, it includes a section for post project value realisation information that should be reflected from the benefits management plan.46 The achievement of outcomes is recorded in the revised post implementation review template and the relevant governance body determines if a project should conduct a post implementation review according to project complexity and budget. Post implementation review templates under both approaches required information for reporting on intended outcomes and benefits.

4.26 As indicated in paragraph 4.22, there were 41 projects with closure reports in the Reinventing the ATO program. Of the 21 projects that had closed under the revised approach, none had completed a post implementation review. The remaining 20 projects had closed under the previous project management approach, with seven projects indicating in their closure reports that a post implementation review was to be conducted, as illustrated in Table 4.8.

Table 4.8: Projects requiring post implementation review

 

No. of projects

Closure report under the previous project management approach

20

Post implementation review indicated in closure report

7

Post implementation report completed

5

   

Source: ANAO analysis of ATO information.

4.27 Two of the completed post implementation review reports identified in Table 4.8 had applied the project management template and had clearly assessed the outcomes described in the business case and reported on benefits realised. For the two projects, 19 of 26 intended project outcomes had been achieved resulting in the realisation of financial and non-financial benefits. The remaining three post implementation review reports did not report against business case success measures, and as a result the achievement of outcomes and benefits was less clear.

Portfolio benefits reporting

4.28 The Value Management Office prepared portfolio benefits status reports for the Integration Forum47 and the Executive. Since reporting began in May 2016, 11 reports were produced to June 2017. The portfolio benefits reports included analysis of projects from all change programs of work in the ATO.

4.29 Initially the portfolio benefits reports captured the status of project documentation required by the relevant project management approach and some reporting on benefits. The format and content of reporting however changed over time as the Connected Benefits Management System was introduced and reporting progressed. For example, six of the 11 reports included reporting on the status of individual projects in a variety of ways, such as:

  • two reports identified priority projects, although how they were prioritised was not clear;
  • three reports identified high engagement projects with the Value Management Office; and
  • one report included individual projects that had been ordered to halt, or stop and close.

4.30 A traffic light system was used for the high engagement projects to indicate the level of confidence that the Value Management Office maintained that projects were on track. The Value Management Office maintained a high level of confidence regarding progress for most of the Reinventing the ATO projects that had high engagement. The Value Management Office did not report a low level of confidence regarding the progress of any Reinventing the ATO projects.

Did the ATO identify and address projects not delivering anticipated benefits?

The ATO has identified and discontinued projects as a result of concerns relating to their relevance and progress. Nevertheless, there is scope for the ATO to improve the: frequency of program status reporting to governance bodies; quality of information provided in relation to projects’ status; and use of governance gates.

4.31 Since the commencement of the Reinventing the ATO program, 11 projects have been cancelled, as a result of the ATO’s review of Reinventing the ATO projects. Table 4.9 outlines the reasons for cancelling these programs.

Table 4.9: Reasons for cancelled Reinventing the ATO projects

Reason for cancellation

No. of projects cancelled

Work is considered to be unnecessary, may no longer be relevant or does not align with organisational priorities

4

Lack of progress made on projects

3

Work is being undertaken as part of another project

2

Work is no longer considered project-based work

2

Total

11

   

Source: ANAO analysis of ATO information.

4.32 These projects were identified for cancellation through a number of different exercises, including queries from program governance bodies regarding the absence of status reporting from specific projects.48 A second exercise was undertaken following the unscheduled system outages experienced by the ATO in December 2016 and January 2017. As a result of these outages, the ATO cancelled a number of projects that did not relate to the four identified priority initiatives for the remainder of 2016–17.49

4.33 Status reporting is one of two regular, structured processes for the ATO to monitor whether projects are achieving their intended benefits and identify where corrective action may be needed; the second process is governance gate reviews.

Status reporting

4.34 Status reporting is required for the Reinventing the ATO program and projects, as illustrated in Figure 4.1.

Figure 4.1: Reinventing the ATO status reporting requirements

Source: ANAO analysis of ATO information.

4.35 As outlined in Table 4.4, project status reporting has been sporadic. The preparation of strategic program status reporting has varied among programs, with programs preparing between 16 and 24 status reports each, as illustrated in Figure 4.2. Program status reporting did not always align with program governance body meetings and there was variance among program governance bodies in relation to the frequency of meetings as also illustrated in Figure 4.2.

Figure 4.2: Frequency of program governance body meetings

Source: ANAO analysis of ATO information.

4.36 Program status reporting includes traffic light reporting to demonstrate individual projects’ overall status positions. Some program status reports also use traffic light reporting to illustrate other aspects of project health including budget, scope, readiness and schedule. Challenges associated with identifying and addressing projects at risk of not delivering their expected benefits include adequate monitoring of project status, and the capacity of program governance bodies to take action to address identified issues.

Monitoring of project status

4.37 The ANAO reviewed the monitoring of Reinventing the ATO project status through traffic light reporting. As illustrated in Table 4.10, 20 of these projects reported an overall status of red or amber for more than half of their status reports.50

Table 4.10: Projects monitored by program governance bodies and status

Program

Total no. of projects that have reported to the governance body

No. of Reinventing the ATO program projects

No. of Reinventing the ATO projects with overall status of red or amber

Tailored Engagement and Support

34

18

2

Optimise Workforce Capability and Culture

31

22

4

Working with all our partners in Tax and Super Systems

16

3

1

Governance, Design and Evaluation

15

9

2

Contemporary Digital Services

88

37

8

Smarter Data

29

11

3

Total

213

100

20

       

Source: ANAO analysis of ATO information.

4.38 The Contemporary Digital Services program had the largest number of projects with an overall status of red or amber (8 of 37 projects).51 As illustrated in Figure 4.3, a significant proportion of these projects were often discussed at program governance body meetings, together with projects that had reported an overall status of green.

Figure 4.3: Discussion of Reinventing the ATO projects with ongoing issues by the Contemporary Digital Services program

Note: No meetings were held in September, November or December 2015, January 2016 or January 2017.

Source: ANAO analysis of ATO information.

4.39 In 18 instances, these projects were subject to follow-up action (Table 4.11). Requesting additional information was the most frequent action, such as in relation to the total expected cost of the project, the original project budget and the amount spent to date (refer Chapter 3). In only one instance was a remediation opportunity identified for action.

Table 4.11: Nature of action taken in relation to projects with ongoing issues

Nature of action

No. of times

Additional information requested

6

Recommendation to put project on hold

3

Request that rating be adjusted

3

Direction for project provided

3

Escalation for action

2

Remediation opportunities identified and to be actioned

1

Total

18

   

Source: ANAO analysis of ATO information.

4.40 Due to the variability of issues affecting these eight projects52 (refer paragraph 4.38), even with greater scrutiny, the extent that the program governance bodies can take corrective action is limited:

  • two projects are subject to intergovernmental dependencies; one of these projects has been put on-hold until a whole-of-government solution can be implemented;
  • two projects are subject to delays while the ATO allocates resources to other organisational priorities; and
  • three projects have been impacted by inadequate project planning; one of these projects has been put on-hold while project outcomes are defined.
Provision of adequate information to program governance bodies

4.41 The ANAO also reviewed the Governance, Design and Evaluation program in further detail, as that program had a high number of instances where reporting was not provided in relation to particular projects.53

4.42 The Governance, Design and Evaluation program has nine projects as illustrated in Table 4.10. For one of the projects, no program-level status reporting was completed.54 While projects were identified in status reports, an update on their progress was often not included and instead the reports would indicate that the projects could not be assessed, as illustrated in Figure 4.4.

Figure 4.4: Proportion of Governance, Design and Evaluation projects that could not be assessed for reporting purposes

Note: No meetings were held in November 2015 or January, June, August, September and October 2016, or January 2017.

Source: ANAO analysis of ATO information.

4.43 In February 2017, six of these projects were closed, one was cancelled and the overall status was not reported for the remaining two projects.

4.44 The absence of reported project information does not support timely corrective action to be identified and implemented when required.

Governance gate

4.45 As discussed in Chapter 3, the previous and revised project management frameworks include governance gates that are intended to serve as decision points for project progression. Under the previous framework, project sponsors were responsible for these decisions, however responsibility shifted to program governance bodies under the revised program management framework.

4.46 Projects should be assessed at the different governance gates to determine whether projects should be continued, modified or discontinued. Both project management frameworks include governance gate templates to assist with this determination.

4.47 There was limited conformance with undertaking these governance gate reviews among the Reinventing the ATO projects:

  • governance gate 1 templates were completed for five projects; of which, three had decisions to proceed recorded and two did not have any decision recorded;
  • governance gate 2 templates were completed for eight projects; of which, seven had decisions to proceed recorded and one did not have any decision recorded;
  • governance gate 3 templates were not completed for any projects; and
  • governance gate 4 templates were completed for one project, which did not record any decision.

4.48 As at June 2017, 18 projects should have the full suite of governance gate documentation as they were either Tier 2 or initiated under the revised project management framework and have closed. Another 36 projects that were Tier 2 or initiated under the revised project management approach were in flight, cancelled, halted or transferred to business-as-usual, and should have passed at least governance gate 1 and possibly governance gates 1 and 2. Fifteen projects were Tier 3 projects and not required to complete the governance gate process.55

4.49 The level of non-conformance with undertaking these governance gate reviews indicates that the ATO was not routinely applying this framework to assess the ongoing viability and relevance of Reinventing the ATO projects.

Recommendation no.2

4.50 The Australian Taxation Office enforces the mandating of status reports and governance gate assurance activities to support assessment of the ongoing viability of projects including delivery of expected benefits.

Australian Taxation Office response: Partially agreed.

4.51 The ATO undertakes to ensure appropriate frameworks are in place to support assessment of the ongoing viability of projects including delivery of expected benefits.

4.52 In line with the response to Recommendation 1, the level of investment will be in accordance with a risk-based approach to ensure the balance of assurance activities is appropriate relative to the risk, materiality, cost and impact of the project.

4.53 The ATO is already taking steps to mature our practices through:

  • increased governance support to projects,
  • the establishment of a project assurance function that covers costs, benefits and risks, and
  • the development and implementation of a revised Benefits Management Framework.

Appendices

Appendix 1 Entity response

Appendix 2 List of Reinventing the ATO projects

Table A.1: In flight Reinventing the ATO projects

In flight projects are those underway.

Number

Strategic programa

Project name

Expected cost

Actual cost

Expected monetary savingsb

Identified outcomes in pre-approval

1

GDE

Strategic costing framework

$823 000

Not applicable (NA) as project is still underway

None identified

Yes

2

GDE

Agile development methodology

$2.0 million

NA

Not applicable

None recorded

3

WTPSS

Working together framework

$1.1 million

NA

None identified

Yes

4

WTPSS

Relationship management framework

$138 000

NA

No financial benefits expected

Yes

5

OWCC

People connect

$10.7 milliond

NA

$25.3 million over five years

Yes

6

OWCC

Video conference refresh

$3.5 million

NA

Identified but not estimated

Yes

7

OWCC

Expense management system

$1.0 million

NA

Identified but not estimated

Yes

8

OWCC

ATO staff release train

$12.9 million

NA

$13.5 million per annum

$18.2 million one-off

Yes

9

CDS

ATO using myGOV

$1.4 million

NA

Identified but not estimated

Yes

10

CDS

Small business newsroom

$3.6 million

NA

Identified but not estimated

Yes

11

CDS

Trust improvement project

$4.6 million

NA

$4.4 million per annum

Yes

12

CDS

Software services

Not reported

NA

Identified but not estimated

Yes

13

CDS

Simplified business activity statements for goods and services taxc

$131 000

NA

None identified

Yes

14

CDS

Relationship and authorisation managerc

$11.1 million

NA

None identified

Yes

15

CDS

Omni phase 1

$2.2 million

NA

$25.0 million over four years

Yes

16

CDS

Omni phase 2

$13.9 million

NA

$25.3 million over four years

Yes

17

CDS

Manage Australian business number connections

$3.1 million

NA

None identified

Yes

18

CDS

Cloud software authenticationc

$743 000

NA

None identified

Yes

19

CDS

Smart device expansion

$267 000

NA

$22.0 million over four years

Yes

20

CDS

Genesys enhancements

$540 000

NA

Not applicable

None recorded

21

CDS

Financial and digital platforms

Not reported

NA

Identified but not estimated

Yes

22

CDS

Outbound communications

$19.3 million

NA

Identified but not estimated

Yes

23

CDS

Improving online services

$3.0 million

NA

Identified but not estimated

Yes

24

CDS

Super analytics

$197 000

NA

Not applicable

Yes

25

CDS

Improving ato.gov.au

$7.3 million

NA

None identified

None recorded

26

CDS

Improving client identity protection

$939 000

NA

None identified

Yes

27

CDS

Census inquiry servicec

$2.7 millione

NA

None identified

Yes

28

CDS

Authentication application

$3.6 million

NA

None identified

Yes

29

CDS

Tax file number notifications

Not reported

NA

Not applicable

None recorded

30

SD

Financial reporting analysis

$2.2 million

NA

None identified

Yes

31

SD

Smarter compliance

$700 000

NA

Not applicable

Yes

32

SD

Enterprise client profile

$8.1 million

NA

None identified

None recorded

33

SD

Data and analytics foundation infrastructurec

$724 000

NA

$5.5 million per annum

Yes

34

SD

Getting the most from our data

$7.5 million

NA

None identified

Yes

35

SD

Enabling effective interventions

Not reported

NA

Not applicable

None recorded

36

SD

Analytics driven services - release train

Not reported

NA

Not applicable

None recorded

             

Note a: The acronyms are: GDE—Governance Design and Evaluation; WTPSS—Working with all our partners in Tax and Super Systems; OWCC—Optimised Workforce Capability and Culture; CDS—Contemporary Digital Services; TES—Tailored Engagement and Support; and SD—Smarter Data.

Note b: ‘Not applicable’ represents those projects that did not have documentation or were not required to prepare documentation.

Note c: The ATO advised that while these projects align with the Reinventing the ATO program, they are new policy proposals or broader whole of government initiatives.

Note d: The ATO advised that as a result of a change request the expected cost for this project increased from $10.7 million to $19.6 million. The additional expected cost approved through the change request was not included in the ANAO’s analysis outlined in paragraph 4.8.

Note e: This figure represents the expected cost of the IT component of the project only.

Source: ANAO analysis of ATO information.

Table A.2: Transfer to business-as-usual Reinventing the ATO projects

Projects that were transferred to business-as-usual included those: where implementation of deliverables was transferred to business lines rather than being undertaken through projects; where subsequent project phases were transferred to business-as-usual; and that were not prioritised under the Reinventing the ATO program.

Number

Strategic programa

Project name

Expected cost

Actual cost

Expected monetary savingsb

Identified outcomes in pre-approval

37

OWCC

Workforce optimisation tool

$1.1 million

Not reported

None identified

Yes

38

OWCC

Virtual classrooms

$217 0000

Not reported

$4.8 million per annum

Yes

39

OWCC

Implement the ATO leadership strategy

Not reported

Not reported

Not applicable

Yes

40

TES

Future advice guidance

Not reported

Not reported

Not applicable

Yes

41

TES

Income tax assurance notification

Not reported

Not reported

Not applicable

None recorded

42

TES

Tax assurance for largest groups

Not reported

Not reported

Not applicable

None recorded

43

TES

Pay-As-You-Go Instalment future improvements strategy

$205 000

Not reported

Identified but not estimated

Yes

44

TES

Behavioural economics in the ATO’s strategy

$1.5 million

$805 000

None identified

Yes

45

SD

Smarter data portfolio and project management software solutionc

Not reported

Not reported

Not applicable

None recorded

46

SD

Smarter data software solutionc

Not reported

Not reported

Not applicable

None recorded

             

Note a: The acronyms are: GDE—Governance, Design and Evaluation; WTPSS—Working with all our partners in Tax and Super Systems; OWCC—Optimised Workforce Capability and Culture; CDS—Contemporary Digital Services; TES—Tailored Engagement and Support; and SD—Smarter Data.

Note b: ‘Not applicable’ represents those projects that did not have documentation or were not required to prepare documentation.

Note c: Documentation provided to the ANAO during the course of the audit indicated that these projects were being carried out as business-as-usual. Subsequent to the completion of the ANAO’s analysis, the ATO advised the ANAO that the status of these projects was in flight.

Source: ANAO analysis of ATO information.

Table A.3: Halted Reinventing the ATO projects

Projects can be halted for a range of reasons, including intergovernmental dependencies, organisational capacity (such as unavailability of relevant resources) and inadequate project planning.

In December 2016 and February 2017 the ATO experienced severe systems outages. These system outages and subsequent remediation work had impacts on the projects under the Reinventing the ATO program—some projects were put on hold as the ATO undertook a review to understand the impacts. As a result of the review, some projects were cancelled, closed or further halted.

Number

Strategic programa

Project name

Expected cost

Actual cost

Expected monetary savingsb

Identified outcomes in pre-approval

47

OWCC

Serraview workforce manager

$184 000

Not reported

Identified but not estimated

Yes

48

CDS

Sub accounting period application

Not reported

Not reported

Not applicable

None recorded

49

CDS

Skilling technology solution

Not reported

Not reported

Not applicable

None recorded

50

CDS

Enabling digital by defaultc

$47.1 million

Not reported

$100.2 million to $158.6 million over four years

Yes

51

CDS

Newsroom for everyone

Not reported

Not reported

Identified but not estimated

Yes

52

CDS

Digital business accountc

$15.5 million

Not reported

No benefits identified

Yes

53

TES

Enterprise decision support capability (myadvice)

$183 000

Not reported

$1.4 million over two years

$20.0 million per annum after the fourth year

Yes

             

Note a: The acronyms are: GDE—Governance, Design and Evaluation; WTPSS—Working with all our partners in Tax and Super Systems; OWCC—Optimised Workforce Capability and Culture; CDS—Contemporary Digital Services; TES—Tailored Engagement and Support; and SD—Smarter Data.

Note b: ‘Not applicable’ represents those projects that did not have documentation or were not required to prepare documentation.

Note c: The ATO advised that while these projects align with the Reinventing the ATO program, they are new policy proposals or broader whole of government initiatives.

Source: ANAO analysis of ATO information.

Table A.4: Closed Reinventing the ATO projects

According to the ATO’s project management approach, projects should be closed when the intended products and services to be delivered through the project have been finalised and/or accepted by the relevant business line (although there were some exceptions as shown in Table 4.7).

Number

Strategic programa

Project name

Expected cost

Actual cost

Expected monetary savingsb

Identified outcomes in pre-approval

54

GDE

Tax gap delivery

$4.0 million

$2.3 million

Not applicable

Yes

55

GDE

Enterprise change portfolio implementation framework

$2.5 million

$2.3 million

None identified

Yes

56

GDE

Enterprise change management framework implementation

$630 000

Not reported

No financial benefits expected

Yes

57

GDE

Risk integration and transparency

$264 000

$353 000

Not applicable

Yes

58

GDE

Define the performance evaluation system

$833 000

$670 000

No benefits identified

Yes

59

GDE

Articulating the ATO future business modelc

$769 000

$243 000

Not applicable

Yes

60

WTPSS

Partner relationship model and pilots

$918 000

Not reported

None identified

Yes

61

OWCC

Develop and communicate the future work environment strategy

Not reported

Not reported

Not applicable

None recorded

62

OWCC

ATO staff application

$756 000

$789 000

Identified but not estimated

Yes

63

OWCC

Reducing injury in the ATO

$202 000

$101 000

Identified but not estimated

Yes

64

OWCC

Future workspace trial

$1.3 milliond

$1.3 milliond

None identified

Yes

65

OWCC

Implement the ATO culture strategy

Not reported

Not reported

Not applicable

Yes

66

OWCC

Integrated work routing

$27.3 million

$24.6 million

$1.5 million to $25.7 million per annum

Yes

67

OWCC

Learner pathways

Not reported

Not reported

None identified

Yes

68

OWCC

Enterprise performance management (Compass)

$506 000

Not reported

$4.5 million over two years

Yes

69

OWCC

myATO

$2.1 million

$52 000

Identified but not estimated

Yes

70

OWCC

Live streaming capability

$29 000

Not reported

Identified but not estimated

Yes

71

CDS

Verint

$3.9 million

$3.0 million

$125 000 per annum

$150 000 one-off

Yes

72

CDS

Improving ato.gov.au navigation

Not reported

Not reported

Not applicable

None recorded

73

CDS

Voice authentication expansione

$26.3 million

$26.2 million

$14.0 million over four years

$6.1 million per annum after the fourth year

Yes

74

CDS

Siebel now

$19.9 million

$19.9 millionf

Identified but not estimated

Yes

75

CDS

ATO labour services request for tender

$2.4 million

$2.1 million

None identified

Yes

76

CDS

Improving online tools

$7.5 million

Not reported

None identified

Yes

77

TES

My rulings phase 1

$55 000

$8000

None identified

None recorded

78

TES

Review of private advice

$296 000

$296 000g

None identified

None recorded

79

TES

Income tax risk report

Not reported

Not reported

Not applicable

Yes

80

TES

Certainty letters to low risk individual taxpayers

$450 000

Not reported

None identified

Yes

81

TES

Tax governance framework

Not reported

Not reported

Not applicable

None recorded

82

TES

Australian Prudential Regulation Authority funds client service model

Not reported

Not reported

Not applicable

None recorded

83

TES

Apportioning fuel use for fuel tax credit claims

Not reported

Not reported

Not applicable

None recorded

84

TES

Lodge and pay

$477 000

Not reported

None identified

Yes

85

TES

Client differentiation framework

Not reported

Not reported

Not applicable

None recorded

86

TES

Show me

Not reported

Not reported

Not applicable

None recorded

87

TES

Certainty project

$244 000

Not reported

Identified but not estimated

Yes

88

SD

Data take on and pre filling

$50 000

Not reported

Not applicable

None recorded

89

SD

Information asset register

$454 000

Not reported

Identified but not estimated

Yes

             

Note a: The acronyms are: GDE—Governance, Design and Evaluation; WTPSS—Working with all our partners in Tax and Super Systems; OWCC—Optimised Workforce Capability and Culture; CDS—Contemporary Digital Services; TES—Tailored Engagement and Support; and SD—Smarter Data.

Note b: ‘Not applicable’ represents those projects that did not have documentation or were not required to prepare documentation.

Note c: As noted in Footnote 39, reported costs for some projects did not include all cost components. The cost for this project reflects consultancy costs only and therefore, does not represent the total project cost.

Note d: These figures were reported in the IT component closure report of this project and do not represent the total project expected and actual costs. This report was not included for the purpose of the ANAO’s analysis of closures as a separate project closure report was completed for the project. The whole of project closure report did not report an estimated cost or actual cost.

Note e: The ATO advised that while this project aligns with the Reinventing the ATO program, it is either a new policy proposal or broader whole of government initiative.

Note f: The closure report for this project indicated that project costs were not effectively tracked over the life of the project and as a result, the approved budget has been reported as the actual cost of the project.

Note g: The approved budget has been reported as the actual cost of the project.

Source: ANAO analysis of ATO information.

Table A.5: Cancelled Reinventing the ATO projects

Projects can be cancelled for a range of reasons, including: being no longer necessary, relevant or aligned with organisational priorities; lack of progress; duplication of work with another project; and, no longer requiring project-based work.

Number

Strategic programa

Project name

Expected cost

Actual cost

Expected monetary savingsb

Identified outcomes in pre-approval

90

GDE

ATO operating model design

Not reported

Not reported

Not applicable

None recorded

91

OWCC

SAP organisational hierarchy restructure

Not reported

Not reported

Identified but not estimated

Yes

92

OWCC

Develop the human resource system strategy

Not reported

Not reported

Not applicable

None recorded

93

OWCC

A strategic approach to capability development

Not reported

Not reported

Not applicable

None recorded

94

OWCC

Develop and shift to enterprise proceduresc

Not reported

Not reported

Not applicable

None recorded

95

CDS

Improving the business experience

Not reported

Not reported

Not applicable

None recorded

96

CDS

Increasing online Australian business number processing

$373 000

Not reported

$900 000 per annum

Yes

97

CDS

Fully digital Australian business number registration

$180 000

Not reported

$504 000 per annum

Yes

98

CDS

Strengthening myGovd

Not reported

Not reported

None identified

Yes

99

CDS

Accounting program of work phase 2

Not reported

Not reported

Not applicable

None recorded

100

TES

Find the right person

Not reported

Not reported

Not applicable

None recorded

             

Note a: The acronyms are: GDE—Governance, Design and Evaluation; WTPSS—Working with all our partners in Tax and Super Systems; OWCC—Optimised Workforce Capability and Culture; CDS—Contemporary Digital Services; TES—Tailored Engagement and Support; and SD—Smarter Data.

Note b: ‘Not applicable’ represents those projects that did not have documentation or were not required to prepare documentation.

Note c: Documentation provided to the ANAO during the course of the audit indicated that this project was cancelled. Subsequent to the completion of the ANAO’s analysis, the ATO advised that this project was supported to remain in flight.

Note d: The ATO advised that while this project aligns with the Reinventing the ATO program, it is either a new policy proposal or broader whole of government initiative.

Source: ANAO analysis of ATO information.

Footnotes

1 The Australian Business Register aims to reduce the costs to businesses of complying with government regulations.

2 Australian Taxation Office, Program blueprint summary [Internet], available from: <https://www.ato.gov.au/ about-ato/about-us/reinventing-the-ato/program-blueprint-summary/> [accessed 26 September 2017].

3 Expected productivity benefits associated with the implementation of the program were intended to be applied to offset remuneration increases included in the ATO’s proposed 2015 Enterprise Agreement. However, that agreement did not eventuate, and the ATO advised that productivity savings associated with the Reinventing the ATO program were not applied to its 2017 Enterprise Agreement.

4 Australian Taxation Office, Program blueprint summary [Internet], available from: <https://www.ato.gov.au/ about-ato/about-us/reinventing-the-ato/program-blueprint-summary/> [accessed 26 September 2017].

5 The ATO advised that when the ‘blueprint’ was released, over 200 existing projects were reviewed and assigned to the Reinventing the ATO program of work, with integration opportunities identified and some projects being stopped or amalgamated.

6 The Connected Benefits Management System was introduced with the revised corporate project management method on 1 July 2016. The Connected Benefits Management System links project and program outcomes to defined corporate benefits categories and ATO corporate priorities (refer paragraphs 2.5 to 2.7).

7 The ANAO received two contributions via the ANAO website.

8 Australian Taxation Office, Program blueprint March 2015 [Internet], available from: <https://www.ato.gov.au /uploadedFiles/Content/CR/downloads/program-blueprint-march-2015.pdf> [accessed 27 September 2017].

9 These benefits categories are identified as performance criteria in the ATO’s corporate plan.

10 The measures of performance are published in the ATO annual report. The 2016–17 report is available from: <https://www.ato.gov.au/About-ATO/Access,-accountability-and-reporting/Reporting-to-parliament/Annual-report/> [accessed 14 November 2017].

11 Projects generally have more than one outcome and can have multiple outcomes contributing to the same benefit category.

12 The ANAO assessed whether indicators had improved, declined or remained stable.

13 The benefits category with the highest number of performance improvements was ‘willing participation’.

14 The ATO is currently developing or finalising measurement frameworks for six of the categories, as indicated by those with a status of ‘baseline set’ or ‘under development’ in 2015–16.

15 These projects were funded within individual business line budgets. The ATO advised that quantifying the number of projects that have been initiated directly as a result of the Reinventing the ATO program was difficult due to the amalgamation and re-scoping of projects.

16 Findings from the certificate of assurance exercise included that: not all projects were supported by business cases; whole-of-life project costs were not well understood; and project management governance obligations were not being consistently discharged. It was confirmed in a statement of conformance with project management obligations completed in 2015 that these issues remained and may not have been fully addressed by the planned due date (mid-July).

17 Tier 1 projects were categorised as high profile and high cost with outcomes that were potentially high impact on the ATO’s reputation and community confidence; Tier 2 projects included major compliance initiatives and risk treatments, were larger high impact administrative projects or were government policy initiatives that required a high level of governance; and Tier 3 included projects that were low risk, impact and cost. There were no Tier 1 projects included in the Reinventing the ATO program.

18 Project sponsors were responsible for leading the implementation of the project and reporting on its progress to the program sponsor.

19 At project governance gates, project sponsors were expected to confirm that the project: was being well managed; remained relevant to business priorities; and, was likely to lead to the expected benefits being realised. Refer paragraphs 4.45 to 4.49.

20 For the Reinventing the ATO program, the project governance body was the Integration Forum (refer Chapter 1).

21 The Enterprise Portfolio Office was responsible for establishing and maintaining portfolio, program and project policy and methods, and assisted the Reinventing the ATO program office with coordination of governance and assurance activities for the six strategic programs. In December 2016, the Enterprise Portfolio Office amalgamated with the Reinventing the ATO program office to create the ATO Portfolio Management Office.

22 The project documentation provided by the ATO indicated there were 104 projects initially scoped, however six projects were subsequently amalgamated to create two projects, resulting in a total of 100 projects.

23 Tailoring templates can, for example, include adjusting corporate project management templates to remove or include information, or creating new templates.

24 As noted in Chapter 1, savings was not a key driver of the Reinventing the ATO program.

25 The ANAO did not undertake a comprehensive review of the savings estimate. This assessment was based on the calculations included in the ATO’s project documentation and whether they appeared reasonable, on the basis that underlying assumptions were feasible.

26 As explained in Footnote 4. The Workplace Bargaining Policy, which applies to the Australian Public Service, requires that remuneration increases associated with Enterprise Agreements must be offset by productivity improvements. Australian Public Services Commission, Bargaining Policy 2015 [Internet], available from: <http://www.apsc.gov.au/publications-and-media/archive/publications-archi...> [accessed 6 November 2017].

27 After a business case was approved, projects were required to develop a project plan. Tier 2 and 3 projects that were subject to the Department of Finance Gateway review process or an ATO IT review process, or discretionary projects where the IT component was estimated at greater than $500 000, were required to develop an outcomes map and benefits management plan. The project plan should refer to the outcomes from the business case or outcomes map and include a benefits management plan.

28 Reflecting lesser complexity, Tier 3 projects required a project outline and schedule, and did not require a business case.

29 The value case replaces the business case under the previous approach. The ATO advised that there may be instances when a value case is not required; in which case, outcomes are required to be recorded in the benefits management plan or project log.

30 Outcomes are not required to be documented in the project plan under the revised project management approach. Instead, they are required to be documented in the outcomes map and benefits management plan/register. The corporate benefits register is a central repository to enable monitoring and reporting on project outcomes and benefits at the portfolio level.

31 The Value Management Office has a central function providing consultancy services and support to project and program teams for value management, including benefits planning, monitoring and realisation. The Office facilitates the assessment of project complexity, and more complex projects are required to develop a benefits management plan. Less complex projects register benefits in the corporate project log or may also develop a benefits management plan.

32 These changes were introduced as part of the ATO’s Connected Benefits Management System in 2016 (refer Chapter 2). Under the previous project management framework, outcomes were described by project managers and detail on how the project aligned with strategic business drivers and corporate plans was to be included in business cases.

33 Three of these projects were not required to identify measurement methods as they were Tier 3 projects.

34 The ATO’s costing guidance indicates that while actual project costs do not need to be exact, project managers should take ‘reasonable efforts’ to ensure that the actual costs recorded for their project are materially accurate.

35 The guidance further states that the process used to track project costs ‘only needs to be as detailed as is necessary for you to report materially actual project costs against your approved project budget’. The ATO advised that this costing guidance supported the previous project management approach. The ATO is currently considering an organisation-wide method to capturing costs in a single system that would result in the guidance being updated.

36 Fifty-seven projects were cancelled, closed or transferred to business-as-usual. One of those projects was advised by the project office that it did not require a closure report.

37 While mandated, the completion of monthly reports was not always adhered to. The ATO advised that the use of tailored reporting templates meant that standard project data was not always being captured and was inconsistent.

38 The project management documentation reviewed included: project briefs, business cases, project plans, financial assurance tables, portfolio work orders, project overviews and closure reports. Portfolio work orders are part of the revised project management approach. They are issued by the Enterprise Portfolio Office and instruct projects of their management, governance and assurance arrangements. Project overviews were one-off reports providing a status update against aspects of project health.

39 For one project, the actual costs represented the IT component of the project only—not the total project cost. For a second project, costs had not been effectively tracked over the life of the project and therefore the estimated project cost was also reported as the actual cost.

40 For some projects, the total cost was not accurately recorded or excluded some cost components. For example, the total cost excluded travel and staff costs.

41 For the purposes of this audit, the ANAO has included savings where they are represented as a reduced monetary cost to the ATO.

42 These reports were produced from September 2016. The ATO advised that reports were informally provided on an ad-hoc basis to the Resource Forum from that time. The Resource Forum is responsible for advising the ATO Executive Committee on budget-related issues and recommending how to manage these issues from a whole-of-budget perspective.

43 The ATO advised that the projects, for which financial benefits could not be verified, were commenced prior to the introduction of ATO Finance’s verification processes. Consequently, the savings estimates and relevant baselines for those projects had not been verified by ATO Finance.

44 The ATO advised that cost, schedule and risk management could also be triggers for a post implementation review.

45 If a post implementation review was indicated, the Tier 2 or Tier 3 closure report should have included the due date for the review, and arrangements such as how it will be conducted and who will be involved. The guidance indicated that generally most Tier 2 projects would be expected to be reviewed.

46 Project value realisation information in the revised closure report includes: outcome number; benefit to be realised post project; any further measurement method; outcomes owner; planned date of measurement; and any further comments.

47 The Integration Forum provides support to the ATO Executive in relation to the Reinventing the ATO program by providing recommendations, guidance and advice to inform decision-making. Further, the forum is responsible for providing strategic advice and guidance to the strategic program governance bodies, and resolving issues and managing risks in relation to the Reinventing the ATO program.

48 Each of the six strategic programs has a program governance body. The governance bodies are responsible for providing overarching direction, guidance and governance for their respective programs. Refer Chapter 1.

49 These four initiatives were: Tax Time 2017, Superannuation new measures, Single Touch Payroll and Single Online Business Registration.

50 This analysis was only undertaken in relation to the reporting provided to the program governance bodies—not all of the reports. Program governance bodies do not exclusively monitor Reinventing the ATO projects. Table 4.10 shows that in some instances the number of Reinventing the ATO projects represented less than half the total number of projects considered by governance bodies. The diversified focus of the governance bodies may impact on the level of scrutiny they can provide Reinventing the ATO projects.

51 There were 11 projects in total, however, three projects were not examined as they had less than five status reports.

52 Of these eight projects, one has been completed.

53 This analysis was only undertaken in relation to the reporting provided to the program governance bodies—not all of the reports.

54 There was no program status reporting for three Reinventing the ATO projects in total. The other two projects were in the Smarter Data program.

55 There were 11 closed projects and another 20 in flight, cancelled, halted or transferred to business-as-usual, where it was unclear whether they should have completed the governance gate process.

Related documents: