Effectiveness of the Planning and Management of Veteran Centric Reforms
Why did we do this audit?
- The Veteran Centric Reform (VCR) program is the most comprehensive set of reforms undertaken by the Department of Veterans’ Affairs (DVA) in decades.
- It aims to significantly improve the effectiveness and efficiency of veterans’ services.
- This Auditor-General report assesses the extent of progress being made against reform objectives.
- Around 5600 members of the Australian Defence Force transition to civilian life each year.
- In 2019–20, DVA supported 225,546 veterans and 103,806 dependants.
- The average age of a veteran is reducing. The population of younger clients aged 25 to 40 years is forecast to increase by 44 per cent from 2020 to 2025.
- In 2018–19, the number of claims increased by 83 per cent, with a further 15 per cent increase in 2019–20.
What did we find?
- DVA's planning and management of the VCR program has been largely effective.
- DVA’s planning for reform was partially effective. While the rationale for investment was sound, decision-makers were not fully informed about budget and operational pressures likely to be created by reform.
- DVA’s implementation of reforms has been largely effective. Greater attention to possible systemic issues affecting performance is needed.
- DVA’s monitoring and evaluation of the VCR has been partially effective. It should more systematically monitor and report progress against all commitments, including savings.
What did we recommend?
- The Auditor-General made four recommendations to DVA aimed at strengthening: business cases; processes for addressing program under performance; the use of risk processes; and benefits reporting.
- DVA agreed to all four recommendations.
VCR funding allocated to DVA and Services Australia.
The VCR program is in its fourth year of implementation.
Pages of client records digitised under VCR.
Summary and recommendations
1. The Department of Veterans’ Affairs (DVA) was provided funding in the 2016–17 Budget to commence planning to significantly improve its provision of services. Since 2017–18, funding allocated to DVA and Services Australia totalling $653.7 million has enabled the implementation of changes to policies, services and ICT infrastructure, as part of a comprehensive set of reforms, termed the Veteran Centric Reform program (VCR).1
2. The department’s original planning envisaged six years of reform. The VCR is currently in its fourth year of implementation.2 This report is therefore a point in time assessment of DVA’s implementation of the program, and does not conclude on whether the Government’s full objectives for reform are likely to be achieved.
Rationale for undertaking the audit
3. The provision of services to veterans and their families is a subject of interest to the Parliament and the public. The VCR program is the most comprehensive set of reforms undertaken by the department in decades. Audit assurance about VCR program implementation provides transparency about whether sufficient progress is being made against reform objectives.
Audit objective and criteria
4. The objective of the audit was to examine the effectiveness of DVA’s planning and management of the VCR program. To form a conclusion against this objective, the following high-level criteria were adopted:
- Was DVA planning for veteran-centric reforms effective?
- Did DVA establish fit for purpose arrangements for implementing the VCR program?
- Does DVA appropriately monitor, evaluate and report on the implementation of reform measures?
5. DVA’s planning and management of the VCR program has been largely effective, to date, with several reform initiatives implemented. However, limitations in DVA’s project management, and the monitoring and evaluation of benefits, present risks to the VCR program being effectively implemented on time, on budget and to the Government’s objectives.
6. DVA’s development of the VCR proposal was partially effective in meeting the Government’s objectives. The VCR business case provided a clear rationale for investment and provided options for meeting the Government’s objectives. However, the department did not adequately draw the attention of decision-makers to key risks. In designing reform activities, the department engaged extensively with the veteran community and stakeholders. Its internal program planning provided a sound basis for the implementation of VCR reform activities.
7. DVA has established largely appropriate arrangements for program governance and implementation. The department has delivered a number of reform elements, and has maintained a focus on reform objectives. However, progress in implementing a number of business capabilities has been slower than expected. A greater strategic focus is needed to ensure root causes of underperformance at the program level are addressed. Partnering with other entities in the delivery of reform is generally effective, with scope for improvement. DVA staff are appropriately engaged in progressing the objectives of reforms and managing the impacts of change.
8. DVA’s monitoring, evaluation and reporting on the VCR has been partially effective. DVA regularly reports on key achievements in its implementation of the program. However, there is a need for DVA to consistently measure and report on the financial and non-financial benefits of the program. Reporting to the Parliament through its annual performance statements does not explain how reforms are affecting the department’s performance in relation to its key operating metrics. Communication with the veteran community occurs mainly through established processes and is effective.
9. The 2016–17 VCR business case established the need for, and the expected benefits of reform. Consultation with the veteran community and within government informed the development of policy and funding options for the Government’s consideration. However, DVA did not provide all information relevant to the Government’s decision-making on investment in reform. It also did not adequately set out how it would manage budget and operational pressures resulting from increases in demand for services in the short to medium-term.
10. DVA has undertaken implementation planning to give effect to VCR Budget decisions. Planning processes have maintained a focus on broad outcomes across years, and supported a coordinated approach to program management. However, changes in the organisation of delivery streams and projects have reduced the ability to track the evolution of work across years, and account for deliverables.
11. DVA’s governance structures and processes for overseeing the management of the VCR program are largely effective. DVA has maintained a focus on reform and cultural change and established accountability for delivery. Risk processes are generally appropriate, however clearer guidance is needed on conducting assurance activities at project level.
12. DVA’s implementation of VCR projects has been largely effective. It has delivered several reform initiatives since the commencement of the program. However, a number of work streams have fallen short of their expected performance. In managing the implementation of projects, greater attention to completing projects within originally planned schedules is required. Arrangements for the high-level monitoring of individual projects are largely appropriate, however DVA has not undertaken sufficient analysis to identify and address root causes of underperformance at the program level. Decision-makers should be more effectively supported to address these.
13. DVA’s management of partnering arrangements is largely effective. There is scope for DVA to further strengthen bilateral processes with Services Australia to increase value for money and manage risk in the delivery of ICT projects. DVA’s cooperation with the Department of Defence is largely effective, but requires clear and consistent measures for monitoring progress across all bilateral VCR initiatives. DVA has appropriately managed organisational changes and their impacts on its workforce. Its engagement with staff on the purpose and value of reform has been largely effective.
14. DVA’s program performance framework is broadly suitable for monitoring and evaluating the outcomes of the VCR program. However, the department’s measurement of program performance has been limited by the lack of a robust methodology and systematic monitoring. DVA has therefore not appropriately monitored the outcomes of the VCR program and is unable to demonstrate the full scope of improvement delivered by it. Work commenced in late 2020 to address this.
15. DVA’s reporting to the Parliament is largely appropriate, but would be strengthened through more complete information about the performance of the VCR program, and its relevance to improving the effectiveness and efficiency of the department. Reporting to the veterans’ community is through established mechanisms and effective.
Recommendation no. 1
The Department of Veterans’ Affairs ensure that future business cases for investment in reform provide clear and viable options for managing the Government’s exposure to risks arising from new policy and program settings.
Department of Veterans’ Affairs response: Agreed.
Recommendation no. 2
The Department of Veterans’ Affairs strengthen governance processes to include regular analyses aimed at identifying and addressing causes of program and project underperformance.
Department of Veterans’ Affairs response: Agreed.
Recommendation no. 3
The Department of Veterans’ Affairs establish, in agreement with Services Australia, risk appetite statements, related to cost and quality of service, for each ICT development project.
Department of Veterans’ Affairs response: Agreed.
Recommendation no. 4
The Department of Veterans’ Affairs develop consistent and reliable indicators of program performance to improve the quality and completeness of reporting on the benefits of veteran-centric reforms.
Department of Veterans’ Affairs response: Agreed.
Department of Veterans’ Affairs summary response
16. DVA’s summary response is provided below. The department’s full response can be found at Appendix 1.
The Department of Veterans’ Affairs (DVA) welcomes the ANAO findings and recommendations. The ANAO report acknowledges that the Veteran Centric Reform (VCR) program is the most comprehensive set of reforms undertaken by DVA in decades, and that VCR has achieved significant improvements to DVA’s culture, service delivery and reputation over the period of the program to date.
From the original VCR business case in 2016, it was recognised that DVA needed to improve its service delivery to the veteran community and simplify access to entitlements and programs so veterans and their families could engage more easily with the Department and benefit from treatment and supports available to them. Prior to implementing VCR, DVA’s ICT infrastructure and service delivery model did not support veterans to the standard expected by Government. As the ANAO observes in the report, DVA faced significant risks prior to the commencement of VCR including fragmented and obsolete ICT infrastructure, a high level of manual interventions in claims processing, a catastrophic risk of ICT failure, and lack of preparedness to meet needs and expectations of younger veterans, particularly for digital access to services.
The VCR program is focused on enabling DVA to better know, connect, support and respect veterans who serve or have served Australia. The ANAO’s report recognises the many successes of the VCR program to date, which are all focused on achieving this intent. Key areas of achievement include the increased recognition by veterans of the support and services available to them when they are transitioning from military service, the increased use by veterans and their families of the digital gateway to DVA through MyService, the connection with members of the Australian Defence Force through early engagement, and an enhanced veteran experience through upgrades to telephony, the Department’s website, digitisation of files and other ICT tools.
The Department acknowledges the ANAO’s findings in the report and agrees the recommendations. Work has already commenced to refine DVA’s governance processes with an increased emphasis on the performance management of the VCR program and, as noted in the report, supporting this by the existing work underway to update DVA’s approach to defining and tracking reliable indicators of program performance. In addition, DVA and Services Australia have commenced the process of refreshing the Bilateral Agreement which governs the overarching ICT relationship between the two agencies. Finally, as an initial step in response to the ANAO’s first recommendation, adjustments have been made to the development of the business case for the next phase of VCR so that it addresses any potential Government exposure to risks that may arise as a result of the implementation of any of the initiatives being proposed.
The recommendations have been provided at a critical point of the VCR program and DVA will greatly benefit from the ANAO’s findings and recommendations in implementing the next phase of the VCR program. DVA will ensure its implementation plan in response to the report actively addresses these findings and recommendations. The achievement of the milestones in the implementation plan will be closely monitored and overseen by DVA’s Audit and Risk Committee, with regular progress reports to be provided to the ANAO.
We also observe that the ANAO’s recommendations and findings align with elements of the ANAO’s cross-entity audit which examined DVA’s bilateral arrangements with Services Australia. DVA will take an enterprise-wide approach to address these findings to deliver long term benefits for the veteran community.
Key messages from this audit for all Australian Government entities
Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.
Performance and impact measurement
1.1 The Department of Veterans’ Affairs (DVA) develops and administers programs to provide support to veterans and their families. The department has three outcomes: Compensation and support; Health; and Commemorations.3
1.2 DVA delivers a wide range of veteran support services, including:
- Rehabilitation — medical, mental health and return to work support;
- Treatment — health care, pharmaceutical, and counselling services;
- Payments — regular income support, compensation for injuries or disease related to service or for permanent impairment, and for economic loss due to the inability (or reduced ability) to work;
- Housing — subsidised loans and home and contents insurance;
- Transport — services and assistance with the modification of vehicles; and
- Commemoration — projects and activities that highlight the service and sacrifice of Australia’s servicemen and women in wars, conflicts and peace operations.
1.3 In 2019–20, DVA employed approximately 1780 permanent staff throughout Australia, and approximately 1200 contracted staff. Offices are located in each state and territory capital, as well as regional areas.4
1.4 Veteran clients are: serving members of the Australian Defence Force (ADF); those transitioning from the ADF; and those no longer serving. Clients also include the families of veterans.
1.5 In 2018–19, approximately 5600 ADF members transitioned to civilian life. In 2019–20, DVA supported 225,546 veterans and 103,806 dependants. DVA estimates there are about 450,000 veterans in the Australian community with which it does not have contact.
1.6 In 2017, the veteran client population was expected to decline by 14.2 per cent by 2022. However, in 2020 the population is projected to steadily increase by seven per cent by 2025. The population of younger clients aged 25 to 40 years is forecast to increase by 44 per cent from 2020 to 2025.5
1.7 Claims and services administered by DVA are underpinned by three main legislative instruments: the Military Rehabilitation and Compensation Act 2004 (MRCA); the Safety, Rehabilitation and Compensation Act (Defence-related Claims) 1988 (DRCA); and the Veterans’ Entitlement Act 1986 (VEA).6 DVA clients usually have entitlements across more than one Act, depending on types of service and the dates during which service occurred.
1.8 The number of claims and the average number of injuries per claim has trended upwards in recent years.7 While the number of veterans receiving benefits under the VEA is declining8, MRCA and DRCA claims are increasing. In 2018–19, there was an 83 per cent increase in the overall number of claims lodged under MRCA and DRCA Initial Liability and Permanent Impairment, and VEA Disability Pension and War Widows Pension, over the previous year, with a further 15 per cent increase in these claim types in 2019–20.9
1.9 The estimated value of future payments associated with claims has also risen over this period. From 2018–2019 to 2019–2020, the value of personal benefits and health care claims increased by 35 per cent and 64 per cent, respectively.10 The rate of increase has been influenced by a range of reform measures, as well as the use of computer-based decision-making enabling claims to be processed more quickly.11 The projected number of claimants under the MRCA, DRCA and VEA Acts is shown at Appendix 2.
Impetus for reform
1.10 Reviews and audits of the veterans’ support system over the past decade have recommended reforms to DVA’s provision of services, business processes and ICT systems. A 2019 Productivity Commission Inquiry into the veterans’ compensation and rehabilitation system concluded that the current system was not fit for purpose and ‘failed to focus on the lifetime wellbeing of veterans’.12
1.11 The Productivity Commission described the system as:
- legislatively and administratively complex, contributing to lengthy and inconsistent claims determinations and levels of support;
- difficult for clients to navigate because of the large number of benefits with different eligibility criteria and rates; and
- poorly administered by DVA, placing stress on claimants.
1.12 It also noted a growing dissatisfaction, particularly among its younger clients, with DVA’s impersonal, transactional and slow service.13
1.13 A timeline and information about key reports and Budget measures are provided in Figure 1.1 and Table 1.1.
Source: ANAO analysis of public information and data provided by DVA.
Capability Review — Department of Veterans’ Affairsa
The review was conducted by the Australian Public Service Commission as part of a program of reviews of Commonwealth entities. DVA’s delivery of services was described as ‘disjointed, inconsistent and slow’. The report recommended DVA implement a client-focussed delivery approach, supported by more efficient business processes and integrated ICT systems.
Senate Foreign Affairs, Defence and Trade References Committee Report — Mental health of Australian Defence Force members and veteransb
The inquiry report recommended the Government address DVA’s complex and lengthy claim processes, its lack of case coordination, and fragmented ICT systems.
DVA Functional and Efficiency Review (FER)
The review was commissioned by the Minister for Finance on 31 August 2015. Functional and Efficiency Reviews were conducted of all Commonwealth entities to determine whether their resourcing and functions aligned with government priorities, including for ‘smaller Government’. The report recommended that DVA become a small policy and program management agency, leveraging Services Australia’s ICT capabilities.
Senate Foreign Affairs, Defence and Trade References Committee report — The Constant Battle: Suicide by Veteransc
Recommendations included a request that the Government make a reference to the Productivity Commission to examine options for simplifying legislation and improving efficiency and effectiveness in service delivery. The Government agreed to this on 24 October 2017.
Productivity Commission (PC) Inquiry into Veterans’ Affairs’ Legislative Framework and Supporting Architecture for Compensation and Rehabilitation for Veteransd
The PC report (A Better Way to Support Veterans) concluded that the veterans’ compensation and rehabilitation system was not fit-for-purpose and required fundamental reform. Further areas for improvement included: communication with clients; supporting vulnerable clients during the claims process; and the quality assurance of claims processing. The Government issued an interim response to the PC report on 8 October 2020, addressing 25 of its 69 recommendations. It also stated a commitment to the VCR program ‘as the primary vehicle for reform’.
Note a: Australian Public Service Commission, Capability Review: Department of Veterans’ Affairs [Internet], APSC, Canberra, 2013, available from https://www.apsc.gov.au/capability-review-department-veterans-affairs [accessed 28 October 2020].
Note b: Senate Foreign Affairs, Defence and Trade References Committee Inquiry, Mental health of Australian Defence Force members and veterans [Internet], Australian Senate, Canberra, 2016, available from https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Foreign_Affairs_Defence_and_Trade/ADF_Mental_Health/Report [accessed 9 October 2020].
Note c: Senate Foreign Affairs, Defence and Trade References Committee Inquiry, The Constant Battle: Suicide by Veterans [Internet], Australian Senate, Canberra, 2017, available from https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Foreign_Affairs_Defence_and_Trade/VeteranSuicide/Report [accessed 9 October 2020].
Note d: Productivity Commission, A Better Way to Support Veterans No. 93 [Internet], Productivity Commission, Canberra, 2019, available from https://www.pc.gov.au/search?collection=productivity-commission-web&form=site-search&query=veteran [accessed 9 October 2020].
Source: ANAO analysis of DVA and public information.
1.14 A number of other review activities were conducted between August 2016 and May 2020, primarily focused on mental health and services relating to the transition of military personnel to civilian life. Further detail about these reviews is provided at Figure 1.1 and Appendix 3.
1.15 In 2019, 56 per cent of claimants aged 18 to 34 years of age submitted their claim online, compared with 16 per cent using paper forms.14 In 2016, before the implementation of reforms, the department had a limited ability to provide online services.15
1.16 A 2016 review of DVA’s operations concluded that its multiple claims IT systems did not support efficient claims management, record keeping, and payment monitoring. It recommended that its compensation and income support systems be redeveloped by Services Australia using whole-of-government ICT infrastructure.16 The ANAO found in 2017–18 that manual interventions and poor controls contributed to inaccuracy in claims determination processes and appeals against decisions.17
Veteran Centric Reform
1.17 DVA was provided funding of $18.7 million in the 2016–17 Budget to commence planning to significantly improve its provision of services.18 Since 2017–18, further funding measures have enabled DVA to implement changes to its policies, services and ICT infrastructure. These funding measures form part of a comprehensive set of reforms, termed the Veteran Centric Reform (VCR) program.
1.18 The VCR program is currently in its fourth year of a planned six years of implementation. A business case for funding to support a further two years of implementation is planned for the 2021–22 Budget.
1.19 Budget measures to date have provided funding totalling $653.7 million to DVA and Services Australia for the implementation of the VCR program.19 DVA has been allocated $375.5 million, with $277.8 million allocated to Services Australia to develop new ICT capabilities. Over this period, DVA is required to deliver savings of $367 million. This results in net DVA funding of $6.1 million.
1.20 The Government has allocated funding for the implementation of the VCR program through three Budget measures from 2017–18 to 2019–20. These measures are:
- 2017–18 Budget — $166.6 million over four years for the first year of VCR program implementation, under the Veteran Centric Reform Budget measure20;
- 2018–19 Budget — $111.9 million over four years to continue the VCR program under the Delivering Australia’s Digital Future — Veteran Centric Reform — Continuation Budget measure21; and
- 2019–20 Budget — continuation of the VCR program under the Veteran Centric Reform — Putting Veterans and their Families First Budget measure, funded through departmental and administered savings.22
1.21 On 6 November 2020, the Minister for Veterans’ Affairs received authority to put forward a bid for further funding in the 2021–22 Budget.
1.22 Funding provided to DVA and Services Australia through each Budget measure is shown in Figure 1.2. These are net figures which include amounts distributed over the forward estimates, and savings requirements for each Budget measure. A breakdown of DVA departmental and administered funding allocations and savings over this period is provided at Appendix 4.
Notes: Figure 1.2 excludes funding of $0.4 million over four years allocated to the Department of Finance to conduct regular assurance reviews of the VCR program.
Capital funding has been allocated to Services Australia for cyber security and ICT systems development. In 2017–18, DVA was allocated $2.4 million in capital funding to mitigate extant cyber security risks.
DVA’s total allocation of $18.6 million for 2016–17 shown in the graph differs from the total cited in paragraphs 1.17 and 2.13 due to rounding.
Source: ANAO analysis of data provided by DVA.
1.23 The overarching goal of the VCR program is to ‘improve the experience of veterans and their families by ensuring they are no longer exposed to unnecessary complexity in their dealings with DVA’, and that they have earlier and easier access to services.23 Reform is intended to enable the department to ‘better know, connect, support and respect veterans who serve or have served Australia’.24
1.24 DVA defined a veteran-centric approach as one that ensures ‘the client is at the centre of the department’s philosophy, operations and ideas’. A principle of reform is that services are designed with veterans, and that this process builds trust and confidence.
1.25 The VCR program is aimed at delivering service improvements through changes in organisational culture and capability. Key reform outcomes described in the original and subsequent business cases are shown below in Figure 1.3:
Source: ANAO analysis of data provided by DVA.
1.26 Key initiatives to achieve these outcomes under the VCR program include:
- improved access to online services (via DVA’s MyService portal) and communication with the veterans’ community via telephone and the DVA website;
- cooperation with the Department of Defence to engage military personnel separating from the ADF; and
- use of advanced data analytics to inform the development and review of policies and programs relating to veterans’ services.25
1.27 The PC Report recommended establishing a model of service provision focused on ‘wellness and ability (not illness and disability)’ over the lifetime of a veteran.26 Principles emphasising early treatment and rehabilitation underpin a number of VCR initiatives, including strengthened case management to support vulnerable clients. Further information about VCR reform measures and the status of their implementation is provided at Appendix 5.
1.28 An early intervention policy approach is also reflected in amendments to legislation made since the commencement of the VCR program. These have authorised certain determinations to be automated, payment for mental health services before a claim is determined, and requirements for claims for liability and permanent impairment to be processed within 90 days. Legislative changes associated with VCR reforms are detailed in Appendix 6.
1.29 The VCR business cases also outlined a longer-term operating model that envisaged DVA becoming a smaller entity, primarily focused on policy design and the commissioning of services, and supported by digital and data analytics capabilities.
Rationale for undertaking the audit
1.30 The provision of services to veterans and their families is a subject of interest to the Parliament and the public. The VCR program is the most comprehensive set of reforms undertaken by the department in decades. Audit assurance about VCR program implementation provides transparency about whether sufficient progress is being made against reform objectives.
Audit objective, criteria and scope
1.31 The objective of the audit was to examine the effectiveness of DVA’s planning and management of the VCR program. To form a conclusion against this objective, the following high-level criteria were adopted:
- Was DVA planning for veteran-centric reforms effective?
- Did DVA establish fit for purpose arrangements for implementing the VCR program?
- Does DVA appropriately monitor, evaluate and report on the implementation of reform measures?
1.32 As the VCR program is still in progress, this report is a point in time assessment of DVA’s implementation of the program, and does not conclude on whether the Government’s full objectives for reform are likely to be achieved.
1.33 The ANAO examined documentary evidence from DVA, Services Australia and the Department of Finance. The audit team also considered evidence from key management personnel within DVA, stakeholder entities, representatives of veterans’ organisations, and public submissions.
1.34 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of $394,145.
1.35 The team members for this audit were Judy Lachele, Dr Cristiana Linthwaite-Gibbins, Jess Kanikula, Cherise Reed and Alex Wilkinson.
This chapter examines whether the Department of Veterans’ Affairs (DVA) proposal for reform aligned with the Australian Government’s objectives for the reform of veterans’ services.
DVA’s development of the Veteran Centric Reform (VCR) proposal was partially effective in meeting the Government’s objectives. The VCR business case provided a clear rationale for investment and provided options for meeting government’s objectives. However, the department did not adequately draw the attention of decision-makers to key risks. In designing reform activities, the department engaged extensively with the veteran community and stakeholders. Its internal program planning provided a sound basis for the implementation of VCR reform activities.
Areas for improvement
The ANAO made one recommendation aimed at strengthening future business cases for VCR program funding to ensure the Government’s exposure to risk is appropriately managed. The ANAO also suggested that DVA ensure greater consistency in the VCR’s program structure to enable a clear view of progress in the implementation of work across years.
2.1 Major government funding proposals are required to be supported by a business case.27 Business cases demonstrate the need for and the benefits of investment, and provide the Government with policy and funding options for addressing its priorities. Costings are to reflect consideration of value for money and risks to the delivery of benefits over the life of the investment through proposed approaches to implementation.
2.2 Following approval of a First Pass Business Case on 27 April 2016, the Government tasked DVA and Services Australia with developing reform options for improving services, and addressing DVA’s limited ICT capability.
2.3 On 12 April 2017, the Government approved the Second Pass Business Case, establishing a high-level framework for the implementation of reform initiatives. Funding bids for the continuation of the VCR program beyond the first year were supported by two further business cases in 2018–19 and 2019–20. The process of business case development for the VCR program is shown below in Figure 2.1.
Source: ANAO analysis of Department of Finance guidance and Budget information.28
2.4 The ANAO assessed whether the VCR program businesses cases provided the Government with appropriate information to support its consideration of investment in reform, including suitable options for meeting reform objectives.
2.5 The ANAO also examined whether the department developed an appropriate high-level plan to guide implementation, and a coordinated program of work to achieve the outcomes detailed in its business cases.
Did DVA’s proposal for reform activities align with the Government’s objectives?
The 2016–17 VCR business case established the need for, and the expected benefits of reform. Consultation with the veteran community and within government informed the development of policy and funding options for the Government’s consideration. However, DVA did not provide all information relevant to the Government’s decision-making on investment in reform. It also did not adequately set out how it would manage budget and operational pressures resulting from increases in demand for services in the short to medium-term.
DVA’s business case for the Veteran Centric Reform Program
2.6 The 2016–17 First Pass Business Case set out current and emerging pressures on the department’s operations, and the risks of not addressing these. Key risks to effective and efficient operations were:
- fragmented and obsolete ICT infrastructure which did not support service delivery to a standard expected by the Government, within acceptable timeframes and costs;
- a high level of manual intervention in claims processing, resulting in errors and inconsistencies in the determination of claims, particularly those made under multiple Acts29;
- a ‘catastrophic’ risk of ICT failure which could disrupt pension payments to veterans;
- the department’s inability to meet the expectations of younger veterans for online access to services, and to respond to increasing demand for mental health services; and
- expenses and risks associated with storing and handling paper files, and maintaining extant systems.
2.7 Drawing on reviews of DVA service provision, the business case outlined key principles for the development of reform options. These included a focus on: simpler and faster veteran access to services, including through online services30; the use of a common government ICT service delivery platform; and commercial off the shelf solutions, wherever possible.31
2.8 The business case stated a commitment to cultural change in the design and delivery of services. It detailed a five month process of consultation with the veteran community in 2016 on the department’s provision of services. Around 210 current and former members of the Australian Defence Force (ADF) were consulted through workshops and interviews. DVA also engaged with specific groups of veterans to identify the diverse needs of its clients. An example of this was the establishment of a new forum called the Female Veterans and Families Forum.32
2.9 In June 2016, DVA established an Interdepartmental Committee, chaired by the department’s Chief Operating Officer, to consult with relevant entities across government on the development of the business case and design of the VCR program.33 It included representatives from: Services Australia; the Digital Transformation Agency; the National Disability Insurance Agency; and the departments of Defence, Employment, Prime Minister and Cabinet, Finance, and Treasury.
Options provided to Government
First Pass Business Case
2.10 A First Pass Business Case allows for the initial consideration of options for achieving government objectives and the costs and benefits of each.34
2.11 The 2016–17 First Pass Business Case proposed two high-level options (A and B) for further development in the 2017–18 Second Pass Business Case, as well as the option of no change (option C). Options A and B envisaged extensive ICT redevelopment, with the difference in costing reflecting a choice between comprehensive reform of DVA’s service provision (option A, put forward as preferred), or an ICT upgrade with only minimal reform of the department’s culture and business operations (option B).35
2.12 ICT modernisation was to involve the transfer of key DVA ICT functions to Services Australia. The business case outlined the advantages of the department’s existing ICT services relationship with Services Australia, and how it would leverage commonality between DVA and Services Australia’s benefit and payment functions, and its scale, capabilities and reliability.36
2.13 On 27 April 2016, the Government provided authority and $18.7 million in funding for DVA to develop a Second Pass Business Case, building on option A.37 The decision supported a shift from claims processing to delivering integrated services, tailored to the needs of veterans and their families, supported by contemporary ICT.
Second Pass Business Case
2.14 The Second Pass Business Case was first considered by the Government on 4 April 2017.38 It set out three detailed options, each underpinned by full ICT redevelopment by Services Australia. The options provided differing levels of improvement in service provision, and financial return to government in the form of savings.39 The Government requested the development of two further lower-cost options (options 4 and 5) for consideration at a later meeting. These options did not alter the proposed direction of reform.
2.15 On 12 April 2017, the Government agreed to option 5 which presented option 2 of the 2017 business case at a lower cost. This resulted in funding of $166.6 million over four years for the first year of VCR program implementation, under the Veteran Centric Reform Budget measure.
Business outcomes and benefits
2.16 To support the Minister’s preferred proposal for ICT and business modernisation, DVA identified key features of a new service approach. These aligned with key business outcomes outlined in DVA’s First Pass Business Case, and are set out below.
- Simpler and faster access — Use of whole-of-government online portals for payments and connecting clients with government and provider services. Use of automated claims determinations when set conditions are met. Self-management online, with support for those needing assistance.
- Tailored and coordinated support — A whole-of-life approach to ensuring veterans and their families could access support, when needed. Systems to enable clients with complex needs to be supported in a timely and effective way.
- Seamless transition from the Australian Defence Force (ADF) — Veteran payments and services in place before separation from the ADF. Proactive provision of services using automated registration and transfer of data from the time of ADF enlistment to separation.
- Circumstances-driven services — Consolidated client information to ensure clients are provided payments and services, when needed and without the need for a claim. Use of ADF service records and risk profiles to automate the determination of payments and services.
- Efficient partnering — An integrated framework of government and partner organisations to deliver services. Co-location of service delivery to increase effectiveness and reduce costs.
2.17 A key reform initiative was to improve the department’s ability to support vulnerable clients, through ICT and business process improvements and cultural change:
- consolidating client files into a single digital record, capturing biographical, service and medical data, all interactions with DVA, and information about eligibility and entitlements; and
- enabling staff to access relevant information about a veteran and their circumstances on a single screen, and to act as a single point of contact.40
2.18 The ability of staff to view integrated client information was implemented in June 2020. Full access to client records depends on the digitisation of existing paper records, a separate VCR initiative still in train.41 This initiative is discussed in Chapter 3 of this report (see paragraph 3.66).
2.19 The Second Pass Business Case proposed a six year reform program to be delivered in two main tranches.
- Tranche 1 (Years 1 to 3) was to focus on establishing foundational ICT42, new business processes, pilots, and changes in service delivery not dependent on significant policy and legislative reform.
- Tranche 2 (Years 4 to 6) was to complete the ICT build, deliver capabilities for client self-management, simplify access to DVA, and make use of data analysis and modelling to target services. It also flagged the potential transfer of functions to other delivery partners.
2.20 The approach was intended to provide the Government with flexibility to consider the outcome of pilots, reviews and future government decision-making on Services Australia’s WPIT program.
2.21 The Government elected to consider future funding more frequently, requesting updated business cases after the first and second year of implementation. These have outlined the scope of work that was proposed to be funded, and identified instances where work previously committed to had not been fully delivered.43
Business Case updates
Year 2 business case
2.22 On 7 March 2018, the Government considered an updated business case to support the continuation of the VCR program for a second year. The business case presented two options — to extend funding for reforms, or to maintain capabilities developed in Year 1 only.
2.23 The preferred option of continuing the VCR program proposed a ‘test and trial’ approach, emphasising pilots to inform the planning of longer-term reforms. It stated that Year 2 would enable the department to provide government decision-makers with evidence and implementation plans to support a longer-term funding commitment.
2.24 The Government elected to continue the program for one year, at a cost of $111.9 million distributed over four years, under the Delivering Australia’s Digital Future — Veteran Centric Reform — Continuation Budget measure. Savings proposed in the 2018–19 Veterans’ Affairs Portfolio Budget Submission were to be used to offset the cost of implementing Year 2.
Year 3 business case
2.25 The Revised Business Case for Year 3 proposed two further years of funding to support the continuation of reform in tranches. The four options presented differed in terms of the amount and distribution of funding over the forward estimates.44
2.26 Options 1 and 2 outlined an implementation approach focussed on income support and client management ICT capabilities in the first year, with the second year to potentially respond to recommendations of the Productivity Commission’s review of compensation and rehabilitation services.45 46 This would include market testing for appropriate ICT solutions. The department assessed that the yet to be delivered PC recommendations were likely to align with the objectives of the VCR program.
2.27 The department’s preferred option (option 1) involved the transfer of DVA staff to Services Australia for the delivery of ICT functions, offering the highest level of savings.47 Option 2 deferred a decision on this to the 2021–22 Budget and excluded some ICT functions, and the decommissioning of obsolete DVA systems from scope. This carried a risk that the overall cost of option 2 would be higher than option 1 due to the need to fund decommissioning at a later stage of implementation.
2.28 On 15 February 2019, the Government approved option 2 enabling the continuation of the VCR program for two years.48 It also requested that the Minister report back to the Government in the 2020–21 Budget with an update on savings achieved by the department.49
Benefits and risks
2.29 While the Second Pass Business Case proposed a benefits-realisation framework, benefits were not explicitly linked to VCR business outcomes. The benefits of reform were broadly defined in terms of better client experience, health and well-being, more effective departmental operations, and savings for the Government.50
2.30 Key Performance Indicators, benchmarks or targets specific to the proposed areas of service improvement were to be developed once the scope of the VCR program had been decided, however this did not occur. As at December 2020, DVA had commenced establishing measures of success, baselines and targets for tracking the benefits identified in its benefits-realisation framework for Year 4. This is discussed at paragraph 4.11.
Analysis of risk
2.31 When key risks are realised the delivery of business outcomes and benefits is likely to be compromised.51 It is therefore important the Government is provided with relevant information about key areas of risk and proposed actions for mitigating these.
2.32 The First Pass Business Case identified risks associated with each reform option. These included risks relating to resourcing, partnering arrangements for the delivery of ICT, schedule and budget, potential legislative constraints, and possible staff and stakeholder resistance to change.
- For option A, all risks were rated low to medium, except for the risk of catastrophic ICT failure before work was completed. This was rated as very high.52
- Similar risks were identified for option B, however an additional risk rated as high was included — stakeholder criticism of investment in ICT without addressing the need for better service delivery.
- Risks associated with a government decision to not pursue reform (option C) were all rated very high.
2.33 All options presented as part of the Second Pass Business Case were assessed as carrying an overall medium level of risk following mitigation. The 2019–20 updated Business Case revised VCR program risks and ratings, identifying a high risk that ICT changes would be delayed or not fit for purpose.53 DVA’s management of program risk is discussed further at paragraphs 3.25 to 3.32.
2.34 The cost of new funding proposals is generally required to be offset by savings identified within the portfolio.54 These may relate to administered or departmental funding, or both.55 Savings commitments are reflected in Budget decisions through adjustments to the entity’s appropriation.
Savings through the avoidance of costs
2.35 Proposed partnering with Services Australia for the use of whole-of-government ICT platforms and systems has been a core component of each VCR business case.56 In 2017, the department advised the Minister that an estimated 96 per cent of VCR business applications could be delivered using existing or planned standard Services Australia capabilities.57
2.36 The use of Services Australia’s capabilities was estimated to save the Government up to $500 million.58 This figure represented costs which could potentially be avoided by the Government, based on the difference between estimated ICT costs of using all WPIT components and estimates for a ‘greenfield’ DVA project.
2.37 DVA advised the ANAO in September 2020 that this level of saving is unlikely to be realised, due to greater differences between DVA and Services Australia business requirements and functions than originally identified. Efficiency savings expected from the retirement of DVA systems have also not been fully realised due to delays in implementation and the decommissioning of obsolete functions.
Departmental and administered savings
2.38 Funding for the VCR program was to be partially offset by departmental efficiency savings identified in the 2016 Functional and Efficiency Review of DVA.59 These related to the automation of certain functions, the digitisation of paper records and more efficient business processes. The bulk of Budget savings were to be realised from 2019–20 to 2022–23.60
2.39 The Second Pass Business also identified a range of savings in government administered spending through more cost-effective contracting or provider arrangements, including for the provision of health care. The profile of savings commitments is detailed at Appendix 4.
2.40 Not all the planned savings have been fully realised. DVA assesses that the majority of savings will be achieved, but not as scheduled for 2019–20. As funds intended to be saved through increased efficiency are withdrawn from an entity’s annual operating budget, shortfalls in the achievement of savings can impact on its operating position. In 2019–20, the department recorded an operating loss of $35.85 million. Significant growth in specific claims types contributed to an operating loss of $28.3 million, the major component of the overall loss.61 62
2.41 DVA has not yet reported back to the Government on its achievement of savings. DVA’s achievement of savings is discussed further at paragraphs 4.15 to 4.20.
2.42 The Government incurs liability for payments to veterans from the point at which an entitlement to rehabilitation, compensation and healthcare services is determined under the Military Rehabilitation and Compensation Act 2004 (MRCA) and/or the Safety, Rehabilitation and Compensation Act 1988 (Defence-related Claims) (DRCA). As costs are incurred into the future, liability reflects future payments over the lifetime of claimants and are a cost to future budgets.63
2.43 The Second Pass Business Case stated that investment in reforms would help to reduce the costs of care over the lifetime of veterans and their families. This was based on assumptions that early and targeted provision of services would improve veteran well-being and significantly reduce reliance on compensation payments.64 Consequently, the Government’s associated liability would decline.
2.44 The Second Pass Business Case detailed rising costs associated with rehabilitation and compensation obligations, particularly in relation to veterans of recent conflicts.65 Their more complex needs would require more time and resources in the management of cases.66 DVA assesses that the per unit cost of processing a claim under the MRCA or DRCA Acts (which generally apply to younger cohorts of veterans) is two to three times that of processing a claim under the VEA Act.
2.45 In 2015–16, the level of liability had increased by $2.5 billion over the previous year to a total of $9.9 billion. This was projected to increase to $14.4 billion in 2025. Liability costs for 2019–20 were estimated at $21.5 billion, well over the previous projection.67
2.46 Liability payments, following the introduction of proposed reforms, were estimated to reduce by $127 million over six years, with benefits evident from the first year of VCR implementation. Reduced liability was put forward as a source of administered savings which would contribute to offsetting the cost of VCR implementation. Reductions in departmental processing costs were also expected as a result of fewer claims over the life of the veteran, however the estimated level of departmental saving was not assessed.
2.47 The Second Pass Business Case stated that without investment in reform, administered and departmental spending would continue to increase. The VCR program would mitigate this growth, with reductions in expenditure evident soon after its commencement. However, actuarial modelling commissioned by the department as part of the development of the business case indicated it could take two to three years before any financial benefits were realised. Total savings of between $500 million and $1 billion were estimated to be realised over a period of 50 years.
2.48 DVA advised the ANAO that savings relating to the payment of liability are likely to be realised later than outlined in the Second Pass Business Case. This is due to a number of factors, including:
- significant growth in the number and size of claims lodged with DVA since the commencement of the VCR program;
- a greater number of claims being positively determined68; and
- the use of automated decision-making which has increased the rate at which certain types of claims are processed.69
2.49 DVA is expected to deliver legislated entitlements and benefits to veterans quickly through efficient and effective claims handling. However, DVA’s backlog of claims has grown, with a majority of targets relating to claims processing in 2019–20 not met, despite the increased use of automatic determinations.70 The department has required financial supplementation for claims processing each financial year following the approval of the Second Pass Business Case.71 This correlates with the upward trend in the number of claims lodged, but may also reflect delays in establishing a more efficient claims processing function.
2.50 The department did not establish the full base of evidence needed to support informed decision-making by the Government on investment in reforms. This would have required the modelling of likely increases expected in the volume and value of accepted claims, and the rate at which determinations would be made, against an estimation of the time needed for the benefits of reform to be realised.
2.51 The business case should also have assessed the likely short to medium increase in claims processing workload for the department and put forward clear proposals setting out options for managing this.
2.52 The department did not clearly set out how its operations would be impacted by a strong policy emphasis on proactive and early intervention, facilitated through easier and faster means of lodging and processing claims. Future funding submissions should provide options for managing the volume and type of claims and associated liability, and other risks, arising from changed settings.
2.53 The Department of Veterans’ Affairs ensure that future business cases for investment in reform provide clear and viable options for managing the Government’s exposure to risks arising from new policy and program settings.
Department of Veterans’ Affairs response: Agreed.
2.54 DVA is building on its existing risk management processes so that current and future business case investment proposals consider the full impacts of the VCR program including on Government outlays. DVA is actively working with the Department of Finance to ascertain the best method to express known risks and provide viable options for managing Government exposure arising from new policy and program settings, particularly for current investment proposals. DVA is addressing this recommendation for the business case in the current Budget cycle, and embedding into future DVA practice and will report quarterly on progress.
Has DVA’s implementation planning established a sound basis for achieving the Government’s reform objectives?
DVA has undertaken implementation planning to give effect to VCR Budget decisions. Planning processes have maintained a focus on broad outcomes across years, and supported a coordinated approach to program management. However, changes in the organisation of delivery streams and projects have reduced the ability to track the evolution of work across years, and account for deliverables.
2.55 DVA has planned and implemented longer-term reform measures iteratively. The first two years of VCR program implementation set out initiatives aligned with the outcomes from the original business case.72 In addition, for the first and second years of the VCR program, planning emphasised the use of pilots to inform longer-term development of services.73
2.56 The major delivery work streams and their objectives established by the Second Pass Business Case are shown in Table 2.1.
Work streams and objectives
Students and Income Support (S&IS) — MyService and student payments
Online claims for income support payment, either through self-service or assisted by DVA staff. Online registration and claims for student education allowances.
Digital Client Experiencea
Streamline the claims process by reducing the time needed to complete and lodge a claim. Automated (‘straight-through processing’) of claims to assess whether a veteran satisfies requirements (against specified Statements of Principles) for certain medical conditions.
Channel Transformation and Client Strategy
Improve online and face to face channels. Consolidate phone numbers to a single 1800VETERAN line.
Better manage veteran welfare and government liability over the long-term, through early and targeted services. Use data analysis to inform policies and evaluate their impact on liability.
Reviews to investigate the scope for simplified, equitable and accessible transport assistance, and for increased choice and online visibility of rehabilitation aids and appliances.
Note a: As at November 2020, 29 per cent of all Initial Liability claims via MyService have been ‘straight through processing claims’ or computer-based decision claims.
Note b: Longitudinal data sets are to be used to identify veterans most likely to benefit from targeted strategies, and to enable policies to be evaluated and adjusted. In 2019, DVA held 133 million records relating to client interactions with the department in relation to claims under MRCA, DRCA and the Veteran Entitlements Act 1986 (VEA).
Source: ANAO analysis of data provided by DVA.
2.57 High-level planning and program implementation processes have been managed by an Enterprise Program Office (EPO).74 Planning for the establishment of the EPO outlined its intended functions, including:
- establishing the VCR program, and managing resourcing and funding for projects;
- identifying and managing program level risks, issues dependencies, quality and the realisation of benefits;
- management of a ‘master schedule’, milestones and integration with Services Australia; and
- the management of suppliers, including contract management.
2.58 The EPO does not have a mandate to performance manage the delivery of projects or initiate corrective action. It does not have a role in monitoring the achievement of savings commitments.75
2.59 Implementation of the VCR program has been supported by:
- a change management unit to support business areas and staff to implement organisational change;
- a business process improvement unit with a focus on identifying opportunities to increase DVA’s operating efficiency76; and
- initiatives to conduct workforce planning and develop ‘a DVA operating model’.
2.60 DVA retained its framework of major work streams for Years 1 and 2. Following this, elements of work streams were re-organised in Year 3 into a new framework of five ‘focus areas’. The Year 4 framework comprises 12 delivery streams. Changes to the organisation of work in Years 3 and 4 reflect the department’s use of a different project methodology. This is discussed in Chapter 3 of this report (see paragraphs 3.44–3.50). A diagram showing changes to work streams across Years 1 to 4 is provided below in Figure 2.2.
Note: Arrows indicate the continuation of elements of delivery streams across years.
Source: ANAO analysis of data provided by DVA.
2.61 DVA’s reorganisation of work streams in Years 3 and 4 reduces its ability to demonstrate coherence and continuity in program planning. Without consistent reference points, it is difficult to track the evolution of the program and to determine whether sufficient progress in delivering outcomes is being made, when assessed against time and funds expended.
Planning and coordination
2.62 Following Budget decisions, DVA has undertaken detailed planning to determine the scope of each work stream for the year, and to identify component projects and key milestones. The EPO has supported a co-ordinated approach to program planning. At the beginning of each year, DVA has undertaken prioritisation processes to determine the projects (or work packages) that can be supported within the funding allocated. In 2017–18, the VCR work streams encompassed 67 projects. This increased to 91 in 2018–19 and 88 in 2019–20. There are 116 projects or work packages scheduled for 2020–21. Completion of work under the program is discussed in the Implementation chapter of this report (see paragraphs 3.51–3.66).
2.63 A yearly ‘dossier’ provides an overview of deliverables within the scope of the VCR program and contains information about outputs, assumptions and risks. In Years 3 and 4, work stream activity has been planned in quarterly ‘increments’. Program schedules set out at a high-level the program’s key deliverables and milestones and dependencies.
2.64 Iterative funding decisions have created opportunities for review and adjustment. DVA has identified risks associated with the year by year approach to funding and implementing reforms, including:
- a requirement to conduct detailed planning of work following government funding decisions, while also completing the previous year’s scope of work;
- a focus on the rapid implementation of ‘demonstration’ projects, and insufficient focus on the completion of longer-term work; and
- a need to ensure staff throughout the department are sufficiently aware of, and engaged with reform activity, and committed to the timely delivery of reform elements.
2.65 DVA’s approach to managing these risks, including through oversight arrangements established for the VCR, is discussed in Chapter 3 of this report (see paragraphs 3.25–3.32).
2.66 DVA’s planning has taken appropriate account of the Government’s policy priorities. It has adjusted reform activities to support the Government’s consideration of the Productivity Commission’s report on Australia’s military rehabilitation and compensation system. In April 2020, the COVID-19 pandemic prompted the redeployment of DVA and Services Australia staff to response teams. The consequent reduction in project resources has impacted the delivery of some components of the VCR program for 2019–20. This is discussed further at paragraphs 3.58 and 3.66.
Use of contracted resources
2.67 Since August 2017, DVA has procured services to assist with the design and implementation of the VCR program. Following the evaluation of quotes sought from a panel managed by the Department of Finance, PriceWaterhouseCoopers (Australia) (PwC) was issued a contract valued at $8,048 million from 8 August 2016 to 18 August 2017 to act as the primary strategic partner for DVA in developing the VCR Second Pass business case.77
2.68 On 9 October 2017, DVA approached eight suppliers from a panel managed by the Department of Home Affairs with a request for proposal (RFP) for consultancy services.78 79 These services were to assist the department to design, establish and provide ongoing management of specific organisational units to support implementation of the VCR program.80 Following the evaluation of proposals received, a Deed of Standing Offer was entered into with PwC. KPMG was also engaged through this RFP process.
2.69 Contracts with a total upper limit of $73 million have been issued to PwC under the original arrangement and 12 extensions (as at November 2020, $58.5 million had been expended).81 Contracts totalling $7.3 million have been awarded over the same period to KPMG for VCR-related services, including for assurance activities (as at November 2020, $7.1 million had been expended). Additional contracts of lesser value have been issued to other providers. DVA advised the ANAO in February 2021 that approximately $58 million was expended on these contracts between 2017–18 and 2019–20.
2.70 Additional costs to the program include the salaries and on costs of personnel allocated to implement the VCR program. The VCR Business Case did not include departmental funding for staff. In 2019, around half of the staff resources needed to implement the VCR program in 2019–20 were proposed to be contracted. DVA staff transferred to reform-related roles were to be backfilled by contracted staff. Contracted staff were estimated to represent 89 per cent of the total VCR staff complement in 2019–20, and 77 per cent in 2020–21.
This chapter examines whether Department of Veterans’ Affairs (DVA) is appropriately managing implementation processes to achieve the objectives of the Veteran Centric Reform (VCR) program.
DVA has established largely appropriate arrangements for program governance and implementation. The department has delivered a number of reform elements, and has maintained a focus on reform objectives. However, progress in implementing a number of business capabilities has been slower than expected. A greater strategic focus is needed to ensure root causes of underperformance at the program level are addressed. Partnering with other entities in the delivery of reform is generally effective, with scope for improvement. DVA staff are appropriately engaged in progressing the objectives of reforms and managing the impacts of change.
Areas for improvement
The ANAO made two recommendations aimed at strengthening governance processes for addressing root causes of program underperformance and establishing risk appetite statements for major ICT development projects. The ANAO also made suggestions in relation to risk and assurance processes, committee arrangements, and the use of project evaluations.
3.1 Achieving the objectives of reform on time, on budget and to expectations requires entities to manage key implementation processes. Clear frameworks for program delivery, consistent quality control and effective relationships provide the foundation for meeting commitments made to government.
3.2 The ANAO examined whether the implementation of the VCR program is supported by:
- fit for purpose governance arrangements;
- an appropriate project management approach; and
- effective relationships with key partner agencies and appropriate engagement with staff.
3.3 As implementation of the VCR program is not complete, ANAO has not assessed DVA’s effectiveness in delivering all outputs and outcomes detailed in VCR business cases, which envisaged six years for the full achievement of the Government’s objectives. This audit report mainly addresses opportunities for DVA to improve its arrangements and processes for the delivery of reforms.
Has DVA established fit for purpose governance arrangements for the implementation of the reform program?
DVA’s governance structures and processes for overseeing the management of the VCR program are largely effective. DVA has maintained a focus on reform and cultural change and established accountability for delivery. Risk processes are generally appropriate, however clearer guidance is needed on conducting assurance activities at project level.
3.4 Effective governance ensures direction and control are exercised over the implementation of program activities, so that objectives are achieved effectively and efficiently, while taking appropriate account of risk.
3.5 In order to assess whether DVA’s governance arrangements for the implementation of the VCR program are fit for purpose, the ANAO examined whether the department has established:
- accountability for implementation;
- effective high-level decision-making processes; and
- appropriate risk and assurance processes.
Accountability for the implementation of reform
3.6 Effective and sustained organisational improvement requires leadership at all levels. Gateway Reviews of the VCR program have commented positively on DVA’s executive team which has been seen as setting the tone for reform implementation and organisational culture change.82 Veteran representatives interviewed by the ANAO described improvements in how DVA staff interact with clients (see Appendix 7).
3.7 The first year of the program was delivered through centralised arrangements. Primary responsibility for implementation was assigned to business areas in the following year to promote greater engagement with the reforms across the organisation.
3.8 In subsequent years, individual responsibility at senior levels for achieving VCR program objectives has been defined through the use of a Senior Responsible Officer (SRO) model.83 A SRO at the Senior Executive Service (SES) Band 3 level has responsibility for the VCR Program as a whole.84
3.9 In August 2020, DVA’s Enterprise Program Office (EPO) engaged with SROs to identify the specific benefits of VCR reform and strengthen accountability for monitoring their realisation. Accountability for VCR outcomes is now shared between senior officers responsible for reform activity and business area owners.
3.10 Senior roles with responsibilities for VCR program elements are:
- Work stream SROs at the SES Band 2 level with overall responsibility for delivering a work stream and authority to make specific decisions;
- Transformation Owners at the SES Band 1 level who provide leadership to delivery teams; and
- Business Outcome Owners at the SES Band 1 level who are accountable for adopting change and ensuring benefits are realised.
3.11 Below SES level, Transformation Delivery Leads are responsible for the successful delivery of project milestones within set timeframes.
3.12 A hand-over checklist is used to transfer a VCR project to business as usual operations following its completion.85 This requires the project manager to provide evidence of SRO and business owner acceptance of the transfer of the project and their commitment to realising benefits.86
High-level decision-making processes
3.13 Executive committees provide advice and assurance to the Accountable Authority of an entity in relation to the proper use of resources in delivering outcomes. Oversight of the VCR program occurs through the following arrangements, and as shown in Figure 3.1 below:
- DVA Executive Management Board (EMB), the department’s most senior executive body;
- Portfolio Investment and Implementation Committee (PIIC), which oversees all major investment projects, including the VCR program and Budget proposals87;
- Transformation Program Board (TPB) — the main committee responsible for overseeing the implementation of the VCR program; and
- a project-level governance board established to oversee the Students and Income Support stream of work.88
Source: ANAO analysis of data provided by DVA.
3.14 In 2017 and 2018, the department provided briefings on the VCR program to a Ministerial Oversight Committee comprising the Minister for Veterans’ Affairs, the Minister for Government Services and the Assistant Minister for Cities and Digital Transformation. Future meetings were to be scheduled in accordance with major deliverables. The occurrence of these meetings has lapsed.
Transformation Program Board
3.15 The TPB has primary responsibility for ‘ensuring the successful implementation of the DVA VCR Program’.89 It was established in June 2017 to:
- oversee the realisation of benefits and dependencies of projects across the VCR program;
- act as the key forum for ensuring the integrity of design for the transformation plan throughout implementation; and
- be responsible for approval of change requests with impact on quality of program outcomes.
3.16 In 2019, its mandate was updated to allow for the consideration of non-VCR projects that may have a direct or indirect impact on deliverables or outcomes of the program.
3.17 At inception, the TPB was chaired by a SES Band 2.90 From late 2018, the Board was chaired at the SES Band 3 level.91 DVA and external entities are represented at the Band 1 and 2 level. DVA members include business areas responsible for reforms, with external membership reflecting the involvement of other entities in the VCR program. External entities are Services Australia, the Digital Transformation Agency, and the departments of Defence and Health. An external adviser was appointed as a member in October 2017.92
3.18 To oversee the performance of the VCR program, TPB members require regular and appropriate reporting.93 The Board has received monthly updates on the implementation of the VCR program.94 These provide information about:
- program planning, including schedules and interdependencies;
- the status and performance rating of work streams, as well as of VCR program overall, using a ‘traffic light’ framework95;
- budget allocations and overviews of expenditure against forecast budgets;
- program and project risks and issues;
- change management processes, (for example, transition to business activities and impacts on staff and the department); and
- specific issues briefs.
3.19 Year 3 and 4 monthly reports contain information limited to those work streams being actively progressed within a three month period, referred to as a Program Increment.
3.20 In December 2017, a joint project board was established at senior executive level for the Student and Income Support ICT work stream. Members are drawn from DVA and Services Australia, with DVA providing the Chair. The project board’s role is to advise the TPB and to ensure business requirements, ICT planning and resourcing are aligned.
DVA higher-level committees and reporting to the Minister
3.21 The PIIC was formed in June 2019 to support DVA’s highest-level committee, the EMB.96 Its role is to support the department’s management of all major programs, including the VCR and Budget priorities. It decides on the priority of VCR work elements, taking into account the department’s available funding.
3.22 While the PIIC’s terms of reference also require it to monitor program outcomes and benefits, this has not occurred in relation to the VCR until recently. In February 2020, the PIIC agreed on the need for better project finance and benefits tracking, including for the VCR, and for this to be a standing item. The EMB has received briefings on the status of the VCR program.
3.23 Meeting agendas indicate that the Secretary and other departmental officers have verbally briefed the Minister for Veterans’ Affairs on the VCR and given presentations on program achievements. There are no minutes of these meetings. DVA has not provided regular written briefs on implementation delays and risks.
Risk and assurance processes
3.24 In addition to its program management role, the EPO is responsible for promoting good governance and standard approaches to implementation. Each year of the VCR program, it has issued a guide which provides information about VCR program roles and responsibilities, mandatory risk and assurance activities, and project planning and management requirements. The extent to which the EPO monitors the conformance of project areas with requirements of the guide is unclear, as documents are not filed in accordance with departmental guidance, and are not checked for completeness.97
3.25 Under the Public Governance, Performance and Accountability Act 2013 (PGPA Act), entities are required to establish and maintain systems and internal controls for the oversight and management of risk.98
3.26 The VCR program risk register for 2020 identifies 14 strategic risks relating to: on-going support for reform; implementation resourcing; unauthorised changes in scope; the transitioning of projects to business owners; availability of data; and project management capability. The EPO issues guidance to assist project areas to implement risk management and reporting.99
3.27 Mandated procedures for managing program risks include:
- a requirement for project managers to maintain a Project Management Log to record emerging risks and actions taken to treat these; and
- the use of escalation mechanisms to allow project-level risks to be elevated for higher-level resolution. All risks with a rating of high or above are to be considered by the TPB.
3.28 Project risks and controls have been identified for each work stream and project. TPB papers and minutes indicate it is provided with detailed information about point in time or on-going risks associated with the implementation of specific projects.100 However, project managers could also identify risks common to a number of projects (for example, a lack of skilled project staff). This may indicate a root cause of underperformance and point to the need for a program level solution requiring consideration at a higher level.
Shared risk management
3.29 Under the PGPA Act, entities are required to put in place arrangements for understanding and managing shared risks.101 The VCR risk register reflects the joint nature of the program and DVA’s and Services Australia’s risk perspectives.
3.30 DVA and Services Australia share responsibility for managing the risk that the VCR program ‘will be unsuccessful in the realising benefits (identified in the business case) as they relate to government, clients and the organisation in the lifetime of the program’.102 Further shared risks include:
- loss of stakeholder support;
- unforeseen project delays or complexities;
- failure to adhere to a client-centric approach; and
- service delivery partners not prioritising or sufficiently resourcing VCR commitments.103
3.31 Shared risks do not include the potential failure of DVA’s ICT systems before the completion of new systems, despite this being identified as a very high risk in the Second Pass VCR business case.
3.32 DVA and Services Australia have joint responsibility for the implementation of controls for a number of shared risks. Where responsibility for managing a risk is shared between entities and/or individuals, control measures should be assigned to specified individuals to avoid ambiguity and maintain clear accountability for risk mitigation.
3.33 The role of assurance is to provide decision-makers with information about the quality of program and project execution. DVA assurance activities have involved a range of processes.
3.34 From 2017 to 2019, DVA had a Quality Management Plan in place setting out internal and external assurance processes.104 When evidence of implementation of the plan was requested by the ANAO, DVA advised that the EPO works with project managers to support best practice project management, as needed. The current assurance policy and processes for the VCR program are not documented.
3.35 DVA advised the ANAO that during the early stages of the VCR program the EPO held twice weekly meetings with delivery areas to review the progress of projects against schedule and to address problems in implementation.
3.36 Between October 2018 and January 2019, the EPO conducted a detailed ‘Health Check’ of VCR work streams and projects. The report provided information about the status and performance of each project, the effectiveness of the first year of program implementation and risks for the next year, and conformance with governance requirements.105 There is no evidence the report was provided to the TPB, or subsequently used by the EPO to improve program and project management.
3.37 Health checks were not conducted for Year 3 of the VCR program. Instead, the TPB requested that ‘return to green plans’106 be provided for projects tracking red for two consecutive months. Project areas have not developed these plans, but there has been ad hoc reporting to the TPB on actions to address red ratings. There would be value in DVA using proactive assurance processes to identify the need for early action before projects experience significant difficulties.
External assurance — Gateway Reviews and Wave reporting
3.38 The VCR program has been subject to the Department of Finance’s Gateway Review process.107 This process provides assurance over complex Australian Government programs and projects.108 Reviews of the VCR program have been conducted each year, commencing 2017.
3.39 Each report provides a point in time assessment of the likelihood that the program will be delivered in line with the Government’s expectations of schedule, cost, quality and benefits. Traffic light ratings indicate the urgency with which identified issues should be addressed. The program was rated green/amber in 2017109, green in 2018110, and green/amber in 2019 and 2020. As part of the Gateway process, DVA has reported on actions taken to implement recommendations.111
3.40 The Digital Transformation Agency (DTA) collects data as part of its monitoring of significant ICT and digital investments. Quarterly reporting by entities is mandatory for ICT projects valued over $10 million. DVA reports provide a high-level view about the extent to which expected benefits are being delivered to veteran clients, stakeholders and staff.
3.41 DVA has drawn on Gateway Review assessments to rate the prospects of successful delivery as medium/high, defined as ‘probable, with constant attention needed to ensure risks did not become major issues threatening delivery’. High-level project status information has been provided to DTA about delays in the implementation of major initiatives, such as the income support payment system.
Is DVA’s project management approach effective in ensuring delivery against the Government’s objectives?
DVA’s implementation of reform elements has been largely effective. It has delivered several reform initiatives since the commencement of the program. However, a number of work streams have fallen short of their expected performance. In managing the implementation of projects, greater attention to completing projects within originally planned schedules is required. Arrangements for the high-level monitoring of individual projects are largely appropriate, however DVA has not undertaken sufficient analysis to identify and address root causes of risks and underperformance at the program level. Decision-makers should be more effectively supported to address these.
3.42 Reform programs made up of many individual projects require effective project management to ensure work is completed to expectation. To assess whether DVA has established an effective project management approach to implementing the VCR program, the ANAO assessed whether the department:
- uses fit for purpose project management processes to implement the VCR program; and
- is delivering the results expected at the current stage of program implementation.
3.43 The ANAO also examined possible reasons for any instances of program underperformance. As most VCR work streams are still being implemented, the audit presents a point in time view.
Project management processes
3.44 In the first quarter of each year of the VCR program, the EPO has produced a yearly overview (termed ‘dossier’) of planned VCR projects. This provides high-level information about the scope and intended outcomes of projects; risks; dependencies; key stakeholders and team structure. Endorsement of the dossier by the TPB allows projects to be initiated. Detailed project management plans are then required to be developed for all work streams.112
3.45 DVA governance policy prescribes the use of project management frameworks. DVA has used two project management frameworks to support the delivery of the VCR program.113
- Years 1 and 2 — the waterfall methodology which requires end deliverables to be determined during project planning. Projects are controlled through set timeframes, milestones and resources.114
- Years 3 and 4 — the agile methodology which involves iterative planning and implementation, allowing for frequent adjustment.115
3.46 Detailed project planning has not always occurred in a sufficiently timely manner.116 The Year 2 joint DVA-Services Australia project management plan and baseline for the Students and Income Support work stream were not in place for several months after the commencement of the year. While there is sometimes a lag between when a project commences and the project plan is formally endorsed, this did not occur until the third quarter of the year of intended delivery (March 2019).117
3.47 For periods of time, project plans have also been absent for a website project; an operational improvement work stream; the development of DVA’s operating model and workforce planning projects; and the data and analytics work stream. The Year 2 Health Check was unable to determine from project documentation whether a significant number of Year 1 outputs had been delivered when intended or removed from scope. The absence of project baselines reduces an entity’s ability to track changes to scope, schedule and the use of resources, and to make controlled adjustments.
3.48 Use of the agile method from Year 3 has aligned DVA more closely with Services Australia’s preferred approach to ICT development. This involves planning iteratively within three to 12 month windows, enabling the project manager to make adjustments in response to unexpected constraints and other factors.
3.49 In 2019, DVA established tolerances for project parameters to make changes transparent. When tolerances are exceeded, and these are likely to have a high impact on the program’s performance, project managers are required to submit a Project Change Request (PCR) to the TPB for approval.118
3.50 The PCR process is an effective mechanism for controlling variations to project planning and understanding the consequences of changes for project outputs and outcomes, or for other business objectives. However, project tolerances can only be triggered if a project plan and baseline are in place. As the agile method involves adjusting baselines in line with emerging priorities within three month periods, there is a risk that the line of sight to approved project objectives may become obscured over time.
Performance against program objectives
Reform initiatives delivered to date
3.51 DVA has implemented initiatives under the VCR program to date which align with the high-level reform objectives and broad business outcomes outlined in the VCR business cases.119 These initiatives are shown at Figure 3.2 below. The timeframes for implementation indicated in Figure 3.2 reflect actual, rather than planned dates for implementation. Paragraphs 3.56–3.66 and Appendix 5 provide further information about the implementation of key initiatives.
Note: Information about the above initiatives is also provided in Table 2.1 and Appendix 5.
Source: ANAO analysis of data provided by DVA.
Performance against expectations
3.52 DVA has undertaken annual stocktakes of work completed under the VCR program. These identify projects or specific deliverables within projects (‘scope items’) and assess whether these have been delivered, not delivered, or delayed. Stocktakes record over 240 deliverables between Years 1 to 3 of VCR program implementation.
3.53 In Year 1 (2017–18), approximately 20 per cent of projects/scope items were not delivered, which increased to around 35 per cent in Year 2 (2018–19) and 45 per cent in Year 3 (2019–20).120 At the end of Year 2, the TPB was advised that several projects/scope items had not yet achieved their objectives. While this work was planned to continue, funding for Year 3 was not allocated. The delivery of work in each year is shown in Figure 3.3 below.
Source: ANAO analysis of data provided by DVA.
3.54 The ANAO analysed DVA’s monthly reporting to the TPB to determine trends in program and project performance and possible reasons for underperformance.121
3.55 DVA uses a ‘traffic light’ reporting framework to report the status of the VCR program each month to the TPB. The performance rating of the program depends on the rating of each major work stream.122 This, in turn, reflects the performance status of the component projects within a work stream. Traffic light ratings for the VCR program from its beginning to October 2020 are shown below in Figure 3.4. Amber and red ratings indicate tolerances have been exceeded.
Source: ANAO analysis of information provided by DVA.
Green rated work streams
3.56 Updates on the status of individual work streams provided to the TPB in Year 1 indicate around 85 per cent of these were rated green. Reporting to the TPB does not include ratings for each work stream each month across all years. The number of work stream ratings provided has varied from two to 22.123
3.57 The following six delivery work streams were consistently rated green in TPB updates between Years 1 and 4: Digital Client Experience (22 ratings); Cyber Security (five ratings); Provider Ecosystems (five ratings); Strengthening the relationship with members from enlistment to transition (three ratings); Build online messaging for clients (two); and Client Experience (CX) Standards (two). As of October 2020, the Single View of Client work stream had been rated green in 13 of 14 reports.
3.58 Overall, the ratings profile for the program over three years shows a trend from green to red. Program amber ratings over a total period of 19 months are attributable to 58 instances when work streams were rated amber. There were also 17 red ratings in this period.124
3.59 Nine months of red ratings for the VCR program in Year 3 are due to approximately two thirds of the work streams for that period having been rated red or amber. Of 78 ratings reported, 33 work streams were rated red and 20 were rated amber. This trend has continued into the fourth year.125 DVA attributes recent adverse ratings, in part, to impacts of the COVID-19 pandemic on the department’s operations (see paragraphs 2.66 and 3.66).126
3.60 The main drivers of amber and red monthly VCR program ratings from Years 1 to 4, as of October 2020, have been projects associated with the following work streams127:
- Students and Income Support — 19 amber and 16 red (90 per cent);
- Proactive Interventions/Data and Analytics — 12 amber and 15 red (69 per cent) 128;
- Channel Transformation — nine amber and two red (61 per cent); and
- Digitisation work streams — nine amber and four red (54 per cent).129
3.61 These work streams are discussed below, with further detail provided at Appendix 5.
3.62 Planning for the release of Students and Income Support (S&IS) work stream envisaged a ‘Go Live decision’ in June 2018.130 However, there has been slippage against the implementation schedule due to delays in allocating staff and technical risks. In 2020, the release was deferred three times, before taking place on 21 November 2020.131 As of December 2020, the S&IS work stream was rated amber.132
3.63 In 2017–18, funding of $31.7 million was provided over four years for the Proactive Interventions measure under the VCR program to improve the health of specific groups of veterans, reducing reliance on benefits. Full implementation was delayed due to difficulties in accessing specialised personnel and infrastructure. The team commenced in February 2019, with the program transitioning to business as usual, as part of DVA’s Data and Insights Branch in July 2020. Actuarial data to support the management of liability was expected to be available in July 2019, but is now expected to be delivered in late 2020–21.
3.64 There have been delays of 12 to 18 months in providing data deliverables under the VCR program. From 2017 to the end of 2018, the Data and Analytics work stream was mainly rated amber due to delays in accessing data sets, the diversion of resources to other tasks, and funding allocations. It has been rated red since March 2020.
3.65 Measures to make it easier for clients to communicate with the department under the Channel Transformation work stream experienced a number of milestone slippages. In Years 3 and 4, telephony, client hub, and face to face projects have been reported as being on track.
3.66 As at 30 September 2020, the Digitisation work stream had managed the conversion of client records equal to 80 million digital pages. The primary reason cited by DVA for adverse ratings in Year 2 was insufficient funding to meet set targets and savings. The department advised the ANAO in February 2021 that the original funding decision was based on incorrect planning assumptions. In 2019–20, changes in archiving policy and COVID-19 pandemic restrictions contributed to longer digitisation processing times and slippage against targets.133 DVA expects the savings to be achieved over a longer timeframe.134
Reasons for adverse ratings
3.67 The ANAO reviewed relevant Project Change Requests (PCRs) and monthly reporting to the TPB to identify the most common reasons, set out below, for adverse ratings across the program135:
- Delays due to a need to clarify scope and/or obtain ministerial approvals.136 The requirement for sign-off on scope changes and cyber risks contributed to adverse ratings in 2020;
- Delays in delivering ICT solutions due to a high level of defect and/or non-acceptance of solutions. This has delayed the release of functions, adding to costs due to the need for project staff to be retained and for DVA staff to use manual processes;
- Technical and other factors affecting data projects. Delays have occurred due to unexpected difficulties in transferring data from an extant to a new system, or lack of DVA legal access to data held by another entity137;
- Delays in the release of funds and allocation of project staff because of delayed forward program planning and prioritisation processes. Insufficient staffing in 2020 is partly due to the redeployment of some DVA and Services Australia staff to the COVID-19 pandemic response; and
- Lack of appropriately skilled project staff. This has been a cause of delay across all years.138
3.68 A PCR is intended to address poor performance against threshold criteria, and provides a mechanism for a new project baseline and performance parameters to be approved. However, the use of PCRs has not led to sustained improvement in work stream ratings. Generally, ratings have deteriorated within two months due to the persistence of specific issues, or because other project elements have encountered difficulty.
3.69 With the exception of the Year 2 Health Check, the EPO has not conducted a review of PCRs or other documentation to identify whether issues affecting the performance of projects have been correctly identified, or there are other underlying causes.139
Role of the Transformation Program Board
3.70 Briefing provided to members of the TPB should support members to fulfil their responsibility to ensure the success of VCR program implementation.140 Over time, reporting to the TPB has expanded to include a significant amount of technical detail relating to project activities. Elements within projects are given a traffic light rating, however explanations do not always provide a clear reason for the rating.
3.71 Excessive detail can have the effect of obscuring important matters or shifting the focus of the Board to operational matters. The TPB should encourage reporting appropriate to its mandate of ensuring the success of the overall program. This would reduce the volume of briefing presented each month, and allow members to focus on information relevant to the Board’s authority, including strategic risks and the analysis and removal of systemic impediments to project progress.
3.72 There would be merit in DVA reviewing the TPB’s role, relationship and communication with other higher-level committees, as well as the support provided by the EPO, to ensure all elements of the governance system are working together effectively. Periodic self-assessment of the Board’s functioning against its terms of reference would assist in maintaining its strategic role and value in relation to the performance of the reforms.
3.73 DVA guidance states project managers should complete project closure and evaluation reports. However, the department advised the ANAO that these are no longer required for the VCR program. Instead, two processes are used.
- A project closure checklist requires project managers to file documents relevant to transferring the project to a business as usual status, (including assets, risk registers, change management documents and lessons learnt). However, the checklist does not require evidence that the objectives and outputs of the project have been delivered.
- An annual program summary lists projects that have been completed and handed over to business areas, or transitioned to the next year.141 Around 70 per cent of projects in Year 1 were transitioned to a business as usual status, followed by 14 per cent in Year 2, and 38 per cent in Year 3.
3.74 In the absence of project evaluation processes, project closure reports could be used to inform a broader understanding of issues affecting performance. There would be benefit in formally capturing this information across the program. While the department and Services Australia have investigated specific underperforming projects, there would be benefit in conducting regular program assessments to identify and address root causes of underperformance. These assessments could be provided to the TPB to gain advice and agreement to strategies for remediation.
3.75 The Department of Veterans’ Affairs strengthen governance processes to include regular analyses aimed at identifying and addressing causes of program and project underperformance.
Department of Veterans’ Affairs response: Agreed.
3.76 DVA has already begun strengthening VCR governance processes to take a holistic enterprise approach aimed at identifying and addressing root causes of program and project underperformance. Initial changes include revising Terms of Reference for Project and Program Boards with an increased emphasis on program/project performance management. To support boards in this task the Enterprise Program Office will take on a more proactive role to improve its assurance of project and program performance. The development of an implementation plan in response to the ANAO Report will complement these actions. This initial tranche of reforms to DVA’s VCR governance processes is planned to be in place prior to the commencement of FY21/22. Further adjustments to DVA’s program governance practices will be prioritised and implemented over the next twelve months. Progress will be reported to the ANAO quarterly.
Is DVA managing relationships with key partner agencies and its staff effectively?
DVA’s management of partnering arrangements is largely effective. There is scope for DVA to further strengthen bilateral processes with Services Australia to increase value for money and manage risk in the delivery of ICT projects. DVA’s cooperation with the Department of Defence is largely effective, but requires clear and consistent measures for monitoring progress across all bilateral VCR initiatives. DVA has appropriately managed organisational changes and their impacts on its workforce. Its engagement with staff on the purpose and value of reform has been largely effective.
3.77 Accountable authorities are obliged under section 17 of the PGPA Act to encourage cooperation with others to achieve common objectives.142 To assess whether the DVA is effective in managing relationships with partner agencies and staff, the ANAO examined its strategies for:
- partnering with key partners to manage risk and to ensure reforms meet the Government’s expectations; and
- engaging with staff to ensure reforms are fully embedded.
DVA’s engagement with key partner agencies to achieve VCR objectives and value for money
3.78 Several major VCR projects are jointly designed and implemented with Services Australia. The ANAO examined the effectiveness of DVA’s partnering arrangements for the delivery of ICT capability.
3.79 Services Australia has been directly appropriated funding for ICT development to act as DVA’s sole provider of services. As DVA is not required to source commercial services, the Commonwealth procurement rules and contract management framework do not apply. However, in receiving ICT services and products, DVA is accountable for determining whether these represent value for money.143 While Services Australia is not subject to contractual terms, it is required to meet agreed performance standard at an appropriate price.144
3.80 The importance of effective arrangements between DVA, as a commissioning entity, and Services Australia, as its delivery partner, has been recognised since 2010, when Services Australia became its provider of ICT infrastructure under a Memorandum of Understanding.145 In 2015, the Secretaries of DVA and Services Australia signed a Statement of Intent which expanded the scope of cooperation. They agreed on a set of principles for managing the partnership.
- DVA will leverage Services Australia’s technological capability to modernise business operations.
- Standard solutions will be used to leverage technology, frameworks, processes and platforms, unless there is a strong business case not to do so, for example, unacceptable client or financial impacts.
- DVA will adjust business processes to the fullest extent possible to take advantage of Services Australia’s technologies and processes.
- Services Australia and DVA will work closely to minimise service delivery impacts on DVA clients with the intent of improving the delivery of services.
3.81 The VCR Second Pass Business Case reiterated these principles as relevant to managing eight of the 13 program risks identified. The majority of these risks centred on the provision of timely and appropriate ICT builds, assessed as presenting a medium level of risk.
Value for money in partnering arrangements
3.82 It is important that entities take steps to ensure value for money146 and risks to the achievement of this are managed throughout the life of a project. The ANAO reviewed DVA’s strategies and actions for ensuring value for money, and its adherence to the agreed principles governing the partnership. An overview of audit findings is provided below, with further detail and evidence provided in Appendix 8.
3.83 There has been a need to better align business operational requirements with ICT development planning. Despite the use of joint planning or co-design processes, delays have occurred as DVA and Services Australia have not always established a common understanding of requirements, business solutions or acceptable levels of risk.
3.84 The extent to which projects have been designed using standard solutions has been less than originally envisaged by the business case. Some assumptions made by DVA and Services Australia about the ability to transfer DVA functions (including their underlying business rules) to Services Australia systems were incorrect.
3.85 In December 2019, DVA established a Requirements Management Office to set standards for, and to support staff in, developing appropriate requirements within scheduled timeframes.147 In March 2020, DVA also reached agreement with Services Australia on a formal costings process.
Implementation and the management of risk
3.86 DVA business areas have not been consistent in signing off projects following user acceptance testing. DVA has had a lower appetite for risk in the delivery of ICT solutions than Services Australia. This is particularly the case when there is a risk that a new system or system component may not deliver the level of performance that may be expected by DVA or its clients.148 The unanticipated complexity of transferring DVA functions to newly built Services Australia systems has contributed to DVA’s risk sensitivity.
3.87 As a client of Services Australia, DVA business areas should ensure that solutions are fit for purpose, conform with legislative requirements, and take into account the well-being or vulnerability of clients, before making a ‘go live’ decision and agreeing to project closure. However, there may be scope for DVA to adjust its expectations of service to avoid the need for customisation, and to enable Services Australia to provide standardised and cost-effective solutions, at acceptable levels of risk. This would align with the principles set out under the Bilateral Statement of Intent.
3.88 While DVA and Services Australia have worked closely together to deliver suitable solutions, bilateral processes have not sufficiently addressed differing tolerances for risk across DVA business areas. While assurance assessments of VCR projects with major ICT components have identified common risks149, there are no agreed risk thresholds that would assist decision-makers to be consistent in judging acceptable risk.
3.89 The Department of Veterans’ Affairs establish, in agreement with Services Australia, risk appetite statements, related to cost and quality of service, for each ICT development project.
Department of Veterans’ Affairs response: Agreed.
3.90 DVA has commenced work on refreshing the Bilateral Agreement between DVA and Services Australia that governs the overarching ICT relationship between agencies. It is anticipated that this Bilateral agreement will incorporate strategic statements on risk tolerance, diversity of client cohorts and expectations relating to cost and quality of service. These statements will be implemented across DVA, including VCR, and reflected in program governance and assurance for ICT development projects. DVA has commenced consultation to develop these risk appetite statements and will progress this recommendation over the next 12 months and will report progress to the ANAO quarterly. DVA and Services Australia have assigned senior responsible officers (SRO’s) for the VCR program and for the DVA Partnership at Deputy Secretary level, who will be responsible for active management and resolution of potential delays and other risks as they arise.
Monitoring and evaluation
3.91 Progress reports are provided regularly to governance forums, however corrective actions have generally not led to sustained improvement in performance. A DVA-Services Australia risk plan for Year 3 identifies shared responsibilities for management of risks to ICT delivery. However, there is limited evidence of active monitoring, updating or management of these risks by the TPB or the S&IS Project Board.
Preventing the erosion of value for money
3.92 While funding for ICT development is allocated to Services Australia, the consequences of project delays and additional costs are borne by DVA. Costs arise from an on-going need to maintain extant systems and the opportunity cost of not realising efficiencies from new systems.
3.93 Savings associated with the transfer of functions to Services Australia’s systems supported the case for government investment in reform. In the first three years of the VCR program, no planning was undertaken to decommission superseded systems. Planning for the decommissioning of some systems from 2021–22 commenced in October 2020, however implementation is contingent on future government approvals and funding.150
Project management performance standards
3.94 The VCR Business Case stated that governance arrangements would detail obligations, service level agreements and costing models between the two agencies.151 The Statement of Intent allows for projects to be implemented in accordance with agreed project plans, aligned with Services Australia’s project management framework.152 However, there are no overarching project management criteria or metrics, against which Services Australia performance can be measured. In November 2020, DVA advised the ANAO that it had commenced a discussion with Services Australia on establishing a project assurance framework.
3.95 A performance framework governing the delivery of all ICT projects should include processes for managing instances when DVA expectations differ from the standard offered by Services Australia, as well as promote consistency and efficiency in making risk-based decisions.153
Department of Defence
3.96 DVA and the Department of Defence (Defence) engage with members of the Australian Defence Force (ADF) during their careers and in their transition from service, and both are responsible for providing support services.154
3.97 The 2019 Productivity Commission review noted a history of ‘government silos…gaps and duplication, and both Defence and DVA losing sight of what is needed to improve veterans’ overall wellbeing’.155 Closer cooperation with Defence was a key focus of the VCR business case. It described an ‘Early Engagement Model’ as an approach and set of initiatives aimed at ‘reaching as far back into Defence as possible’ to access information relevant to future claims, and to deliver health and rehabilitation services as close to the time of an injury as possible.
3.98 In 2016, the Government established a Joint Transition Taskforce to engage current and former ADF members, family members and caregivers in identifying barriers to successful transition. Recommendations were to: make better use of data to improve the veteran support system; integrate services of DVA and Defence; address barriers created by privacy requirements; and enhance career and employment assistance.
3.99 Under the VCR program, measures include having payments and services in place before separation, using automated registration, and transferring data from Defence to DVA from enlistment to separation.156 DVA now receives notification of serious and very serious injury to current serving members on occurrence.
3.100 DVA requires access to records held by Defence to enable the determination of claims by separating members. This involves issuing individual requests under administrative arrangements agreed in 2011.157 In 2019–20, DVA issued close to 33,000 requests for information to Defence.
3.101 Proposals for a more efficient system of automated and real-time transfer of Defence information to DVA have been regularly discussed between DVA and Defence for over a decade. The DVA-Defence Electronic Information Exchange (DDEIE) initiative underway since 2017 is due to be completed in June 2021.158 It will encompass all serving personnel. Further to this, the Government’s interim response to the 2019 Productivity Commission report in October 2020 announced Stage 1 funding of $0.5 million for the development of a platform to bring together Defence and DVA de-identified data in one place to better understand the cost of service injuries and to inform prevention strategies.159 Other initiatives include:
- the introduction of the White Card for Mental Health in January 2019, issued by DVA, after an ADF member has completed at least one day of continuous service.160 This has supported automatic registration of veterans with DVA161; and
- a Special Operations Forces Pilot conducted in Year 1. The pilot introduced a single point of contact for members on Defence bases, and informed the implementation of a national service model.
3.102 While the VCR business case included a focus on promoting veterans’ employment, work to date under the program has been limited.162 DVA has not measured progress against this objective which forms part of its benefits realisation plan (sub-benefit 2.2: ‘Higher return to work rates’).
3.103 DVA-Defence initiatives are not consolidated in a single work stream. Reporting on progress in implementation is therefore distributed across several work streams. The ANAO identified two Key Performance Indicators for the Early Engagement Model — the total number of welcome emails to newly enlisted ADF members, and new registrations which are automatically processed.163
3.104 There would be merit in DVA establishing regular, consistent and comprehensive reporting to assess the effectiveness of joint transition initiatives, and to provide a clear line of sight to the business case objectives of supporting a seamless transition from the ADF.
DVA engagement with staff
3.105 Poor management of organisational change can pose significant risk to realising the benefits of reform. To address these risks, DVA has undertaken a range of activities to support cultural change. These initiatives include:
- establishing a cultural vision in 2016 and dedicated change management unit to direct and coordinate change activities;
- engaging with delegates in 2018 to promote a client focus and greater standardisation in making claims determinations164;
- leadership development and team-based training programs; and
- regular communication with staff, including fortnightly briefings to middle management staff, and all staff showcases of new capabilities and services.
3.106 Reform planning and implementation are supported by change management strategies and change plans for VCR work streams. In 2019, program reporting to the TPB detailed how the introduction of new systems or business processes were likely to affect business areas.165 In Year 4, reporting on the expected impacts of changes form part of individual project reports.
3.107 As of February 2021, the department was revising its approach to delivering change management support to VCR and non-VCR projects to more effectively align resourcing with different levels of need. Reporting on change activity to the Portfolio Investment and Implementation Committee is to occur via a quarterly Change Snapshot which sets out key upcoming project milestones and requirements for change management support. VCR projects receive priority in the allocation of resources.
3.108 DVA has used specialist change managers to prepare business areas for the implementation of workplace changes.166 It has also established two staff-based support networks. These are a Staff Reference Group, which provides input to a weekly update to staff by the Secretary, and a Transformation Champion Network made up of staff who are motivated to promote reform activities.
3.109 An 2019 internal review of DVA’s change management capabilities identified a number of on-going reform challenges:
- a large number of projects with technology components requiring staff to work differently;
- staff confidence in the management of change and sufficient communication about DVA’s longer term workforce strategy; and
- change support not effectively targeted to priority areas.
On-going attention is needed to build and maintain staff confidence that reform will fully deliver against commitments.
4. Monitoring, evaluation and reporting
This chapter examines whether the Department of Veterans’ Affairs (DVA) is appropriately measuring the performance of the Veteran Centric Reform (VCR) program and reporting on this to the public and the Parliament.
DVA’s monitoring, evaluation and reporting on the VCR has been partially effective. DVA regularly reports on key achievements in its implementation of the program. However, there is a need for DVA to consistently measure and report on the financial and non-financial benefits of the program. Reporting to the Parliament through its annual performance statements does not explain how reforms are affecting the department’s performance in relation to its key operating metrics. Communication with the veteran community occurs mainly through established processes and is effective.
Areas for improvement
The ANAO made one recommendation and several suggestions aimed at strengthening measurement of the VCR program. The ANAO also suggested that DVA provide more complete information to the Parliament on program performance, and its relevance to increasing the department’s effectiveness and efficiency.
4.1 The VCR program is seeking to significantly improve its provision of services to veterans and their families. The Public Governance Performance and Accountability Act 2013 (PGPA Act) provides the basis for the Commonwealth performance framework. Performance information should allow an account to be given of how reform activities are meeting program and broader government objectives.
4.2 The ANAO examined whether DVA uses a suitable performance framework to monitor and evaluate its achievement of reform objectives, and to strengthen program implementation. It also reviewed whether the department’s engagement with the Parliament, the veterans’ community and the public on the VCR program is effective in communicating progress in implementation and the delivery of outcomes.
Does DVA appropriately monitor and evaluate the outcomes of veteran centric reforms?
DVA’s program performance framework is broadly suitable for monitoring and evaluating the outcomes of the VCR reform program. However, the department’s measurement of program performance has been limited by the lack of a robust methodology and systematic monitoring. DVA has therefore not appropriately monitored the outcomes of the VCR program and is unable to demonstrate the full scope of improvement delivered by the VCR program. Work commenced in late 2020 to address this.
VCR performance framework
4.3 As part of its 2017 business case to the Government, DVA committed to putting in place a performance framework to demonstrate the results of investment in reform. A benefits realisation plan was to capture key areas of benefit to clients and the Government, as well as provide a base of evidence to support future Budget submissions.
4.4 In its early planning of the VCR program, the department identified the steps it would take to develop and implement the benefits realisation plan. This included:
- documenting the expected benefits of work streams, and establishing success measures, baselines and targets;
- assigning responsibility for the achievement of targets within defined timeframes; and
- establishing regular monitoring, evaluation and reporting of benefits.
This approach aligns with principles of effective performance measurement.167
4.5 DVA’s high-level framework identifies two categories of benefit:
- non-financial benefits — benefits for veterans and their families; and
- financial benefits — mainly benefits to government, including departmental efficiency savings, reduced expenditure on programs, and reductions in liability expenditure over time.
4.6 Five key areas of program benefit, comprising 16 sub-benefits, are expected to arise from program implementation, as outlined in Table 4.1 below. These benefit descriptions have been retained across the three years of VCR implementation.
1. Improved veteran and family experience
1.1 Improved client satisfaction
1.2 Faster access to information and support
1.3 Services aligned to veteran needs and expectations
2. Better health and well-being outcomes
2.1 Improved physical and mental health and well-being
2.2 Higher return to work rates
3. Enhanced DVA operations
3.1 Streamlined business processes
3.2 Improved DVA reputation
3.3 Reduced risk to service delivery
3.4 Improved internal culture
3.5 Improved capabilities
Reduced administered expenditure and unfunded liability
4.1 Reduction in the unfunded liability
4.2 Lower demand or supply of goods and services
4.3 Lower unit costs
Reduced departmental expenditure
5.1 Reduced staff expenses, overheads and other operating costs
5.2 Lower capital expenditure
Source: ANAO analysis of data provided by DVA.
4.7 The framework was outlined in the 2017 VCR Second Pass Business Case, with subsequent VCR funding submissions reiterating a commitment to capturing benefits. Some initial targets were identified in 2017. By the beginning of 2018, some baseline data, targets and timeframes had been identified by project managers.
4.8 Reporting to the Transformation Program Board (TPB) between February 2018 and May 2019 focused on three performance measures: number of files digitised and reduction in mail costs; reduction in the number of telephone lines; and time taken to determine claims. Progress against these was measured against baselines and with reference to targets. In May 2020, these measures were related to the benefits realisation framework and strategic business outcomes.168
4.9 Following a recommendation of the 2019 Gateway Review, DVA commissioned a review to strengthen its tracking and reporting of outcomes and benefits. The review identified that of 77 active measures, 28 had no target and no baseline, 31 had a baseline and no target, ten had a target and no baseline, and only eight had both. Where measurable indicators had been identified, there was often a lack of reliable data to support assessments.169
4.10 While some indicators for measuring project outputs and outcomes have been established, these are not clearly linked to the benefits outlined in the program level performance framework. Changes to the organisation of work within and across years has reduced the department’s ability to demonstrate clear and consistent links between work undertaken and longer-term outcomes. See paragraphs 2.55 to 2.70 for more information on DVA’s planning and coordination of implementation processes for the VCR program.
4.11 As at December 2020, DVA had commenced a process of revising its approach to capturing and reporting benefits. Programs involving significant reform sometimes use ‘logic models’ or ‘theories of change’ to develop performance measures.170 Strategic benefits can then be shown as resulting from specific activities that have been measured and monitored. These principles are being applied to DVA’s revision of its benefits-realisation framework.171 DVA could consider using a logic model approach to cogently explain the relevance and impact of individual measures under the VCR program.
Delivery of benefits
4.12 Planning for Year 4 noted that an inability to demonstrate benefits would limit the scope to identify savings to offset future funding submissions. The status of individual VCR projects has been reviewed at the conclusion of each year, and updated business cases have provided examples of program achievements. However, these stocktakes have not been undertaken with reference to the VCR benefits-realisation framework.
4.13 The Second Pass VCR Business Case stated that benefits (including savings) would be monitored regularly against targets to gauge progress, and where necessary, allow for policies and activities to be adjusted to maximise the value of reform for veterans and government.172
4.14 Without a framework clearly linking project activity to strategic benefits, the department has a limited base of evidence for adjusting priorities and spending. There is a risk that decisions will be made primarily with reference to implementation activity rather than a broader view of reform outcomes.
4.15 VCR savings measures are given effect through reductions in DVA’s annual appropriation. The department is required to ensure it delivers savings equivalent to those anticipated by the Budget decision.173 Failure to meet saving commitments results in added pressure on the department’s budget (see paragraph 2.40).
4.16 Some financial benefits have been realised. The department advised the ANAO in February 2021 that of five administered measures with savings forecast for Year 3, two have been achieved.174 Savings from the remaining measures were not achieved.
4.17 Workforce savings included in the Year 3 business case were determined after consultation with DVA executives. However, some executives did not have direct responsibility for making necessary changes, and accountability for implementation was unclear following organisational and staffing changes. In Year 3, the EPO did not receive updates on the achievement of savings by business areas. The finance branch also did not monitor the progress of business areas in achieving savings.
4.18 A 2018–19 financial savings target relating to work undertaken by an ‘operational improvement’ team, proposed by the Second Pass Business Case, was reduced from $9.2 million to an estimate of $1 million.175 The work of this team has ceased, in addition to that of an enabling services team, also intended to deliver savings.
4.19 Increased revenue is stated as a sub-benefit in the VCR benefits framework, however DVA has not identified measures relevant to delivering revenue benefits.
4.20 The 2019 Gateway Review recommended on-going reporting to governance bodies on progress and actions in delivering departmental and administrative savings. In November 2020, regular, standardised reporting on the performance of savings measures was in development.176
4.21 When revising its financial and non-financial performance framework, DVA should:
- more clearly show how key VCR investments, process improvements and capabilities contribute to changes intended by reform;
- establish information and continuous data collection methods to enable it to assess whether investments are performing to expectation;
- consider measuring qualitative or efficiency improvements through a consistent mix of proxy measures, if necessary, to ensure completeness in reporting177; and
- ensure appropriate records are maintained to demonstrate accuracy and reliability in reporting.
4.22 It is sometimes necessary to invest resources in order to realise savings from the implementation of reforms. Benefits should be recorded as net benefits within the context of the department’s overall financial performance.178
4.23 The Department of Veterans’ Affairs develop consistent and reliable indicators of program performance to improve the quality and completeness of reporting on the benefits of veteran-centric reforms.
Department of Veterans’ Affairs response: Agreed.
4.24 DVA acknowledges the benefits realisation approach requires strengthening and is committed to doing so. DVA reviewed its benefits framework in August 2020 and commissioned additional expertise to reshape its benefits approach (referenced at point 4.11 of the ANAO report). The commissioned work, which is expected to be completed by the end of June 2021, seeks to provide consistent and reliable program level performance measures as well as linking of project level metrics to the program benefits. DVA will then be able to better track progress towards achieving the desired program performance. DVA will work actively to implement the strengthened approach to benefits realisation, to take effect as soon as completed and to be fully embedded over the next twelve months, with a progress report to the ANAO quarterly.
4.25 Both the positive and negative effects of reform on processes of service delivery are relevant factors in evaluating the benefits of investment in reform. As an example, easier access to services may lengthen the time taken to process claims if this capability is not increased also, creating an unintended dis-benefit. In monitoring the realisation of benefits the costs and impacts of not transitioning to a business as usual status should also be taken into account.
Veterans’ experience of the VCR program
4.26 DVA veteran representatives interviewed by the ANAO indicated they have seen improvements in DVA’s ability to meet the needs of the veteran population.179 Appendix 7 sets out specific improvements cited by interviewees.
Does DVA appropriately report on the progress of its reform activity to the Parliament and the public?
DVA’s reporting to the Parliament is largely appropriate, but would be strengthened through more complete information about the performance of the VCR program, and its relevance to improving the effectiveness and efficiency of the department. Reporting to the veterans’ community is through established mechanisms and effective.
4.27 Under the PGPA Act, Australian Government entities are required to demonstrate to the Parliament and the public that the Government’s objectives are being met, and that public investments represent value for money.180
4.28 The ANAO examined whether the department is effective in providing assurance through public information that the intent and full scope of VCR benefits are being delivered.
Reporting to the Parliament
4.29 A key means by which a Commonwealth entity provides regular and complete information about its performance is through the Commonwealth Performance Framework.181 DVA’s outcome is articulated in its Corporate Plan182 and Portfolio Budget Statements.183 The Corporate Plan sets out high-level criteria against which DVA’s performance will be measured. KPIs relate to core activities it undertakes to achieve its purposes.
4.30 DVA’s Corporate Plans and annual reports provide some information about the VCR program. It is referred to three times in its 2019–23 and 2020–21 Corporate Plans, as part of a broader discussion of ‘changing needs and expectations of veterans and their families’.184 Annual reports have included updates on improvements, but do not provide an account of program performance, or detail whether benefits have been delivered, as intended by the Government. The narrative text does not, therefore, provide a complete view of performance.
4.31 The VCR program aims to significantly improve the efficiency and effectiveness of the department’s operations. There would be merit in the department assessing how progress in the implementation of reforms is likely to contribute to the entity’s overall operating performance.
4.32 DVA has supported its Minister in providing information to the Parliament about the VCR program through ministerial statements, explanatory memoranda as part of introducing proposed amendments to legislation, and responses to parliamentary inquiries and other reviews.185
4.33 The department has made submissions to inquiries and the Secretary has provided opening statements to Senate Estimates Hearings. The main focus of this reporting has been the Government’s reform objectives and initiatives aimed at addressing the concerns of the veteran community.
Communicating with the veterans’ community
4.34 DVA provides regular information to the veterans’ community through well-established communication processes. These communication mechanisms include the:
- National Consultation Framework — veteran and ex-service community organisations, and the Repatriation and Military Rehabilitation and Compensation Commissions;
- an Ex-services Organisation Round Table (ESORT); and
- Female Veterans and Veterans’ Families Forum.
4.35 In 2017, DVA developed a Stakeholder Engagement and Communication Strategy to promote awareness of the reform program and provide information about new services. The strategy envisages the use of established communication channels, including conferences, DVA and Defence websites, Services Australia shopfronts, newsletters and social media.
4.36 Some veteran representatives interviewed by the ANAO indicated that DVA has become more receptive to using forums to ‘solve problems’, rather than simply for the one way presentation of information.
Appendix 1 Department of Veterans’ Affairs response
Appendix 2 Projected numbers of claimants under the MRCA, DRCA and VEA Acts
Source: ANAO analysis of data provided by DVA.
Appendix 3 Key reviews addressing veterans’ mental health issues
National Mental Health Commission (NHMC), Review into the suicide and self-harm prevention services available to current and former serving ADF members and their families. The Government issued its response in June 2017.
DVA, Department of Defence, and the Veterans’ and Veterans Families’ Counselling Service (VVCS) joint inquiry into the management of Jesse Bird’s case (‘Joint Inquiry’) at the request of the Hon Dan Tehan, MP.
The report was not published, however its recommendations were accepted and tabled in Parliament on 24 October 2017, with a commitment to independently review the implementation of recommendations after a year. The review report was delivered to government on 15 March 2019.
Internal DVA report on DVA’s compliance with legislation and policy in the handling of Jesse Bird’s case (C.Spiers).
Veterans’ Advocacy and Support Services Scoping Study (R. Cornell).
Independent review of the implementation of the 19 recommendations made by the 2017 Joint Inquiry Report (Professor R. Creyke). Seven recommendations related directly to the objectives of the VCR. The review determined that of these, three were complete, two were substantially complete and two were partially complete.
Inquiry into Transition from the Australian Defence Force, Joint Standing Committee on Foreign Affairs, Defence and Trade.
Coronial Inquest Report — Inquest into the Death of Jesse Bird by Coroner Jacqui Hawkins.
The Mental Health Impacts of Compensation Claim Assessment Processes (A. Collie).
Source: ANAO analysis of public information.
Appendix 4 VCR funding and savings commitments
Notes: Figure A.1 excludes funding of $0.4 million over four years allocated to the Department of Finance to conduct Gateway Reviews of the VCR program.
Services Australia has been allocated a total of $46.6 million in capital funding for VCR ICT systems development. In 2017–18, DVA was allocated $2.4 million in capital funding to mitigate extant cyber security risks.
Source: ANAO analysis of data provided by DVA.
Source: ANAO analysis of data provided by DVA.
Appendix 5 Status of key VCR work streams
Status of key VCR work streams
Students and Income Support (S&IS)
Proactive Interventions/Data and Analytics
Source: ANAO analysis of data provided by DVA.
Appendix 6 VCR-related legislative changes
In 2018–19, a number of VCR measures were enacted via amendments to legislation to support more efficient claims processes.
- Veterans’ Affairs Legislation Amendment (Veteran-centric Reforms No.1) Act 2018
- Veterans’ Affairs Legislation Amendment (Veteran-centric Reforms No.2) Act 2018
- Veterans’ Affairs Legislation Amendment (Omnibus) Act 2018
- Treatment Benefits (Special Access) Act 2018
- Treatment Benefits (Special Access) (Consequential Amendments and Transitional Provisions) Act 2018
- Australian Veterans’ Recognition (Putting Veterans and their Families First) Act 2019 is a new Act
VCR measures were also implemented by legislative instruments made by the Repatriation Commission or the Military Rehabilitation and Compensation Commission (MRCC):
- Veterans’ Affairs (Treatment Principles — Veteran Suicide Prevention Pilot) Amendment Instrument 2018
- Military Rehabilitation and Compensation (Catastrophic Injury or Disease) Determination 2018
- Safety, Rehabilitation and Compensation (Defence-related Claims) (Catastrophic Injury) Rules 2018
- Safety, Rehabilitation and Compensation (Defence-related Claims) (Full time Study) Instrument 2018
- Veterans’ Entitlements (Provisional Access to Medical Treatment) Amendment Determination No. 2 2020
Appendix 7 Improvements to veterans’ services cited by veteran representatives
Initiative and business case outcome
VCR benefits cited by veteran representatives interviewed by the ANAO
Leadership and consultation with veterans
MyService digital portal (April 2017)
(simpler and digitally enhanced experience with DVA)
Website (March 2020)
(simpler and digitally enhanced experience with DVA)
(simpler and digitally enhanced experience with DVA)
Non-liability mental health care
(improved mental and physical health, and well-being)
Provisional Access to Medical Treatment (PAMT)
(improved mental and physical health, and well-being)
Complex case management
(improved mental and physical health and well-being, and improved ICT platform)
Single view of client (SVoC)
(improved mental and physical health, and well-being, and improved ICT platform)
Defence Early Engagement model (2017)
(successful transition from the ADF)
(improved mental and physical health, and well-being, and improved ICT platform)
Note: The ANAO interviewed nine members of veterans’ organisations. These interviews were facilitated by DVA. The information presented in this table may not reflect the views of the broader veteran community.
Source: ANAO analysis of information provided in interviews conducted by the ANAO.
Appendix 8 DVA’s management of value for money and risk in partnering with Services Australia
Project phase and tests
Design — Clarifying the purpose and expected results of a project proposal is necessary in order to understand whether the investment of funds is appropriate.
Implementation — Agreed project governance and management processes are necessary to preventing an erosion of value over the course of implementation.
Monitoring and evaluation — Frameworks for monitoring progress in implementation are necessary for assessing whether value for money is being achieved. Project closure processes ensure a new service is transitioned to standard operations and managed under standard service arrangements.
Source: ANAO analysis of information provided by DVA.
1 Services Australia provides ICT design and build services for the VCR program. The allocation includes $0.4 million over four years to enable the Department of Finance to conduct regular assurance reviews.
2 As at October 2020, the Government is yet to make a decision on the continuation of the VCR program. Its interim response to the Productivity Commission Report on 8 October 2020 stated that the VCR program is ‘the primary vehicle for reform’. See Department of Veterans’ Affairs, Interim Government Response to the Report of the Productivity Commission: A Better Way to Support Veterans No. 93 [Internet], DVA, Canberra, 2020, p. 2, available from https://www.dva.gov.au/sites/default/files/files/consultation%20and%20grants/reports/interim-government-response-to-productivity-commission-report-a.pdf [accessed 9 October 2020].
3 These are set out in DVA’s Portfolio Budget Statements (PBS). It reports on the achievement of these outcomes in its annual report.
4 These offices are part of the Veterans’ Access Network (VAN) offices. In partnership with Services Australia, DVA also supports the Mobile Services Centre Pilot Program which provides access to services in rural and regional areas.
5 DVA assesses that this increase is attributable, in part, to early engagement strategies, improved digital services, and access to non-liability healthcare. It considers the greater rate of deployment by the ADF from 1995–2000, compared with 1970–1995, to be an additional key factor.
6 From 12 October 2017, the Safety, Rehabilitation and Compensation (Defence-related Claims) Act 1988 (Cth) (DRCA) replaced the Safety, Rehabilitation and Compensation Act 1988 (Cth) (SRCA) for current and former ADF members who have injuries or illnesses arising from their service prior to 1 July 2004.
7 Between 2017–18 and 2018–19, the number of claims lodged across four major claim types rose between 54.4 per cent and 125.8 per cent. The average number of injuries per claim increased from 3.8 to 6.1 (based on MRCA Permanent Impairment outlays). Information about claim trends is provided in DVA annual reports: 2014–15, pp. 59 and 62; 2015–16, pp. 49–50; 2016–17, pp. 225–226; 2018–19, pp. 234–236; and 2019–20, pp. 194–196. See Department of Veterans’ Affairs, Annual Reports [Internet], DVA, Canberra, 2020, available from https://www.dva.gov.au/documents-and-publications/annual-reports [accessed 4 November 2020].
8 The VEA scheme provides benefits to specific veteran cohorts, primarily those who are older. These include war widowers (84 per cent are aged over 75 years), and Service Pensioners or Disability Pensioners (78 per cent are over 65 years). See Department of Veterans’ Affairs, DVA Stats at a Glance [Internet], DVA, Canberra, 2020, available from https://www.dva.gov.au/about-us/overview/research/statistics-about-veteran-population [accessed 7 October 2020].
9 Department of Veterans’ Affairs, Annual Report 2019–20 [Internet], DVA, Canberra, 2020, p. 37, available from https://www.dva.gov.au/documents-and-publications/annual-reports [accessed 4 November 2020].
10 This is based on the present value of future payments of claims incurred at 30 June 2020, and includes potential claims, not yet reported. See pages 96 and 130 of DVA’s annual report for 2019–20.
11 Seven low risk conditions are processed via computer-based decisions and approved instantly. In February 2019, claims processed via computer-based decisions in less than one day made up 18 per cent of all claims processed.
12 Productivity Commission, A Better Way to Support Veterans No. 93 [Internet], Productivity Commission, Canberra, 2019, p. 2, available from https://www.pc.gov.au/inquiries/completed/veterans/report [accessed 7 October 2020]. The Inquiry followed the Government’s response to a report by the Senate Standing Committee on Foreign Affairs, Defence and Trade on suicide by veterans and ex-service personnel. See Australian Government, Australian Government Response to the Foreign Affairs, Defence and Trade Committee Report — The Constant Battle: Suicide by Veterans [Internet], Canberra, 2017, available at https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Foreign_Affairs_Defence_and_Trade/VeteranSuicide/Government_Response [accessed 7 October 2020].
13 Ibid., p. 399.
14 The remaining 28 per cent submitted their claim via a veterans’ advocate, either online or using a paper form.
15 In 2016, 50 per cent of survey respondents under 45 years first contacted DVA via telephone, with contact through DVA’s online portal at four per cent.
16 Before 1 February 2020, Services Australia was called the Department of Human Services. In this report, the term ‘Services Australia’ refers to both the current agency and the former Department of Human Services.
17 Refer to the Auditor-General Report No.52 2017–18 Efficiency of Veterans Service Delivery by the Department of Veterans’ Affairs, pp. 9, 31 and 41. In 2016–17, 46.7 per cent of all decisions referred to the Veterans’ Review Board were set aside and 33.9 per cent of decisions referred to the Administrative Appeals Tribunal in 2016–17 were set aside, p. 51. A range of factors contribute to these decisions.
18 An amount of $6.1 million was also provided to Services Australia to support the development of the VCR business case.
19 This includes $0.4 million over four years allocated to the Department of Finance to conduct regular assurance reviews.
20 Australian Government, Budget Measures: Budget Paper No. 2: 2017–18, Commonwealth of Australia, Canberra, 2017, pp. 178–9.
21 Australian Government, Budget Measures: Budget Paper No. 2: 2018–19, Commonwealth of Australia, Canberra, 2018, p. 190.
22 Australian Government, Budget Measures: Budget Paper No. 2: 2019–20, Commonwealth of Australia, Canberra, 2019, pp. 174–5.
23 2017–18 Veteran Centric Reforms Second Pass Business Case, p. 42.
24 Department of Veterans’ Affairs, Annual Report 2019–20 [Internet], DVA, Canberra, 2020, p.vi, available from https://www.dva.gov.au/documents-and-publications/annual-reports [accessed 4 November 2020].
25 Updates to the original business case in Years 2 and 3 retained its description of outcomes and the means for achieving VCR outcomes. The Year 2 submission, including an updated business case, was approved by the Government on 7 March 2018 (Delivering Australia’s Digital Future — Veteran Centric Reform — Continuation). A further submission and business case was approved by the Government on 15 February 2019 (VCR – Putting Veterans and their Families First).
26 Productivity Commission, A Better Way to Support Veterans No. 93 [Internet], Productivity Commission, Canberra, 2019, p. 6, available from https://www.pc.gov.au/search?collection=productivity-commission-web&form=site-search&query=veteran [accessed 7 October 2020].
27 Department of Finance, Developing a business case [Internet], Finance, Canberra, 2020, available from https://www.finance.gov.au/government/commonwealth-investment-framework/commonwealth-investments-toolkit/developing-business-case [accessed 1 October 2020]. There are generally two main stages at which the Government is presented with a business case. A First Pass Business Case outlines the objectives and estimated costs of a major proposal for investment at a high-level, and seeks government approval and funding for the development of a Second Pass Business Case. This then details options for implementation at different levels of cost.
28 Department of Finance, Resource Management Guide 308 — Commonwealth Investments [Internet], Finance, Canberra, 2020, available from https://www.finance.gov.au/publications/resource-management-guides/commonwealth-investments-rmg-308 [accessed 20 October 2020].
29 A 2018 Auditor-General report on DVA’s delivery of services found there were weaknesses in DVA’s processes for managing manual records associated with multiple claims made by individual clients, contributing to inaccuracies and a lack of timeliness in claims processing. See Auditor-General Report No.52 2017–18 Efficiency of Veterans Service Delivery by the Department of Veterans’ Affairs, p. 42.
30 Digital service standards were introduced in May 2016 for all Australian Government services. These establish criteria for simpler, faster and easier to use services. See Digital Transformation Agency, Digital Service Standard: Lighthouse — Alpha assessment [Internet], DTA, Canberra, 2016, available from https://www.dta.gov.au/help-and-advice/digital-service-standard/assessment-reports/lighthouse-alpha-assessment [accessed 20 October 2020].
31 The Welfare Payment Infrastructure Transformation (WPIT) program, announced in April 2015, and under development by Services Australia, is intended to provide web portal (myGov), telephone, face-to-face, online and mobile solutions for entities across government under shared services arrangements.
32 See Department of Veterans’ Affairs, Consultation with female veterans and veterans’ families [Internet], DVA, Canberra, 2020, available from https://www.dva.gov.au/about-us/overview/consultations-and-grants/consultation-female-veterans-and-families/consultation [accessed 20 October 2020].
33 Following the Government’s decision, the Interdepartmental Committee was replaced by the Transformation Program Board, discussed in Chapter 3, (see paragraphs 3.15–3.20).
34 A cost–benefit analysis compares different options by valuing each option in terms of its net benefits. This supports resource allocation decisions where choices are to be made between alternative uses of funds.
35 Option A (‘DVA Transformation’) proposed significant changes to DVA’s way of operating and ICT-based service delivery. The proposed cost was $450 million–$499 million over six years. Option B proposed modernisation of ICT systems with some changes to business processes. The proposed cost was $320 million– $370 million over five years.
36 Business cases involving significant ICT components are required to detail how proposed solutions meet requirements for: functionality; security and privacy; reliability; availability and maintenance; compliance with design standards; and flexibility and scalability. See Department of Finance, ICT Business Case Guide [Internet], Finance, Canberra, 2015, available from https://www.finance.gov.au/sites/default/files/2019-11/ICT_Business_Case_Guide.pdf [accessed 14 October 2020].
37 Services Australia received $6.1 million between 2015 and 2017.
38 The Second Pass Business Case was considered by the Digital Transformation Sub-committee of Cabinet. Decisions made by a sub-committee of Cabinet have the status of a Cabinet decision.
39 Estimated departmental savings arising from staff reductions and improvements in operational processes ranged from $117 million to $509 million.
40 A review by Ernst and Young in 2015 found that DVA’s multiple ICT systems were inadequate for efficient claims processing and developing an integrated view of the client (termed a ‘a single view of person’).
41 On the basis of current VCR funding, the digitisation measure is scheduled to conclude in 2022–23.
42 The first tranche of ICT work was also intended to stabilise DVA infrastructure and applications, as these were at a lower base of capability than Services Australia’s Centrelink program, and to establish projects to demonstrate quick improvements.
43 As an example, the updated Business Case for Year 2 (approved 7 March 2018) advised that work scheduled to be completed in Year 1 to consolidate telephone numbers and implement a 1800VETERAN number would need to be continued into Year 2.
44 Option 1 proposed net funding of $264.2 million over four years from 2019–20. Option 2 proposed a net cost of $125.9 million over the same period.
45 Compensation and rehabilitation services include transport, community nursing, aged care and medical services.
46 Option 3 proposed one year of funding and option 4 offered the suspension of the VCR program for six months.
47 The number of staff proposed to be transferred was equivalent to 238 Average Staffing Level. Savings were also expected from: improved options for the delivery of treatment to veterans; the use of analytics to increase oversight of health expenditure; and business improvement initiatives.
48 The approval given on 15 February 2019 left open a possible decision to proceed with option 1, subject to subsequent consideration via correspondence between the Prime Minister and relevant Ministers. On 5 March, the Minister for Veterans’ Affairs wrote to the Prime Minister. The Government reaffirmed its original selection of option 2.
49 As a consequence of the deferral of the Budget from May 2020 to October 2020, the department was required to obtain new approval to bring a submission forward. The requirement to report on savings has been deferred to the 2021–22 Budget.
50 DVA was not required to deliver savings in 2016–17 and 2017–18. DVA was required to provide departmental savings of $29.7 million in 2018–19 and $91.4 million in 2019–20; and administered savings of $30.3 million in 2018–19 and $215.6 million in 2019–20.
51 Risk is defined as the ‘effect of uncertainty on objectives’. See International Organization for Standardization, AS/NZS ISO 31000:2018 Risk management — Principles and Guidelines [Internet], ISO, Switzerland, 2018, pp. 1–2, available from https://www.iso.org/standard/65694.html [accessed 12 November 2020].
52 The DVA Risk Assessment Matrix rated risks from low to very high.
53 This formed part of the Veteran Centric Reform — Putting Veterans and their Families First submission. Data risks related to access, transmission and the appropriate storage of data.
54 Offsets are required to be made either in the Budget year, over the forward estimates, or a combination of both.
55 Government entities receive two appropriations — departmental and administered. Departmental funding is for the day-to-day operation of the department, including salaries and the purchase of goods and services. Administered funding refers to items the entity does not control, but administers on behalf of the Government. This appropriation is used for expenditure on the design and delivery of government programs. DVA also receives an unlimited special appropriation under administered expenses to support payments to eligible persons, their dependents and other providers. See Public Governance, Performance and Accountability (Financial Reporting) Rule 2015 (Cth), section 5 — Definitions.
56 This was also a key recommendation of the 2016 Functional and Efficiency Review.
57 The department’s advice was based on analysis undertaken by Services Australia.
58 A 2018 review conducted of the WPIT program by Mr David Thodey AO and the Nous Group consultancy assessed the extent to which WPIT deliverables could be ‘re-used’ for a number of other projects, including VCR initiatives. It determined that savings would be driven by the Services Australia Payment Utility and Entitlement Calculation Engine.
59 The Functional and Efficiency Review (FER) was commissioned by the Minister for Finance on 31 August 2015 and commenced on 20 January 2016. Reviews were conducted of all Commonwealth entities to determine whether their resourcing and functions aligned with government priorities, including for ‘smaller Government’. The DVA FER was conducted by David Tune AO PSM, former Secretary of the Department of Finance.
60 Some annual efficiency savings continue beyond the forward estimates published in the department’s Portfolio Budget Statements.
61 The increase in claim numbers relates primarily to permanent impairment, incapacity and other compensation claims. See Department of Veterans’ Affairs, Annual Report 2019–20 [Internet], DVA, Canberra, 2020, p. 93, available from https://www.dva.gov.au/documents-and-publications/annual-reports [accessed 4 November 2020].
62 The department also recorded a loss of $7.55 million associated with its Defence Service Homes Insurance Scheme. This amount made up the balance of the overall loss.
63 The level of liability incurred is estimated annually by the Australian Government Actuary, and is included in DVA’s balance sheet. Annual liability is paid through special appropriations.
64 The approach was termed ‘Priority Investment Approach — Veterans’, to reflect the proposition that expenditure should be considered an investment in improving outcomes for both veterans and their families. Government actuarial data underpins the management of liability levels (termed Active Scheme Management).
65 DVA assessed that younger veterans are at greater risk of being long-term recipients of incapacity payments and rehabilitation services.
66 DVA attributed increased complexity of claims to the tempo of ADF operations since 1999, with a greater number of younger veterans engaged in multiple, longer-term deployments, heightening their exposure to injury.
67 The main contributors to the increase are determinations relating to permanent impairment (30 per cent), medical (57 per cent) and long-term incapacity expenses (30 per cent) under the MRCA scheme. There has also been an increase in benefits payable under the DRCA scheme (125 per cent).
68 The claim acceptance rate has increased from 36 per cent in 2006 to 88 per cent in 2019. A 2019 review by Emeritus Professor Robin Creyke, AO attributed this increase in positive determinations to a shift in DVA decision-making toward a broader interpretation of legislative provisions, to the benefit of the veteran.
69 An initiative to automate the determination of claims under specified conditions (termed straight-through processing, STP), has reduced the time taken by DVA to assess liability. There are 40 conditions that are automatically assessed using STP and streamlining rules.
70 DVA’s annual report for 2019–20 stated the department experienced an increase in claim numbers, in part due to implementation of reforms. In 2019–20, there was an increase of approximately 13,600 claims (15 per cent) across the four major contemporary compensation streams, in addition to the 83 per cent increase in 2018–19. In these streams, DVA was unable to meet set timeliness targets for the processing of these claim types. See Department of Veterans’ Affairs, Annual Report 2019–20 [Internet], DVA, Canberra, 2020, p. 32 and p. 36, available from https://www.dva.gov.au/documents-and-publications/annual-reports [accessed 4 November 2020].
71 On 13 November 2019, as part of the Mid-Year Economic and Fiscal Outlook (MYEFO) process, the Government allocated an additional $20 million in supplementation for DVA to process an increased number of Permanent Impairment claims, and to enable the processing of a backlog of other claims. Supplementation had been provided over the preceding five years. The submission stated that additional departmental funding would be required in 2021 and following years to address the growth and backlog of all claim types. On 10 September 2020, DVA received a further $59.8 million for 2020–21 only, comprising $36.6 million for DVA’s operations and $23.2 million to support claims processing. This was in addition to an amount of $11.6 million previously provided for 2020–21.
72 Additional enabling work streams have been established under the VCR program to support the implementation of reform measures.
73 Pilots included: the development of a predictive model of veteran vulnerability, and Vietnam veterans’ functional loss; a special operations forces pilot for transitioning members; a transition medical assessment pilot; and an Australia Post easy access trial.
74 The EPO has also been called the Transformation Portfolio Office (2019) and Enterprise Program Office (2020). This audit report uses the term ‘Enterprise Program Office’ to describe this function. Services Australia also established a Program Management Office to support implementation of the VCR.
75 An external review in 2020 noted that EPO requests for information were often not fulfilled.
76 This became an Enabling Services Improvement work stream in 2019. Reporting to DVA’s Audit and Risk Committee in October 2020 stated DVA’s work on improving departmental efficiency through standardised corporate services had been defunded due to other priorities.
77 The panel is called the Department of Finance Business Advisory Panel. See AusTender, Standing Offer Notice View — SON33005648 [Internet], AusTender, Canberra, 2015, available from https://www.tenders.gov.au/Son/Show/c0f228f1-b9d4-30f4-d1d6-d541a1a9383f [accessed 22 February 2021]. A Standing Offer Arrangement consists of a Deed of Standing Offer with a supplier that enables the entity to obtain particular goods or services where the prices and contractual terms are already agreed. See Department of Finance, Australian Government Contract Management Guide 2020, v1.3 [Internet], Finance, Canberra, 2020, available from https://www.finance.gov.au/government/procurement/contract-management-guide [accessed 1 October 2020]. Other services to be delivered under the contract were to support the streamlining of DVA’s business processes and the replacement of legacy ICT systems. The final value of the contract was $11 million following two amendments.
78 AusTender, Standing Offer Notice View — SON1700081 (Management Advisory Services Panel — Department of Home Affairs) [Internet], AusTender, Canberra, 2013, available from https://www.tenders.gov.au/Son/Show/9EF01E95-D79C-2555-7DB8-8D34335030ED [accessed 1 October 2020].
79 There are currently 168 suppliers on the panel.
80 These were called: the Portfolio Program Unit; Change Management Unit; and the VCR Program Management Office (the latter is referred to as the Enterprise Program Office in this audit report).
81 The Standing Offer Arrangement contains a clause which allows a work order to be extended beyond the initial term for further periods of up to three years.
82 Conducted by the Department of Finance.
83 See Department of Finance (United Kingdom), Roles and responsibilities of the Senior Responsible Owner [Internet], Department of Finance, UK, available from https://www.finance-ni.gov.uk/articles/roles-and-responsibilities-senior-responsible-owner [accessed 7 October 2020].
84 This is a Deputy Secretary role.
85 A project is defined by having an endpoint, whereas a business as usual function delivers work with no planned end date.
86 An end of the year summary produced by the EPO identifies projects which have transitioned to business as usual status.
87 The PIIC was formed in June 2019 at the direction of the EMB.
88 The establishment of project boards for smaller scale work streams was not considered necessary.
89 Transformation Program Board Terms of Reference.
90 This is a First Assistant Secretary role.
91 This is a Deputy Secretary role. The new Chair had previously participated in the TPB in his role as Chief Technology Officer.
92 The Chair of the TPB is a Deputy Secretary of DVA responsible for Transformation and Enabling Services. Other DVA members are Senior Business Owners (First Assistant Secretaries: Transformation & Organisational Performance; Veterans’ Services Design; Client Engagement & Support Services; Clients Benefits; and Business Support Services). Services Australia is represented by three General Managers. External members are: a Director-General from the Department of Defence; an Assistant Secretary from the Department of Health, Head of Portfolio from the Digital Transformation Agency; and an external adviser.
93 The Enterprise Program Office provides the secretariat to the Transformation Program Board.
94 The VCR projects report monthly. These reports are collated into a consolidated program report.
95 Work streams, projects and project elements (‘features’) are rated green, amber or red to indicate their performance status.
96 The PIIC is co-chaired by two DVA deputy secretaries: Transformation and Enabling Services; and Policy and Programs. Other members are: DVA’s Chief Health Officer; five First Assistant Secretaries; and eight Assistant Secretaries. In 2019, the PIIC was attended by a special advisor from the Productivity Commission Taskforce.
97 Project artefacts are emailed to the EPO or stored in various locations in DVA’s record management system.
98 A guiding principle is that ‘engaging with risk is a necessary step in improving performance’. Refer to Public Governance, Performance and Accountability Act 2013 (Cth), section 16. The Commonwealth Risk Management Policy supports entities in giving effect to the PGPA Act. See Department of Finance, Commonwealth Risk Management Policy [Internet], Finance, Canberra, 2014, available from https://www.finance.gov.au/sites/default/files/2019-11/commonwealth-risk-management-policy_0.pdf [accessed 3 November 2020].
99 A VCR Risk and Issue Management Plan was first issued in September 2017, with the most recent plan produced in September 2019.
100 Examples of risks identified by individual project managers are: ‘significant change impact with no change resource currently identified’; ‘additional work load due to changed policy requirements’; and ‘inability to access historical client data’.
101 Shared risks are those which extend beyond a single entity and which require shared oversight and management. See Department of Finance, Commonwealth Risk Management Policy [Internet], Finance, Canberra, 2014, p. 6, available from https://www.finance.gov.au/sites/default/files/2019-11/commonwealth-risk-management-policy_0.pdf [accessed 3 November 2020].
102 This risk statement is listed in the VCR risk register for May 2020.
103 Insufficient prioritisation and resourcing were rated as low and medium risks, respectively, in the VCR business case. During implementation, these risks have been consistently rated high, even after controls have been applied, and are included in DVA’s enterprise risk framework.
104 The plan was last updated in November 2019. It provided an option for project managers to use a Quality Log to record quality checks and actions taken to implement review recommendations. Project managers have generally elected not to use the log to record their assurance activities.
105 It also recommended that a ‘performance driven contract with Services Australia’, with attention to ‘process, quality, gates, governance, expectations, resources and deliverables’ be established before the commencement of Year 3.
106 DVA uses a ‘traffic light’ reporting framework to report the status of the VCR program to the Transformation Program Board each month (see paragraph 3.55).
107 The performance of Services Australia in delivering components of the VCR program is not assessed.
108 Gateway Reviews form part of the Australian Government Assurance Reviews Framework. The reviews examine programs and projects at key decision points to provide advice and assurance to Senior Responsible Officers. Three different reviews occur in a program’s lifecycle. These are: a first stage review (conducted before or soon after government approval); a mid-stage review (assesses program execution); and an end stage review (assesses program closure processes). Reports are treated as confidential. See Department of Finance, Resource Management Guide 106 — Australian Government Assurance Reviews [Internet], Finance, Canberra, 2017, p. 22, available from https://www.finance.gov.au/sites/default/files/2019-10/RMG-106R.pdf [accessed 25 October 2020].
109 A green/amber is assigned when ‘successful delivery of the program to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery’.
110 A green rating is assigned when ‘successful delivery of the program to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly’.
111 Actions taken by the assessed entity in response to Gateway Review recommendations since the last point of review are reported in an annex to the new review report.
112 Following a recommendation from the 2017 Gateway Review, a set of short project plans for each work stream was provided to the TPB.
113 Guidance issued to project areas in August 2019 stated that for Year 3, the VCR would ‘use a blend of program and project management methodologies and principles, depending on the individual requirements of a project’.
114 This methodology is referred to as the ‘waterfall’ approach because each stage of the project cascades to the next in a defined sequence.
115 Sometimes referred to as the Scaled Agile Framework (SAFe), the methodology is considered suitable for projects where solutions cannot be fully defined at the outset.
116 A project plan establishes an implementation pathway, allowing project outputs to be measured with reference to an agreed scope, timeline and cost.
117 Year 2 work commenced July 2018. A project management plan was absent between August 2018 and March 2019.
118 High and very high impacts include: those that impact on the department; threats to the achievement of the program/project’s objectives; and greater than 20 per cent cost impact or excessive impact on schedule.
119 High-level reform objectives were: simpler and digitally enhanced experience with DVA; successful transition from the ADF; improved mental and physical health, and well-being for veterans and families; and an ICT platform that mitigates risk and improves service delivery (see paragraph 1.25 and Figure 1.3 of this report). The key business outcomes detailed in the VCR business cases are detailed at paragraphs 2.16 to 2.18.
120 Some uncompleted projects were re-baselined for continuation in the following year, or reprioritised from scope as a VCR deliverable. In Years 2 and 3, around 45 per cent of projects were re-baselined and around 40 per cent of projects were reprioritised from scope. As an example, delays with the commencement of Year 2 telephony work packages (due to a shortage of DVA and Services Australia project resources, vendor delays and quality issues) meant not all 1800VETERAN project deliverables could be completed in that year. These were assigned to a backlog of work for Year 3.
121 In 2019, Year 1 and 2 work streams were reorganised into new streams of work, with reporting reflecting the agile project management approach. These changes complicate an assessment of work across years, as a number of later work streams cannot be clearly linked to earlier work streams.
122 In this report, the ANAO refers to work streams and projects as general terms to describe different levels of work organisation and implementation. In Years 3 and 4, DVA adopted a new project management methodology. The term ‘work stream’ is used instead of ‘Epic’, and ‘project‘, instead of ‘feature’, to be consistent in describing the organisation of work. It is acknowledged that these terms may not be fully equivalent.
123 Some projects were of short duration.
124 This three year period was between 2017–18 and 2019–20.
125 The fourth year of VCR program implementation is over 2020–21. As at February 2021, performance ratings were available to October 2020.
126 Impacts include: a need to redeploy staff to Services Australia; deferment of the release of the Students and Income Support system due to changed risk tolerances; and reduced project activity for digitisation and telephony (call recording) due to restrictions imposed by working from home arrangements.
127 Percentages represent the total proportion of amber and red ratings for a work stream.
128 The Proactive Interventions initiative is also termed ‘Priority Investment Approach — Veterans’ (PIA-V).
129 The four red ratings were from July to October 2020.
130 As funding was only provided for one year, deliverables for 2017–18 were designed to ‘stand-alone’. In scope were the development of: a new client portal; client registration; and a payments template for veteran claims processing and student assistance.
131 Decisions to defer the release were taken in March, May and August 2020.
132 The work stream is rated amber due to risks associated with the releases planned for the first half of 2021.
133 A new national archiving policy came into effect on 1 January 2021. DVA advised the ANAO in February 2021 that National Archive Australia changes to standards for digital record-keeping have led to longer digitisation processing times to ensure records meet the new standards. The department has consequently not been able to close repositories, when originally planned. See National Archives Australia, Building trust in the public record [Internet], NAA, Canberra, 2021, available from https://www.naa.gov.au/information-management/information-management-policies/building-trust-public-record-policy [accessed 15 February 2021]. Operating restrictions linked to the COVID-19 pandemic included the pausing of digitisation during the Melbourne lock-down period and reduced staffing levels.
134 These assumptions related to the expected tempo of digitisation and closure of repositories.
135 Reporting in 2017 generally did not explain the reasons for adverse ratings. In 2019 and 2020, not all Epics are reported each month, and information provided does not always clearly explain the reasons for adverse ratings.
136 This was a cause of delay for several pilot projects in 2018.
137 An example of this is the transfer of certain personnel information from the Department of Defence to DVA under the Early Engagement Model.
138 This was most evident at the commencement of the VCR program, and in 2020.
139 As an example, the majority of PCRs are submitted for the purposes of adjusting the scope of projects, however DVA advised scope changes are mainly due to schedule delays.
140 The 2019 Terms of Reference for the TPB state that its primary objective is ‘to ensure the successful implementation of the DVA Putting Veterans and their Families First Program’.
141 Not all projects are able to be tracked from one year to the next. The audit team found that summaries did not provide updates on all projects; nor indicate those which had been closed or discontinued, or had been merged with another project.
142 Public Governance, Performance and Accountability Act 2013 (Cth), section 17.
143 The Second Pass Business Case stated that DVA would be accountable for the success of reform.
144 See also Auditor-General Report No.30 2019–20 Bilateral Agreement Arrangements between Services Australia and Other Entities, paragraph 1.130.
145 The five year arrangement was for corporate ICT infrastructure services. The arrangements were not legally binding and had no commercial status.
146 Value for money is determined by: encouraging competition; using public resources in an efficient, effective, economical and ethical manner; and ensuring accountable and transparent decision-making. Value for money may be reduced through poor management of risk. The Commonwealth Procurement Rules (CPRs), issued by the Minister for Finance under subsection 105B (1) of the PGPA Act, govern how entities buy goods and services.
147 Requirements are a documented specification of a need which is used to inform the design, build and testing of a process, system or service.
148 DVA cited concerns about potentially adverse impacts of poor systems performance on veteran clients during the COVID-19 pandemic as a key reason for deferring its release.
149 Risks associated with releases typically relate to technical, staff and client issues.
150 A ‘Joint Decommissioning Strategy and Roadmap’ is expected to be developed by March 2021.
151 The business case referred to Recommendation 4 of an audit of shared services arrangements between the departments of Employment and Education and Training. The audit recommended: transparent and sound processes for agreeing performance and cost parameters with clients, and for monitoring and reporting performance against these. See Auditor-General Report No.25 2016–17 The Shared Services Centre, paragraph 4.30.
152 This includes administrative matters, such as information management and cyber security. Service schedules specify standard services, performance measures and responsibilities of each party.
153 DVA ICT was transitioned to Services Australia between July 2017 and June 2018. While specific capabilities have been delivered under the VCR program, Services Australia has not yet retired an entire system, as partial functionalities remain.
154 This was recognised via a Memorandum of Understanding (MoU) on the Cooperative Delivery of Care signed in 2016, and refreshed in December 2019. Governance arrangements include an annual meeting between: the Secretary of DVA; the Secretary of Defence; the Chief of the Australian Defence Force; and a Band 3 level Defence/DVA Links Steering Committee (DLSC).
155 Productivity Commission, A Better Way to Support Veterans No. 93 [Internet], Productivity Commission, Canberra, 2019, p. 314, available from https://www.pc.gov.au/search?collection=productivity-commission-web&form=site-search&query=veteran [accessed 7 October 2020].
156 Defence provides information to DVA about ADF members throughout their career in response to triggers or events. These are: enlistment in the ADF (after 1 January 2016); serious incident leading to administrative discharge; commencement and completion of transition; eligibility due to qualifying service under the VEA; or serious injury.
157 The Defence Single Access Mechanism (SAM) sources records and information from relevant areas within Defence and provides these to DVA in accordance with categories of priority.
158 The DDEIE is a combined initiative between Defence, DVA, the Commonwealth Superannuation Corporation, and Services Australia aimed at allowing system to system requests, search and retrieval of information.
159 This additional funding does not form part of VCR program funding. See Department of Veterans’ Affairs, Interim Government Response to the Report of the Productivity Commission: A Better Way to Support Veterans No. 93 [Internet], DVA, Canberra, 2020, p. 5, available from https://www.dva.gov.au/sites/default/files/files/consultation%20and%20grants/reports/interim-government-response-to-productivity-commission-report-a.pdf [accessed 10 October 2020].
160 A White Card demonstrates entitlement to mental health treatment.
161 Since July 2017, 27,700 new registrations have been made through the Early Engagement Model.
162 The Interim Government Response to the Productivity Commission Report provided $6 million for veteran employment initiatives. This is in addition to VCR program funding. See Department of Veterans’ Affairs, Interim Government Response to the Report of the Productivity Commission: A Better Way to Support Veterans No. 93 [Internet], DVA, Canberra, 2020, pp. 3 and 5, available from https://www.dva.gov.au/sites/default/files/files/consultation%20and%20grants/reports/interim-government-response-to-productivity-commission-report-a.pdf [accessed 10 October 2020].
163 In Year 3, DVA sent 19,020 welcome emails and processed 26,718 new registrations.
164 DVA advised the ANAO that delegates are convened every three months to discuss their approach and processes for claims determinations.
165 Impacts relating to workload, resourcing, responsibilities, organisational functions or team and client relationships, were rated from low to high.
166 These are generally contracted personnel. Changes may involve the commencement of new business functions, technical solutions and processes.
167 Department of Finance, Assurance Review Process — Lessons Learned Benefits Realisation Management [Internet], Finance, Canberra, 2012, available from https://www.finance.gov.au/sites/default/files/2019-11/Benefits_Lessons_Learned_flyer.pdf [accessed 11 November 2020].
168 File digitisation and reduced number of telephone lines were related to Benefits ‘1.2: Faster access to information and support’ and ‘2.1: Improved physical and mental health and wellbeing’. These were linked to Business Outcome C — ‘streamlined assessment and processing’. Benefits associated with faster claim processing through the MyService application were: improved veteran health (2.1); DVA capabilities (3.5); and reduction in unfunded liability (4.1). These were related to Business Outcome B — ‘understanding of veterans’ needs and tailored services’.
169 As examples, the ability to determine financial savings associated with the more efficient use of staffing resources has been hampered by poor workforce data. An absence of indicators of success in Year 3 impeded tracking of ‘4.2: Lower demand or supply of goods and services,’ while ‘2.1: Improved physical and mental health and wellbeing’ lacked both indicators and relevant data.
170 A logic model describes how specific policy initiatives and/or project activities will lead to changes that directly contribute to short, medium and long-term outcomes. See Department of Finance, Resource Management Guide 131 — Developing Good Performance Information [Internet], Finance, Canberra, available from https://www.finance.gov.au/government/managing-commonwealth-resources/developing-good-performance-information-rmg-131 [accessed 29 October 2020].
171 In September 2020, DVA conducted a workshop to link specific initiatives to the program’s broader reform goals. This included an assessment of its achievement of benefits against commitments made to government in Year 3 (2019–20 Business Case).
172 In 2017, DVA planned to use benefits management software to monitor and report against KPIs to enable it to identify which projects were performing well, which projects were underperforming, and which projects DVA should invest in to maximise benefits. DVA advised the ANAO that this approach was not implemented due to cost constraints.
173 Savings are detailed in costing agreements which show the financial impact of a new policy proposal. These are finalised with the Department of Finance before being incorporated into Cabinet submissions. DVA is required to deliver savings of $367 million over four years. No departmental or administered savings were required in 2016–17 and 2017–18. In 2018–19, DVA was required to deliver $29.7 million in departmental savings and $30.3 million in administered savings. In 2019–20, DVA is required to deliver $91.4 million in departmental savings and $215.6 million in administered savings. These figures include the relevant Budget year and the forward estimates.
174 These were the development of a new procurement arrangement for medico-legal reports to support more efficient and effective claims processing; and the transitioning of the provision of care for a small number of veterans from hospital settings to community-based care or residential aged care. Savings are not required in 2020–21 for two additional measures (Aids and Appliances, and Improved Treatment Pathways — Rehabilitation).
175 The Operational Performance Improvement team was responsible for: working with DVA business areas to identify opportunities for standardisation and efficiencies in claims processing; on-base support to ADF Special Forces; delivery of health and rehabilitation services; and the preparation of medical and legal reports. It was estimated that process changes would result in some reduction in staff effort, with 15 per cent of savings realised after two years across all divisions, except the corporate division.
176 DVA has also commissioned work to undertake a full review of benefits and savings from Years 1 to 3, as well as the likely profile of benefits from Year 4 forward.
177 A proxy measure is an indirect measure of activity which is strongly correlated with the activity, for example, when data is not available, or when the cost of gathering and analysing information outweighs the potential benefits of collecting and reporting on it. See Department of Finance, Resource Management Guide 131 — Developing Good Performance Information [Internet], Finance, Canberra, p. 19, available from https://www.finance.gov.au/government/managing-commonwealth-resources/developing-good-performance-information-rmg-131 [accessed 29 October 2020].
178 In 2019–20, departmental funding of $29.2 million was allocated over the forward estimates to enable the implementation of savings measures. In 2018–19, the Operational Performance Improvement team was allocated $6.8 million to identify efficiencies. DVA has not considered this investment in determining the total achieved savings.
179 The ANAO interviewed nine members of veterans’ organisations. These interviews were facilitated by DVA. The information presented in Appendix 7 may not reflect the views of the broader veteran community.
180 See paragraph 15(1)(a) of the Public Governance, Performance and Accountability Act 2013 (Cth). Under the Act, the Accountable Authority of an entity must promote the proper use and management of public resources. Proper means efficient, effective, economical and ethical use of resources.
181 The Commonwealth Performance Framework requires entities to prepare Portfolio Budget Statements (PBS), Corporate Plans, and annual reports. Department of Finance, Resource Management Guides 131, 132 and 134 [Internet], Finance, Canberra, 2019, available from https://www.finance.gov.au/government/managing-commonwealth-resources [accessed 7 October 2020].
182 Department of Veterans’ Affairs, Corporate Plan 2019–23 [Internet], DVA, Canberra, 2019, available from https://www.dva.gov.au/documents-and-publications/dva-corporate-plans [accessed 29 October 2020].
183 Department of Veterans’ Affairs, Portfolio Budget Statements 2020–21 [Internet], DVA, Canberra, 2020, available from https://www.dva.gov.au/sites/default/files/final_dva_2020-21_pbs_web_ready.pdf [accessed 20 October 2020].
184 Department of Veterans’ Affairs, Corporate Plan 2020–21 [Internet], DVA, Canberra, 2020, pp. 5, 11 and 17, available at https://www.dva.gov.au/documents-and-publications/dva-corporate-plans [accessed 29 October 2020].
185 In September 2016, the Government provided a response to the Senate Inquiry Report on the mental health of Australian Defence Force (ADF) members and veterans. The Government agreed in its response to provide an annual Ministerial statement to Parliament on key issues impacting the veteran community and to measure the performance of DVA. These statements have been delivered each year from 2017, with the most recent statement delivered on 10 November 2020. See Australian Government, Australian Government Response to the Foreign Affairs, Defence and Trade Committee Report — Mental health of Australian Defence Force Members and veterans [Internet], Canberra, 2016, p. 19, available from https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Foreign_Affairs_Defence_and_Trade/ADF_Mental_Health/Government_Response [accessed 20 October 2020].