The audit objective was to assess the effectiveness of the Department of Veterans’ Affairs monitoring and implementation of ANAO and internal performance audit recommendations.

Summary and recommendations

Background

1. Performance audits play an important role in stimulating improvements in the administration and management practices of public sector organisations. Australian National Audit Office (ANAO) performance audits involve the independent and objective assessment of the administration of Australian Government entity programs, policies, projects or activities, including identifying any significant risks to the successful delivery of relevant outcomes. Performance audits initiated by entities with resources under their control—known as ‘internal’ audits—fulfil a complementary role, providing assurance to agency management on the effectiveness of the internal control environment and identifying opportunities for performance improvement.1 The appropriate and timely implementation of audit recommendations agreed by management is an important part of realising the full benefit of an audit.2

2. The Department of Veterans’ Affairs (Veterans’ Affairs) is responsible for providing a range of programs of care, compensation, income support and commemoration for the veteran and defence force communities and their families. In 2015–16, the department will administer $11.8 billion of Commonwealth funds.3 Between 1 July 2011 and 31 December 2014, the ANAO completed five performance audits relating to Veterans’ Affairs and made 12 recommendations. Over the same period, the department has completed 60 internal performance audits which included over 300 recommendations.

Audit objective and criteria

3. The audit objective was to assess the effectiveness of the Department of Veterans’ Affairs monitoring and implementation of ANAO and internal performance audit recommendations. The audit continues a series of ANAO performance audits in recent years on the implementation of audit recommendations by Australian Government entities.4

4. To conclude against the audit objective, the audit examined whether the department’s arrangements for monitoring the implementation of performance audit recommendations:

  • provided adequate visibility and assurance to management regarding the status of recommendations, with appropriate involvement by the audit committee and the internal audit function; and
  • facilitated the appropriate implementation of ANAO and internal audit recommendations in a timely manner.

Conclusion

5. The Department of Veterans’ Affairs internal arrangements for monitoring the implementation of performance audit recommendations performed well against the above audit criteria. The department has fully or partially implemented all the performance audit recommendations in the ANAO sample.5

6. There is room for improvement in some aspects of the existing quarterly reporting process used to inform the department’s Audit and Integrity sub-committee, and the ANAO has made a recommendation in this regard. The recommendation is intended to assist the sub-committee in monitoring the risks to departmental operations pending implementation of open recommendations. It would also enable the sub-committee to have greater confidence that recommendations proposed for closure had in fact been fully implemented.

Supporting findings

Governance arrangements

7. Governance arrangements for monitoring the implementation of performance audit recommendations are well-developed. They facilitate the provision of reasonable assurance to the Secretary as the department’s accountable authority. In particular, clear responsibilities and reporting arrangements have been implemented to guide the work of the audit committee and relevant sub-committee in monitoring implementation.

8. Consideration of how audit recommendations should be implemented was a formal part of the department’s internal audit process. In terms of ANAO recommendations, there was not the same formal process. The department took a number of different approaches to planning the implementation of ANAO recommendations. Adopting an approach similar to the existing internal audit process would improve departmental planning and advisory processes for ANAO recommendations.

9. Supporting departmental systems, including the audit database and quarterly reporting processes, were appropriate for monitoring the progress of relevant recommendations against agreed implementation timeframes. Quarterly reporting on ‘open’ recommendations—that is, where implementation is not complete—would be improved by including clear information on the management of risks by business areas pending implementation. Likewise, reporting on recommendations before their closure could be improved by ensuring that the summary reports address all relevant actions taken to fully implement the recommendations. The ANAO has made a recommendation on these matters.

10. There also remain opportunities to enhance the operations of the audit committee and sub-committee in respect to monitoring the implementation of recommendations. In particular, there would be benefit in developing guidelines, in consultation with the sub-committee, setting out when a management representative should be invited to a meeting of the sub-committee or audit committee to discuss implementation delays.

Implementation of recommendations

11. Seventy one per cent of the performance audit recommendations sampled by the ANAO had been adequately implemented by the department, with 29 per cent partially implemented. This result was slightly better than the average of the eight entities previously audited by the ANAO.

12. On average, the department took 311.3 days to implement ANAO audit recommendations and an average of 216.6 days to implement internal audit recommendations. This is superior to the implementation times of other Commonwealth entities examined by the ANAO in 2013–14 and 2014–15.

13. For both ANAO and internal audits, around 75 per cent of recommendations were fully implemented after the original estimated time. Of the ANAO recommendations, 50 per cent were ‘overdue’ by less than three months and 25 per cent by more than three months. For internal audit recommendations, the equivalent figures were 43 per cent and 34 per cent respectively.

Recommendation

Recommendation No. 1

Paragraph 2.22

To increase the level of assurance provided to the Audit and Integrity sub-committee, the ANAO recommends that the department:

  • provide the sub-committee with clear information on the management of risks to departmental operations pending the implementation of open performance audit recommendations which are rated as a high or medium priority; and
  • in proposing to close a recommendation, ensure that the summary of implementation actions provided to the sub-committee addresses all relevant actions taken to fully implement a performance audit recommendation.

Department of Veterans’ Affairs response: Agreed

Summary of entity response

14. The Department of Veterans’ Affairs summary response to the proposed report is provided below, while its full response is at Appendix 1.

The Department of Veterans’ Affairs was pleased with the results of the audit and thanks the Australian National Audit Office for the opportunity to respond to the issues raised.

The recommendation from this audit will be used to bring about improvements in audit processes.

1. Background

Introduction

1.1 Performance audits play an important role in stimulating improvements in the administration and management of public sector entities as well as providing independent assurance to Parliament on the administration of programs. Recommendations in audit reports highlight actions that are expected to improve entity performance when implemented. The appropriate and timely implementation of audit recommendations agreed by management is an important part of realising the full benefit of an audit.6

1.2 Primary responsibility for implementing agreed audit recommendations generally lies with senior managers in the business area of the entity that was subject to the audit. Successful implementation of audit recommendations requires strong senior management oversight and implementation planning to set clear responsibilities and timeframes for addressing the required action.7 Implementation planning should involve key stakeholders, including the internal audit function.

1.3 Audit committees, through their position in an entity’s governance framework8, also have an important role in monitoring the implementation of recommendations. Audit committees assist the accountable authority to ensure that the anticipated benefits of audit reports are realised, through the effective and timely implementation of recommendations. The audit committee’s role in maintaining oversight of implementation is supported by the entity’s internal audit function, including by monitoring and providing advice on management’s progress in implementing recommendations.

1.4 This audit examines the Department of Veterans’ Affairs (Veterans’ Affairs or the department) systems for monitoring the implementation of recommendations arising from ANAO performance audits—often referred to as ‘external’ audits—and the department’s internal performance audits.

Veterans’ Affairs framework for managing recommendations

1.5 As required by the Public Governance, Performance and Accountability Rule 2014, the functions of the department’s Audit and Risk Committee (the audit committee), are set out in a written Charter, approved by the Secretary in July 2014. Under the Charter, the audit committee is to provide independent assurance to the Secretary in relation to both internal and ANAO audit matters, including the monitoring of management’s implementation of audit recommendations and advising the Secretary on action to be taken on ‘significant issues’ raised in relevant audit reports.

1.6 The Charter also provides for the establishment of sub-committees to assist the audit committee in carrying out its responsibilities. In relation to internal and ANAO performance audits, this function is performed by the Audit and Integrity sub-committee (the sub-committee), whose Chair is also an independent member of the audit committee.

1.7 Under the sub-committee’s terms of reference, it is to provide independent assurance to the audit committee on audit matters, by monitoring management’s implementation of audit recommendations and advising the audit committee on any action to be taken on ‘significant issues’ raised in relevant audit reports. In consequence, the sub-committee undertakes the monitoring function, providing a brief status report on the progress of the implementation of audit recommendations at each audit committee meeting. The status report, amongst other issues, seeks the audit committee’s endorsement of the sub-committee’s proposals for the ‘closure’ of recommendations on the basis they have been implemented.

1.8 In relation to their responsibilities for monitoring the implementation of audit recommendations, both the audit committee and sub-committee are supported by departmental officials (the ‘Assurance Team’) from the Legal Services, Assurance and Deregulation Branch. The department’s contracted internal auditors9 also work with the Assurance Team to support the monitoring effort. The monitoring process is underpinned by a system of quarterly reporting to the Assurance Team by departmental business areas.

1.9 Figure 1.1 illustrates Veterans’ Affairs framework for managing audit recommendations.

Figure 1.1: Department of Veterans’ Affairs framework for managing audit recommendations

Source: ANAO analysis of Veterans’ Affairs information.

Parliamentary interest and previous audits

1.10 In recent years, the Joint Committee of Public Accounts and Audit (the JCPAA) and other parliamentary committees have expressed interest in the performance of Australian Government entities in relation to implementing audit recommendations. For example, the JCPAA noted that:

… the purpose of internal and external auditing is to identify weaknesses and better enable an organisation to address risk. The benefits of this work are undermined if agencies do not institutionalise robust monitoring, implementation, reporting and oversight mechanisms … the Committee continues to support the strategic use of ‘follow-up audits’, as part of the ongoing process of improving agency performance.10

1.11 In response to this feedback, the ANAO has conducted a series of performance audits on the implementation of audit recommendations. In the past three years, the ANAO has completed the following audits:

  • ANAO Audit Report No.8 2014–15 Implementation of Audit Recommendations which examined the Department of Health’s implementation of ANAO performance audit and internal audit recommendations;
  • ANAO Audit Report No.34 2013–14 Implementation of ANAO Performance Audit Recommendations which was a cross-agency audit that examined the implementation of ANAO performance audit recommendations by two agencies: the Department of Human Services; and the Department of Agriculture;
  • ANAO Audit Report No.25 2012–13 Defence’s Implementation of Audit Recommendations which examined the Department of Defence’s implementation of ANAO performance audit and internal audit recommendations; and
  • ANAO Audit Report No.53 2012–13 Agencies’ Implementation of Performance Audit Recommendations which was a cross-agency audit that examined the implementation of ANAO performance audit recommendations by four agencies: the Department of Education, Employment and Workplace Relations; the Department of Families, Housing, Community Services and Indigenous Affairs; the Department of Finance and Deregulation; and the Department of Infrastructure and Transport.

1.12 This series of ANAO performance audits has highlighted that a structured and planned approach to the oversight and implementation of audit recommendations assists entities to manage timeliness, completeness and adequacy of implementation, by allowing progress to be clearly targeted and monitored.11

Audit objective, criteria and scope

1.13 The audit objective was to assess the effectiveness of the Department of Veterans’ Affairs monitoring and implementation of ANAO and internal performance audit recommendations.

1.14 To conclude against the audit objective, the audit examined whether the department’s arrangements for monitoring the implementation of performance audit recommendations:

  • provided adequate visibility and assurance to management regarding the status of recommendations, with appropriate involvement by the audit committee and the internal audit function; and
  • facilitated the appropriate implementation of ANAO and internal audit recommendations in a timely manner.

1.15 To assess the timeliness of Veterans’ Affairs implementation of the recommendations, the ANAO examined all 12 ANAO recommendations and 236 internal audit recommendations from performance audit reports finalised between 1 July 2011 and 31 December 2014 that were recorded as implemented (closed) by the department as at 30 June 2015.

1.16 The 12 ANAO performance audit recommendations12 and a sample of 12 closed internal audit recommendations were also analysed in detail to assess the adequacy of implementation. The result of this analysis was compared against the results for the eight other Commonwealth entities examined in this series of ANAO audits.

1.17 In conducting the audit, the ANAO met with the Chairs of the audit committee and sub-committee, the Veterans’ Affairs Chief Audit Executive, Assurance Team staff, and senior members of the contracted internal auditors. The ANAO also reviewed key documentation related to the monitoring of audit recommendations, including audit committee and sub-committee papers, the department’s audit database, and documentation provided to evidence the implementation of a sample of closed ANAO and internal audit recommendations.

1.18 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of approximately $230 000.

2. Governance arrangements

Areas examined

This chapter examines the governance arrangements that the Department of Veterans’ Affairs has in place to monitor the implementation of performance audit recommendations.

Conclusion

The Department of Veterans’ Affairs internal arrangements for monitoring the implementation of performance audit recommendations performed well against the audit criteria.

Recommendation

The ANAO has made one recommendation aimed at increasing the level of assurance provided to the department’s Audit and Integrity sub-committee relating to the implementation of performance audit recommendations.

Introduction

2.1 Governance refers to the arrangements and practices which enable an organisation to set direction and manage its operations to achieve expected outcomes and discharge its accountability obligations. Veterans’ Affairs governance arrangements for managing the implementation of internal and ANAO performance audit recommendations are supported by the department’s Legal Services, Assurance and Deregulation Branch—particularly the Branch’s Assurance Team and the contracted internal auditors. The department’s Audit and Integrity sub-committee (the sub-committee) has direct oversight of the progress made by the department in implementing audit recommendations. The sub-committee reports to the Audit and Risk Committee (the audit committee), which has ultimate responsibility for providing independent assurance to the Secretary on audit matters.

Have clear responsibilities and reporting arrangements been established for monitoring implementation?

Governance arrangements for monitoring the implementation of performance audit recommendations are well-developed. They facilitate the provision of reasonable assurance to the Secretary as the department’s accountable authority. In particular, clear responsibilities and reporting arrangements have been implemented to guide the work of the audit committee and relevant sub-committee in monitoring implementation.

2.2 Audit committees do not displace or change the management and accountability arrangements within entities, but are intended to enhance the existing governance framework, risk management practices and control environment, by providing independent assurance and advice.13

2.3 Figure 2.1 shows the responsibilities of the Veterans’ Affairs audit committee in relation to internal and external (ANAO) audits, including monitoring the implementation of both internal and ANAO audit recommendations. The scope of the audit committee’s responsibilities in relation to such audits is consistent with the requirements set out in section 17 of the Public Governance, Performance and Accountability Rule. The sub-committee’s responsibilities, which are set out under its terms of reference, largely mirror those of the audit committee. The three-member sub-committee reports to the audit committee and is chaired by an independent external member of the audit committee.14

Figure 2.1: Audit and Risk Committee responsibilities

Source: Department of Veterans’ Affairs, Audit and Risk Committee Charter.

2.4 The sub-committee generally meets quarterly, with an update of the progress of the implementation of audit recommendations as a standing item at each meeting. The sub-committee considers a report provided by the Assurance Team summarising the status of each open recommendation.15 The report includes proposals for closing recommendations on the basis that implementation is complete, as well as applications for a formal extension of implementation timeframes. The audit committee’s meetings are held quarterly16, usually about two weeks after the sub-committee meeting. At the audit committee, a standing item is a report from the sub-committee, which includes a summary of the sub-committee’s proposals regarding relevant audit recommendations and a request that the audit committee endorse those proposals.17

2.5 Consistent with its Charter, the audit committee provides a short written annual report to the Secretary. The 2014 report observed that one of the key themes identified from internal audit reports was that ‘implementation of agreed audit recommendations is variable and often impacted by changes in responsible officers’. The report concluded that ‘overall [Veterans’ Affairs] open audit recommendations are being adequately managed.’18 The audit committee’s Charter also requires it to provide a report to the Secretary after each committee meeting on significant issues and outcomes. In order to meet this requirement, the Chair of the audit committee advised the ANAO that he meets with the Secretary within two weeks of every audit committee meeting. Following audit committee meetings, the department’s Executive Management Board also receives a briefing on implementation of recommendations. Such briefings focus on recommendations where implementation has been delayed, or where actions by the business area19 may not adequately address the intent of the recommendation.

Do departmental processes for managing audits include planning for implementation of audit recommendations?

Consideration of how audit recommendations should be implemented was a formal part of the department’s internal audit process. In terms of ANAO recommendations, there was not the same formal process. The department took a number of different approaches to planning the implementation of ANAO recommendations. Adopting an approach similar to the existing internal audit process would improve departmental planning and advisory processes for ANAO recommendations.

2.6 Previous ANAO performance audits have highlighted that the successful implementation of audit recommendations requires strong senior management oversight and implementation planning that sets clear responsibilities and timeframes for addressing the required action.20

2.7 Within Veterans’ Affairs, the primary responsibility for the implementation of audit recommendations rests with the relevant business area. The content of internal audits is discussed by business area senior managers and the relevant internal audit team at ‘findings and recommendations’ workshops.21 Key issues considered at the workshops are: the general findings; the rating to be applied to the findings22; and indicative recommendations. The agreed implementation actions and timeframes to implement recommendations appearing in internal audit reports are formally signed-off at First Assistant Secretary level.23,24 Audit reports contain the specific actions that the business area(s) have committed to undertake to implement the recommendations, as well as a date by which actions are anticipated to be completed. As such, the final reports effectively contain a summary implementation strategy and overall this process is consistent with better practice.25

2.8 In contrast, the department did not consistently identify implementation timeframes for the external (ANAO) audit recommendations examined in this audit prior to finalisation of the relevant audit. Adopting an approach similar to the existing internal audit process would improve departmental planning and advisory processes for ANAO recommendations. In particular, information on estimated implementation timeframes for each proposed ANAO recommendation and any risks to implementation within the relevant timeframe could usefully be included.

2.9 For both internal and ANAO audit recommendations, information on the status of implementation, including an anticipated timeframe for completion, is collected as part of the quarterly update process undertaken by the Assurance Team in preparation for the sub-committee meetings. For ANAO recommendations, particularly where implementation involves more complex issues such as potential legislative changes, more detailed implementation strategies may not be developed until some months after the relevant ANAO report is finalised.

Are appropriate systems in place for monitoring the implementation of recommendations?

Supporting departmental systems, including the audit database and quarterly reporting processes, were appropriate for monitoring the progress of relevant recommendations against agreed implementation timeframes. Quarterly reporting on ‘open’ recommendations—that is, where implementation is not complete—would be improved by including clear information on the management of risks by business areas pending implementation. Likewise, reporting on recommendations before their closure could be improved by ensuring that the summary reports address all relevant actions taken to fully implement the recommendations. The ANAO has made a recommendation on these matters.

There also remain opportunities to enhance the operations of the audit committee and sub-committee in respect to monitoring the implementation of recommendations. In particular, there would be benefit in developing guidelines, in consultation with the sub-committee, setting out when a management representative should be invited to a meeting of the sub-committee or audit committee to discuss implementation delays.

Audit recommendations database

2.10 While the implementation of audit recommendations is a management responsibility, an entity’s internal audit function is well placed to monitor progress.26 Effective monitoring requires a system that accurately tracks progress and records the actions of the business area responsible for progressing action against timeframes.

2.11 Once internal audit reports are finalised, they are formally tabled at the audit sub-committee’s quarterly meetings.27 ANAO audits involving Veterans’ Affairs are tabled at both audit committee and sub-committee meetings following their tabling in Parliament. These audits, and the relevant recommendations, are then entered into a customised database (the audit database) developed by the department’s contracted internal auditors. The database contains records for both ANAO and internal performance audits since 2000. For each audit report, the database contains:

  • the individual recommendations;
  • the rating (critical recommendation 1 (CR1), CR2, CR3 or business improvement recommendation) applying to each recommendation;
  • the business area responsible for implementation;
  • the original business area response or comment on the recommendation, including agreed implementing actions;
  • the original estimated timeframe for implementation;
  • for CR recommendations, any agreed extensions of time for implementation; and
  • for CR recommendations, whether the recommendation has been implemented, and if so, the month and year that the recommendation was closed.

2.12 For CR recommendations, the successive quarterly updates from the business area against each recommendation are also included in the audit database. At least for the more recent records28, the update is accompanied by a brief commentary by the Assurance Team, including whether progress towards implementation is progressing satisfactorily. The database is also capable of generating a range of reports.

Reporting processes

2.13 As previously noted, the sub-committee reviews the status of recommendations at its quarterly meetings and then makes proposals to the audit committee for closures and extensions of time. As the initial step in this process, the Assurance Team email a list of open audit recommendations to the relevant business areas for review and updating.29 Table 2.1 outlines the key steps in the reporting process.

Table 2.1: Key steps in reporting implementation progress to the audit sub-committee and audit committee

Timeline

Action

Description

6 weeks before sub-committee meeting

Audit follow-up.

Assurance Team

Request quarterly update from business areas on progress towards implementation of recommendations.

2–4 weeks before sub-committee meeting

Review and analysis of business area updates.

Assurance Team/Internal Auditors

Updates, including extension of time requests, are reviewed.

Further follow-up/ Investigation.

Assurance Team

Clarification or additional information is requested from business areas as necessary.

1–2 weeks before sub-committee meeting

Report produced for the sub-committee.

Assurance Team

Report provided to the sub-committee which contains edited business area update summaries and Assurance Team proposals for recommendation closures and extensions of time for completion.

Sub-committee meeting

Sub-committee considers proposals for recommendation closures, extensions of time requests and any other matters as it sees fit.

Assurance Team

Sub-committee decisions and any relevant commentary recorded in meeting minutes and used to generate report for consideration by audit committee.

1–2 weeks before audit committee

Report provided to the audit committee.

Assurance Team

Report provided to the audit committee which contains proposals, as endorsed by the sub-committee, for recommendation closures and extensions of time.

Audit committee meeting

Audit committee considers sub-committee’s proposals for recommendation closures and extensions of time requests; endorsing these as appropriate.

Assurance Team

Audit database updated to reflect audit committee decisions, including closure of relevant recommendations.

Source: ANAO analysis of Veterans’ Affairs information.

2.14 The reporting arrangements have been refined over time to improve the quality of the quarterly reporting received from business areas. In particular, changes were made in mid-2014 to provide more detail in relation to how the risks associated with open recommendations were being managed by business areas while action on the recommendations was incomplete. A separate ‘extension request’ process was also developed for cases where the previously agreed date for completing implementation action on a recommendation would not be met.30 Under the revised arrangements for extensions of time, the senior manager of the relevant business area is required to provide details of the reasons for the delay in action on the recommendation and provide a new target date. The process also requires the manager to confirm that they are satisfied that the risk of not implementing the recommendation by the previously agreed date is acceptable, and confirm that the action will be implemented by the revised date.

Quarterly Reporting

2.15 Departmental policy is that the quarterly business area updates, like the extension of time requests, are required to be approved by the senior manager of the responsible area. In the case of Divisions (which make up the majority of the department’s business areas), this is a First Assistant Secretary. The updates are generally provided to the Assurance Team by Divisional Coordinators. Variances in record keeping practices meant that in some cases evidence of the required approval was not retained by the business areas in either hard or electronic format and the department could not provide evidence of full compliance with departmental policy.

2.16 Examination of the quarterly business area updates by the ANAO shows that a minority of the updates did not always contain sufficient information, including the necessary information to enable the Assurance Team and internal auditor to determine whether implementation was complete or not. In such cases the Assurance Team sought clarification or further information from the business area via email or phone. Based on comparing the original updates with the revised updates, this follow-up process was successful in obtaining additional information in order to provide a reasonable level of assurance in reports provided to the sub-committee.

2.17 The reports provided to the sub-committee are accompanied by a covering brief that highlights, amongst other things, the number of open recommendations that are more than 12 months old and the number of open recommendations for each business area, all identified against the CR rating. This assists the sub-committee in readily identifying any issues of concern.

2.18 For the 31 recommendations proposed for closure at the May 2015 sub-committee meeting, the relevant business area had provided the Assurance Team with TRIM31 reference numbers to documentation providing evidence of implementation actions for 13 recommendations. For 10 of these 13 recommendations, the report provided to the sub-committee noted that the Assurance Team ‘sighted’ the relevant evidence. The ANAO was able to locate the relevant evidentiary documents in the nominated TRIM files for 12 of the 13 recommendations.32 Requiring the inclusion of TRIM reference numbers for all recommendations that are being considered for closure would: assist the Assurance Team to more readily verify the actions being taken to implement recommendations33; and increase the level of assurance provided to the sub-committee and through it to the audit committee.

2.19 A key area in which the quarterly reports provided to the audit sub-committee could be improved is commentary on risk management of open recommendations. While this was a significant component of changes made to the reporting process from 2014, the most recent quarterly reports (May and August 2015) contain limited information on how the responsible business area was managing relevant risks to departmental operations pending implementation of the relevant recommendation. This was in spite of the Assurance Team specifically reminding business areas, before the August 2015 sub-committee meeting, of the need to address risk management issues in their quarterly updates.

Reporting on recommendations before their closure

2.20 The ANAO also reviewed the reporting for those recommendations which it assessed as only partially implemented when closed.34 It is important that the reports provide an adequate basis for advising the audit sub-committee that the recommendations should be closed. Premature closure of recommendations reduces the ability of the sub-committee and the audit committee to adequately identify and monitor risks or identify the need for remedial action where a business area is not making adequate or timely progress in its implementation of a recommendation.

2.21 Two recommendations examined as part of the ANAO sample identified that the advice to the audit sub-committee (as recorded in the Veterans’ Affairs audit database) did not fully address the specific actions to implement the recommendations. For example, one internal audit recommended the development of seven performance measures.35 The business area’s advice to the sub-committee stated that ‘improvements for reporting [Military Rehabilitation and Compensation Act] matters have been implemented’ and the recommendation was closed on this basis. However, during this audit, the department acknowledged that its systems failed to capture the necessary information to report on four of the seven measures to be implemented. To provide additional assurance to the sub-committee where a recommendation is being considered for potential closure, there would be benefit in the business area ensuring that its quarterly update report—which forms the basis of advice to the sub-committee—explicitly address whether it has undertaken all the actions necessary to fully implement the recommendation.36 Where changes of policy or other developments mean that actions as originally contemplated cannot be carried out or may be delayed, any proposal for closure should note these circumstances so the sub-committee is fully informed, and can appropriately advise the audit committee and through it, the accountable authority.

Recommendation No.1

2.22 To increase the level of assurance provided to the Audit and Integrity sub-committee, the ANAO recommends that the department:

  • provide the sub-committee with clear information on the management of risks to departmental operations pending the implementation of open performance audit recommendations which are rated as a high or medium priority; and
  • in proposing to close a recommendation, ensure that the summary of implementation actions provided to the sub-committee addresses all relevant actions taken to fully implement a performance audit recommendation.
  • Entity response: Agreed

2.23 Business areas have been specifically reminded to clearly detail the management of risks for all open audit recommendations when responding to the quarterly follow-up report.

2.24 Where business areas have completed remediation, actions are clearly detailed as well as the TRIM reference for the location of evidence to support closure. Documentation is checked by Audit Management staff and is clearly noted to give assurance to committee members that evidence has been sighted.

2.25 A Businessline will be distributed to Divisional management outlining these processes. This will ensure the committee has clear oversight of the status of all recommendations and assurance that business areas are fully remediating issues detailed in recommendations prior to closure.

The audit committee and sub-committee

2.26 A distinguishing feature of an audit committee within an entity’s governance framework is its potential for objectivity, as audit committees do not undertake management responsibilities. An effective agency system for implementing audit recommendations will be supported by an audit committee that monitors management’s implementation of audit recommendations, prioritising recommendations that are overdue or that pose significant risk or exposure to the department.

Considering implementation progress reports

2.27 The ANAO does not normally attend the sub-committee meetings. For the purposes of the current audit, the audit team attended the May 2015 meeting as observers. At that meeting, the implementation of audit recommendations report was considered in detail, with the sub-committee reviewing the status of each individual recommendation before adopting the Assurance Team’s proposals for the closure of specific recommendations, and the granting of extensions of time for other recommendations. At the subsequent (June 2015) meeting of the audit committee37, progress in implementing audit recommendations was briefly touched upon by the sub-committee Chair when speaking to the sub-committee’s overall written report.

2.28 The ANAO’s review of audit committee and sub-committee minutes for 2013 to 2015 indicates that the committees have been conscious of the need for timely implementation of audit recommendations and placed some emphasis on seeking improvement. Notably, discussions at the sub-committee’s February 2014 meeting, and subsequently at the audit committee, resulted in changes to various procedures to: strengthen the involvement of senior business area managers in finalising internal audit reports; and improve quarterly business area reporting.

Seeking updates direct from business area management

2.29 As part of the process for maintaining strong relationships with entity senior management, audit committees may request management representatives to attend meetings, to facilitate further discussion on action to implement audit recommendations, or to explain why any recommendation has not been addressed appropriately or in a timely way.38

2.30 The ANAO’s attention was directed by Veterans’ Affairs to one such occasion in which a business area senior manager attended an audit committee meeting in June 2012 to report on the status of a recommendation from a 2008–09 internal audit. The audit committee subsequently decided that where an audit recommendation is affected by significant developments, the responsible business area manager should attend the audit committee to discuss the surrounding issues, instead of just reporting through the regular committee reporting process.39

2.31 In reviewing the audit committee and sub-committee meeting minutes from the last two years, the ANAO found only one other instance, at the October 2014 sub-committee meeting, where a business area manager attended a meeting to discuss progress in implementing a recommendation. Over the last three years there have been 16 internal audit recommendations that have been given three or more extensions of time, indicating significant issues with implementation of these recommendations. There would be benefit in developing guidelines, in consultation with the sub-committee, setting out when a management representative should be invited to a meeting of the sub-committee or audit committee to discuss implementation delays.

3. Implementation of recommendations

Areas examined

This chapter examines the extent to which the Department of Veterans’ Affairs has implemented audit recommendations from both ANAO and internal performance audits. It also examines the timeliness of implementation.

Conclusion

The department has fully or partially implemented all the performance audit recommendations in the ANAO sample.

Introduction

3.1 Audit recommendations identify risks to the successful delivery of outcomes consistent with policy and legislative requirements, and highlight actions aimed at addressing those risks, and opportunities for improving entity administration. Entities are responsible for the implementation of audit recommendations to which they have agreed, and the timely implementation of recommendations allows entities to realise the full benefit of audit activity.

Audit approach

3.2 The ANAO assessed Veterans’ Affairs performance in implementing 12 recommendations from five ANAO performance audit reports tabled in Parliament between 1 July 2011 and 31 December 2014. These five reports represented all ANAO audit reports tabled in this period that had recommendations directed towards Veterans’ Affairs.40 The ANAO also examined the implementation of a sample of 12 recommendations from 12 internal performance audit reports finalised during the same period. All ANAO and internal audit recommendations had been recorded as implemented (that is, closed) by the department’s Legal Services, Assurance and Deregulation Branch as at 30 June 2015.

3.3 The ANAO also assessed the timeliness of Veterans’ Affairs implementation of the 12 recommendations from the ANAO reports plus 236 recommendations from 60 internal performance audit reports finalised between 1 July 2011 and 31 December 2014. The 236 recommendations represent all internal audit recommendations made during this period that were also recorded as implemented as at 30 June 201541, with the exception of 70 business improvement recommendations.42

3.4 Table 3.1 summarises the number of audit recommendations assessed by the ANAO for progress in implementation and the timeliness of implementation.

Table 3.1: Recommendations assessed July 2011–December 2014

Recommendations assessed

ANAO

Internal audit

Total

Assessed for implementation

No. of audit recommendations

12

12

24

No. of audit reports

5

12

17

Assessed for timeliness

No. of audit recommendations

12

236

248

No. of audit reports

5

60

65

Source: ANAO.

3.5 The ANAO reviewed entity documentation, interviewed key staff, and extracted data from the department’s audit database to inform its assessment.

Did Veterans’ Affairs implement audit recommendations adequately?

Seventy one per cent of the performance audit recommendations sampled by the ANAO had been adequately implemented by the department, with 29 per cent partially implemented. This result was slightly better than the average of the eight entities previously audited by the ANAO.

3.6 The definitions used to assess the extent to which recommendations were implemented are provided in Table 3.2.

Table 3.2: ANAO’s categorisation of implementation

Category

Explanation

Adequate implementation

The action taken met the intent of the recommendation, and sufficient evidence was provided to demonstrate action taken.

Partial implementation

This category encompasses three considerations:

  • Action taken was less extensive than recommended by the ANAO. Action either fell short of the intent of the recommendation, or only addressed some of the identified risks.
  • The entity may have established a process or procedure to address an issue, however, the specific action noted in the recommendation was not complete at the time of the assessment.
  • The entity may have commenced action to address a recommendation but subsequent policy changes may influence how it might be implemented.

Not adequate implementation

This category encompasses two considerations:

  • There is no supporting evidence that action has been undertaken.
  • The action taken does not address the recommendation.

Source: ANAO.

3.7 Table 3.3 provides a summary of Veterans’ Affairs implementation of 12 ANAO audit recommendations and 12 internal audit recommendations. The audit recommendations have been provided in full at Appendix 3 including the ANAO’s assessment of them. In summary, 17 (71 per cent) of the sampled recommendations were adequately implemented. The remaining seven recommendations (29 per cent) were only partially implemented, despite having been recorded as ‘closed’ in the department’s audit database.

Table 3.3: Overview of the ANAO’s assessment

Implementation category

ANAO recommendations

Internal audit recommendations

Total

Adequate implementation

8 (67%)

9 (75%)

17 (71%)

Partial implementation

4 (33%)

3 (25%)

7 (29%)

Not adequate implementation

0

0

0

Total

12

12

24

Source: ANAO analysis.

3.8 The department performed slightly better in implementing the recommendations from internal audits compared to ANAO recommendations. Eight (67 per cent) ANAO recommendations were adequately implemented, compared to nine (75 per cent) internal audit recommendations.

ANAO recommendations

3.9 Five of the twelve ANAO recommendations were ‘agreed with qualification’ by Veterans’ Affairs. For three of these recommendations, the department took actions that met the intent of the recommendation, and were assessed as being adequately implemented. For the other two recommendations that were ‘agreed with qualification’, while the department took a variety of actions to improve departmental performance (including through staff training, staff reminders and some limited internal quality assurance processes) these did not fully meet the intent of the recommendations. These actions were however broadly consistent with the department’s responses to the recommendations contained in the audit report.43

3.10 In the case of the remaining two ANAO recommendations that were fully agreed by Veterans’ Affairs but were assessed by the ANAO as only being partially implemented, the department had either not fully completed implementation, or some of the intended actions had not proceeded due to uncertainty about whether they would be fully consistent with government policy on performance reporting.44

Internal Audit Recommendations

3.11 In relation to the three internal audit recommendations that the ANAO assessed as partially implemented, the basis for reaching this assessment varied. In one case, there were delays in negotiations with other jurisdictions about obtaining health data and all relevant actions could not be implemented to fully meet the intent of the recommendation. In another instance, involving the development and application of a quality assurance process for departmental procurement activity, a draft quality management framework was put in place by early 2014, but application of the framework remained incomplete as at mid-2015.45

How Veterans’ Affairs compared with other entities

3.12 In recent years the ANAO has conducted a program of performance audits that have assessed the implementation of audit recommendations by eight entities, using the same assessment criteria outlined in Table 3.2. While recognising that individual audit recommendations may vary in their scope and complexity, the ANAO’s published findings provide a basis for comparative assessment of entity performance.

3.13 Figure 3.1 presents the results from the four ANAO audit reports tabled in Parliament to date46, alongside the results of the ANAO’s assessment of Veterans’ Affairs in the current audit. The figure shows the proportion of recommendations that were assessed by the ANAO as adequately, partially or not adequately implemented by the audited entities, including Veterans’ Affairs. In summary, Figure 3.1 indicates that:

  • an average of 65 per cent of recommendations were assessed as adequately implemented across the eight entities, as compared to 71 per cent in Veterans’ Affairs;
  • an average of 24 per cent of recommendations were assessed as partially implemented across the eight entities, as compared to 29 per cent in Veterans’ Affairs; and
  • an average of 11 per cent of recommendations were assessed as not adequately implemented across the eight entities, whereas Veterans’ Affairs had no recommendations in this category.

Figure 3.1: Implementation of audit recommendations by Australian Government entities

Source: ANAO analysis.

Did Veterans’ Affairs implement audit recommendations in a timely manner?

On average, the department took 311.3 days to implement ANAO audit recommendations and an average of 216.6 days to implement internal audit recommendations. This is superior to the implementation times of other Commonwealth entities examined by the ANAO in 2013–14 and 2014–15.

For both ANAO and internal audits, around 75 per cent of recommendations were fully implemented after the original estimated time. Of the ANAO recommendations, 50 per cent were ‘overdue’ by less than three months and 25 per cent by more than three months. For internal audit recommendations, the equivalent figures were 43 per cent and 34 per cent respectively.

3.14 Audit recommendations can vary in scope and complexity, and as a consequence the implementation task may require coordination across a range of business areas within an entity. The risks involved and the time taken to implement recommendations within entities can also vary. If implementation is not progressed promptly, and individual risks remain untreated, the full value of the audit may not be achieved. In this context it is important that the entity’s assurance function and the audit committee keep the accountable authority informed on progress with implementing recommendations that are complex, difficult or overdue.

3.15 Table 3.4 provides an overview of the timeliness of Veterans’ Affairs implementation of 248 recommendations from performance audit reports finalised between 1 July 2011 and 31 December 2014.47 On average, ANAO recommendations took 311.3 days to implement, as compared to 216.6 days for internal audit recommendations.48 This is superior to the implementation times of other Commonwealth entities examined by the ANAO in 2013–14 and 2014–1549 although direct comparisons are difficult as: the relevant information (particularly implementation dates) is often recorded in different ways by entities; and the complexity of the recommendations will vary.

3.16 The average time taken for Veterans’ Affairs to implement internal audit recommendations has shown some improvement over time, dropping from an average of 279 days for recommendations contained in reports published in 2011–12 to 197.1 days for reports published in 2013–14.50,51 Similarly, the average time taken for Veterans’ Affairs to implement ANAO recommendations has dropped from an average of 319.4 days for recommendations contained in ANAO reports tabled in Parliament in 2011–12 to 216.5 days for reports tabled in 2013–14.52

Table 3.4: Timeliness of implementation of audit recommendations

Recommendations

ANAO

Internal audit

Total/average

Average number of days originally estimated to implement recommendations

248.8

110.9

117.4

Average actual number of days to implement recommendations

311.3

216.6

221.2

Number and percentage of recommendations implemented on or before the original estimated implementation date

2 (17%)

45 (18%)

47 (18%)

Number and percentage of recommendations implemented up to three months after the original estimated date (overdue recommendations)

6 (50%)

106 (43%)

112 (42%)

Number and percentage of recommendations implemented over three months after the original estimated date (overdue recommendations)

3 (25%)

84 (34%)

87 (33%)

Average number of extra days that it took for overdue recommendations to be implemented

102.7

144.4

142.4

Total no. of recommendations

12

236

248

Notes: Percentages are rounded to the nearest whole number. The percentages provided in the table do not equal 100 as 1 (or 8 per cent) of the ANAO recommendations and 14 (or 6 per cent) of the internal audit recommendations did not have clear estimated implementation dates.

Source: ANAO analysis.

3.17 Overall, the average time taken to implement recommendations has been considerably longer than originally estimated by the department.53 For both ANAO and internal audits, around 75 per cent of the sampled recommendations were implemented after the original estimated time, although the majority of these completions were ‘overdue’ by no more than three months.54 For overdue recommendations, the extra time taken to complete implementation as compared to the original estimate averaged 102.7 days for ANAO recommendations and 144.4 days for internal audit recommendations.55

3.18 Figure 3.2 provides a more detailed breakdown of the time taken to implement recommendations. Notably, 28 percent of internal audit recommendations were implemented within three months of the report being finalised, a significant factor underpinning the considerably shorter average period to implement internal audit recommendations as compared to ANAO recommendations. For reports published from 1 July 2011 onwards, the longest time taken to implement an internal audit recommendation was 1247 days (3.4 years); for an ANAO recommendation it was 749 days (2.1 years).56

Figure 3.2: Time taken to implement audit recommendations

Source: ANAO analysis.

3.19 Strategies to avoid under-estimating implementation timeframes (where the relevant actions involved multiple business areas or were reliant on IT systems changes) were discussed by the audit sub-committee and audit committee in the first half of 2014. For internal audits, it was decided that from July 2014: business areas should be represented at least at Assistant Secretary level at the findings and recommendations workshops57; and that the agreed implementation actions and associated timeframes to implement the recommendations should be signed-off at First Assistant Secretary level.58 The department’s contracted internal auditors advised the ANAO that the changes were expected to assist in a more accurate estimation of implementation timeframes.

Implementing priority recommendations

3.20 The department applies a useful priority rating scale to all recommendations, so as to ‘communicate [its] importance [to] the audit committee, management and staff of the Department.’ The department moved to a new three-point rating scale in March 2015. Before then, the scale consisted of three main ratings—with the highest priority being ‘CR1’ (critical recommendation 1) followed by CR2 and CR3—and a separate rating for ‘business improvement recommendations’.59 As a matter of policy, Veterans’ Affairs gave a default CR2 rating to ANAO recommendations. Table 3.5 shows that, in relation to internal audit recommendations from reports finalised between 1 July 2011 and 31 December 2014, only three have been rated as CR1, with the remainder fairly evenly divided between CR2 and CR3. The average time to implement the three CR1 recommendations was 182 days.

Table 3.5: Priority rating scale and assessment of the sample recommendations

Priority ratings

Implementation timeframes

ANAO

Internal audit

Total

CR1

Immediate commencement of corrective action.

0

3

3 (1%)

CR2

As soon as practical within the next three to six months.

12

112

124 (50%)

CR3

When resources permit at the discretion of the organisation.

0

121

121 (49%)

Total no. of recommendations

 

12

236

248

Note: The relevant departmental policy document also notes that ‘corrective actions vary considerably in complexity and the complexity of an agreed action influences the timeframe for its implementation. Notwithstanding the need to take prompt corrective action, some aspects of a permanent solution to a CR1/CR2 observation may require more time than the nominal three to six months associated with this level of severity.’

Source: Department of Veterans’ Affairs records.

3.21 Figure 3.3 shows the length of time taken to implement the CR2 recommendations. According to the department’s priority rating scale, moderate recommendations should preferably be implemented within six months. Some 58 per cent of all CR2 recommendations were implemented within the six month timeframe. The proportion implemented within six months falls to 33 per cent if only the ANAO recommendations are counted. The average time to close all CR2 recommendations is 255.3 days. Over time there has been improvement, from an average of 348.5 days for recommendations reported in 2011–12 reports to 206.9 days for recommendations reported in 2013–14 reports. The average figure for 2014–15 (to 31 December 2014) is lower again at 124.5 days, but this will increase once the remaining open recommendations are implemented.

Figure 3.3: Time taken to close CR2 recommendations

Source: ANAO analysis.

3.22 Figure 3.4 shows the length of time taken to implement CR3 recommendations. According to the department’s priority rating scale, CR3 recommendations should be implemented by the business area ‘when resources permit’. A relatively high proportion of recommendations (70 per cent) was implemented within six months, with over half the remaining recommendations implemented within twelve months. Over time there has been improvement, from an average of 214.3 days for CR3 recommendations reported in 2011–12 reports to 187.6 days for CR3 recommendations reported in 2013–14 reports. The figure for 2014–15 (to 31 December 2014) is lower again at 131.9 days, but this will increase once the remaining open recommendations are implemented.

Figure 3.4: Time taken to close CR3 recommendations

Source: ANAO analysis.

Appendices

Appendices

Please refer to the attached PDF of the report for the Appendices:

  • Appendix 1: Entity Response
  • Appendix 2: Classification of audit recommendations
  • Appendix 3: The ANAO’s assessment of implementation

Footnotes

1 ANAO Better Practice Guide—Public Sector Internal Audit: An investment in assurance and business improvement, September 2012, Canberra, p. i.

2 Commonwealth, Joint Committee of Public Accounts and Audit (JCPAA), 13 February 2014, Auditor-General, Opening Statement, p.1.

3 Department of Veterans’ Affairs, 2015–16 Budget Related Paper 1.4B, p. 74.

4 The program to date is outlined in paragraph 1.11 of this audit report.

5 The ANAO examined 12 ANAO recommendations and 12 internal audit recommendations from performance audit reports finalised between 1 July 2011 and 31 December 2014 that were recorded as implemented (closed) by the department as at 30 June 2015. Further details are in paragraphs 3.2–3.4.

6 Commonwealth, Joint Committee of Public Accounts and Audit (JCPAA), 13 February 2014, Auditor-General, Opening Statement, p.1.

7 ibid., p. 2.

8 Section 45 of the Public Governance, Performance and Accountability Act 2013 requires the establishment of an audit committee by the accountable authority. The Public Governance, Performance and Accountability Rule 2014 states that audit committees are intended to provide independent advice and assurance to the accountable authority on the appropriateness of the entity’s accountability and control environment including: financial and performance reporting, system of risk oversight and control, and system of internal control. The ‘accountable authority’ for an Australian Government department is the Secretary.

9 KPMG has held the contract for the provision of internal audit services to Veterans’ Affairs since 2005.

10 Joint Committee of Public Accounts and Audit, Report 443: Review of Auditor-General’s Reports Nos. 23 and 25 (2012–13) and 32 (2012–13) to 9 (2013–14), Canberra, 2014, paragraphs 2.32 and 3.30.

11 ANAO Audit Report No.53 2012–13 Agencies’ Implementation of Performance Audit Recommendations, p. 53.

12 The following ANAO performance audits tabled between 1 July 2011–31 December 2014 had recommendations directed at Veterans’ Affairs: Report No. 32 2011–12 Management of Complaints and other Feedback; Report No.48 2011–12 Administration of the Mental Health Initiatives to Support Younger Veterans; Report No.29 2012–13 Administration of the Veterans’ Children Education Schemes; Report No.46 2012–13 Compensating F-111 Fuel Tank Workers; Report No.46 2013–14 Administration of Residential Care Payments.

13 ANAO Better Practice Guide—Public Sector Audit Committees: Independent Assurance and Advice for Accountable Authorities, March 2015, Canberra, p. 3.

14 This approach is consistent with the principle that ultimate responsibility for the functions carried out by relevant audit sub-committees remains with the full audit committee: see ANAO Better Practice Guide—Public Sector Audit Committees, op. cit. p. 21.

15 The reports do not cover ‘business improvement recommendations’ from internal audits. This category of recommendation was discontinued in March 2015. Until that time responsibility for monitoring implementation of such recommendations rested with the department’s Performance and Change Committee, and the Audit and Risk Committee (audit committee) had a relatively limited oversight role. More information on how the department classifies audit recommendations is in Appendix 2.

16 The audit committee holds a further meeting in September each year, but this is for the purpose of considering annual financial statement issues only.

17 The audit committee is also provided with the full ‘status report’ considered by the audit sub-committee. This provides audit committee members with more detail on individual open recommendations.

18 The sub-committee also provided an annual report in 2014. It stated that the ‘sub-committee closely monitored the implementation of audit recommendations, both internal and external. Any significant risks were brought to the attention of the audit committee.’

19 The majority of business areas are divisions, where the senior manager is a First Assistant Secretary, but some branches report directly to the department’s Chief Operating Officer, as do some statutory Commissions.

20 See also the opening Statement by Auditor-General, JCPAA Inquiry into Audit Report No.53 2012–13, Agencies’ Implementation of Performance Audit Recommendations, 13 February 2014, p. 2.

21 From June 2014 the department’s policy has required the business area to be represented at these workshops at Assistant Secretary (Branch Head) level. The ANAO’s examination of relevant workshop records indicates a high level of compliance with this policy.

22 A specific rating is given to both internal audit and ANAO audit recommendations to communicate the importance of the recommendation to management, the audit committee and staff in the department. In relation to the recommendations within the scope of this audit (1 July 2011–31 December 2014), there were three ratings that could be assigned to a recommendation. The highest rating for recommendations was ‘CR1’ (critical recommendation 1) followed by CR2 and CR3.This rating classification structure was discontinued in March 2015. More detail about the criteria underpinning the classifications is in Appendix 2.

23 This requirement was also introduced in June 2014. The ANAO’s examination of the relevant report sign-offs likewise indicates a high level of compliance with this policy. There were three reports finalised from July 2014 (out of a total of 15 examined by the ANAO) where sign-off was by an Assistant Secretary rather than the relevant First Assistant Secretary.

24 The department advised the ANAO that the Legal Services, Assurance and Deregulation Branch is not involved in the finalisation process unless difficulties arise in reaching agreement. While internal audit reports have very occasionally been finalised with the business area noting in the report that they disagree with one or more recommendations, the matter may also be elevated to the audit committee for discussion. This occurred at the March 2015 audit committee meeting, which was attended by the relevant business area’s First Assistant Secretary to help resolve a disagreement over recommendations in a draft internal performance audit report.

25 ANAO Better Practice Guide—Public Sector Internal Audit, September 2012, Canberra, p. 42.

26 ANAO Better Practice Guide—Public Sector Internal Audit, September 2012, Canberra, pp. 42–43.

27 Internal audit reports containing either CR1 or CR2 recommendations are also provided to the audit committee.

28 The focus of this performance audit was recommendations made since 1 July 2011. However, in reviewing the audit database, the ANAO noted that the amount of information in the database for older records (particularly those closer to 2000) was generally less comprehensive compared to the post-July 2011 records.

29 The email includes guidance by the Assurance Team to business area staff as to what key information should be ‘used to frame [business area] responses’, including whether the implementing action was on track for completion by the due date and the actions taken to manage risks pending completion.

30 There had been a long-standing requirement to seek extensions of time from the audit committee (through the sub-committee). The department advised the ANAO that changes were introduced because the reasons behind seeking the extension were not always clearly identifiable amongst the large volume of information supplied by the business area in the quarterly reporting template.

31 TRIM is an electronic document record management system used by a large number of Australian Government entities.

32 Veterans’ Affairs advised the ANAO that the documentation for the remaining recommendation had been saved in the wrong TRIM file and subsequently provided it to the ANAO.

33 There may be a limited number of recommendations for which evidence of implementation is more difficult to document and store in TRIM.

34 See paragraphs 3.2–3.13 for the ANAO’s analysis of the department’s implementation of recommendations.

35 Internal Audit Report No.3 of 2011–12 VEA Reviews and MRCA Reconsiderations, Recommendation 3.

36 Particularly for internal audit reports, there would sometimes be several discrete agreed implementing actions listed against each relevant recommendation.

37 The ANAO routinely attends audit committee meetings as an observer.

38 ANAO Better Practice Guide—Public Sector Audit Committees, op. cit. p. 33.

39 At that time, the audit committee had sole responsibility for monitoring the implementation of audit recommendations.

40 The five reports contained a total of 15 recommendations directed to Veterans’ Affairs, but three recommendations remained open (as implementation was incomplete) as at the end of audit fieldwork on 30 June 2015. See footnote 12 for a list of the ANAO performance audits.

41 There were 13 internal audit recommendations that remained open as at 30 June 2015.

42 The category of ‘business improvement recommendations’ was discontinued in March 2015. Prior to that, they were subject to a different implementation monitoring process, outside of the responsibility of the Legal Services, Assurance and Deregulation Branch. The Audit and Risk Committee had a relatively limited oversight role.

43 Veterans’ Affairs advised that the issue of how the department responded to proposed recommendations in draft ANAO performance audits was discussed by its Executive Management Board (EMB) in July 2015. In particular, the EMB highlighted the desirability of only agreeing to recommendations that could be fully implemented by Veterans’ Affairs.

44 The recommendation was closed by the audit committee on the basis that all the intended actions would be implemented—the uncertainties only emerged after the closure. As such, the recommendation was closed prematurely.

45 During 2014–15, Veterans’ Affairs had initiated follow-up internal performance audits to monitor progress in piloting application of the framework. According to the most recent of these reports, Review of Procurement Quality Management Framework Implementation Health Check Status Report No.3, full implementation of the framework was expected in September 2015.

46 The audits are listed in paragraph 1.11.

47 The audit sample is described in paragraph 3.3.

48 In comparison to ANAO recommendations, implementation of some of the internal audit recommendations could be accomplished fairly readily. Departmental records indicate that over 10 per cent of relevant internal audit recommendations were implemented by the time the relevant audit report was finalised.

49 ANAO Audit Report No.8 2014–15, which examined the Department of Health, reported that the equivalent averages were 514 days for ANAO recommendations and 354 days for internal audit recommendations. ANAO Audit Report No.34 2013–14, which examined implementation of ANAO recommendations by the Department of Human Services and the Department of Agriculture, reported that the average implementation was 368 days for Human Services and 391 days for Agriculture.

50 The average of 279 days for internal audit recommendations from 2011–12 was affected by a substantial number of recommendations in which implementation was delayed. By comparison, the median time to implement internal audit recommendations from that year was only 157.5 days, with a similar median time in 2012–13 (160 days) and 2013–14 (161 days).

51 As at 30 June 2015, the average days to implement recommendations from reports published between 1 July 2014 and 31 December 2014 was only 128.5 days, but that average will increase somewhat as 13 recommendations still remain to be completed.

52 However, once the two remaining open recommendations from ANAO Report No.46 2013–14 Administration of Residential Care Payments are implemented, the average time for that year will rise to around 300 days.

53 This trend was also observed in ANAO Audit Report No.8 2014–15 Implementation of Audit Recommendations, see Table 3.5 at p.53.

54 The completion date used in these calculations is the date recorded in the audit database where the Assurance Team considered the recommendation was implemented and therefore should be closed. However, in the case of internal audit recommendations that were considered to have been implemented at the time that the relevant audit report was finalised, the report finalisation date is used as the completion date.

55 This average overdue figure for internal audits was affected by a substantial number of recommendations from 2011–12 in which implementation was delayed. The median overdue figure (for recommendations from 1 July 2011 to 31 December 2014) was 83 days.

56 As at June 2015, there was also one recommendation from ANAO Report No.28 2008–09 Quality and Integrity of Department of Veterans’ Affairs Income Support Records for which implementation is incomplete, although the business area estimated this would be done by July 2015. This recommendation has had twelve extensions, with previous business area updates noting that ‘responsibility for implementing the recommendation has been reassigned numerous times … across a range of divisions’.

57 These workshops are a feature of the department’s internal audit process. As discussed in chapter two, the general findings and indicative recommendations in a draft audit report are discussed by the relevant business area and the relevant internal audit team before the report is finalised.

58 As noted in chapter two, these actions are included in the report itself.

59 The ‘business improvement recommendations’ rating was subject to a different monitoring process and was not included in the scope of this audit. More information on rating scale classifications is in Appendix 2.