Engagement performance — general

Chapters 201 to 204

201. Contract out audits

Background

201.1 Under section 27 of the Auditor-General Act 1997 (Cth) (A-G Act), the Auditor-General, on behalf of the Commonwealth, may engage any person under contract to assist in the performance of any Auditor-General function.1 For various reasons, the ANAO engages firms to undertake some performance audits or discrete parts of performance audits on his behalf.

Policy

201.2 Tenders and contracts for a performance audit or part of a performance audit shall require contractors to comply with the requirements of:

  1. the ANAO Auditing Standards;
  2. legislation or regulations relevant to the audit, including minimum security clearance requirements;
  3. APES 110 Code of Ethics for Professional Accountants (including Independence Standards) (APES 110), as supplemented by ANAO policy in relation to the provision of other services to ANAO auditees; and
  4. the ANAO Audit Manual.

201.3 The ANAO Engagement Executive is the ‘lead assurance practitioner’ for the purposes of the Standards on Assurance Engagements (ASAEs) and retains responsibility for the overall quality on the engagement. This includes responsibility for:

  1. the engagement being planned and performed (including appropriate direction and supervision) to comply with professional standards and applicable legal and regulatory requirements;
  2. review of audit work being performed in accordance with the ANAO Audit Manual and reviewing the engagement documentation on or before the date of the performance audit report;
  3. appropriate engagement documentation being maintained to provide evidence of achievement of the audit objectives, and that the engagement was performed in accordance with relevant ASAEs and relevant legal and regulatory requirements;
  4. appropriate consultation being undertaken by the engagement team on difficult or contentious matters;
  5. remaining alert for evidence of non-compliance with relevant ethical requirements by members of the engagement team; and
  6. considering the results of the ANAO’s monitoring procedures and whether deficiencies noted and communicated by PSRG may affect the engagement.

201.4 Where an Engagement Quality Reviewer (EQR) is assigned to the engagement in accordance with ANAO Audit Manual – Shared Content, paragraph 8.42, the Engagement Executive shall take responsibility for discussing significant matters arising during the engagement with the EQR and shall not submit the final draft report to the Auditor-General for clearance until completion of that review.

201.5 The contractor is required to plan and conduct the engagement in accordance with the ANAO’s methodology for performance audits. This requires compliance with the ANAO Audit Manual and use of ANAO templates including for the Audit Work Plan (AWP), Progress Review briefings, Report Preparation Papers, draft proposed report under section 19 of the A-G Act and final draft report.

201.6 The Engagement Executive shall make such enquiries and document their review of such work papers as is necessary so as to be satisfied that the quality management procedures applied to the audit are in accordance with the requirements of the contract, including:

  1. meeting ANAO Auditing Standards and ANAO Audit Manual;
  2. the use of contractor staff with appropriate competence, qualifications and experience.

201.7 For high risk audits, the Engagement Executive’s involvement shall be extended with a greater involvement in audit planning and execution, more regular meetings with firm and auditee, and review of significant matters arising during the audit.

201.8 Prior to submitting the final draft report to the Auditor-General for clearance, the Engagement Executive shall be satisfied that the contractor’s work provides sufficient appropriate audit evidence to support the audit findings and conclusions and the subsequent release of the performance audit report. The contract engagement partner (or equivalent) shall also provide written sign off on the audit in accordance with the approved form Contractor’s Representation Letter, available to download from TeamStore.

201.9 Audit documentation, including contractor firm files, shall be complete and ready for finalisation in accordance with ANAO Audit Manual – Shared content, paragraph 9.13. In order to facilitate this, the ANAO shall provide the contractor with access to the ANAO premises and secure access to the ANAO’s document management system (E-Hive and TeamMate).

201.10 The contractor shall be required to support the ANAO’s internal and external quality assurance processes by providing audit documentation and any additional information requested relating to both in progress and completed engagements in a timely manner.

Guidance

201.11 The mandatory requirements governing ANAO audits need to be made known to contractors via tender and contract documentation.

201.12 ANAO policy is to comply with the requirements of APES 110. Those requirements apply equally to contractor firms and their staff. ANAO policy also adds to these professional requirements; for example, ‘Provision of other services by ANAO Contractors to ANAO auditees’ includes additional prohibited services to the requirements of APES 110.

201.13 To help ensure contractors comply with ANAO Policy, paragraph 201.2 of this manual requires tenders and contracts for audit services to make provision for the ANAO Engagement Executive to make the contractor aware of policies and procedures the ANAO requires be followed on the audit. The procedural steps relevant to performance audits are included in the PASG Workflow, which contains corporate templates to be used during an audit.

201.14 The ANAO Engagement Executive is formally the ‘lead assurance practitioner’ under the ASAEs. In practice, when the ANAO contracts out an audit, or part of an audit, some of the duties of the lead assurance practitioner (e.g. supervising the conduct of fieldwork) may be fulfilled by the contactor partner (or equivalent).

201.15 In accordance with ASQM1, the ANAO’s review policies and procedures (including the ANAO Audit Manual and PASG Workflow) are determined on the basis that the work of less experienced team members is reviewed by more experienced team members.

201.16 The Engagement Executive shall comply with the requirement under the ANAO Auditing Standards to be satisfied that sufficient appropriate audit evidence has been obtained to support the conclusions reached and auditor’s report to be issued. The Engagement Executive’s review covers critical areas of judgement, especially those relating to difficult or contentious matters identified during the course of the engagement, significant risks and other areas the Engagement Executive has identified as important.

201.17 The audit contractor is responsible for ensuring that audit work undertaken on behalf of the Auditor-General is performed in accordance with professional standards and to have in place quality management policies and assurance procedures to be employed throughout the audit engagement.

201.18 Whereas financial audit work conducted by the Auditor-General is broadly similar to the assurance work performed by the external auditing profession in Australia, performance auditing is the almost exclusive domain of Auditors-General. While based upon the same assurance framework as compliance, controls and similar assurance work commonly performed by the private sector, there are extensive differences. The ANAO Engagement Executive should be conscious of their level of familiarity with the ANAO performance audit methodology in determining the extent of their involvement in the performance audit.

201.19 The expected milestone dates should be communicated to the contract partner at the commencement of the audit. Milestones usually include:

  • preparation and approval of the AWP;
  • entry meeting conducted with the auditee;
  • regular progress review meetings with the relevant ANAO Engagement Executive during the period of fieldwork;
  • progress review briefing to the ANAO Executive during the finalisation of fieldwork;
  • provisions of draft Report Preparation Papers (RPPs) to the ANAO for clearance prior to issuance of RPPs to auditee(s) for comment;
  • exit meeting conducted with the auditee;
  • preparation of a formal draft audit report (section 19 report) and workshop/clearance of the report with the ANAO Executive prior to issuance to auditee(s) for comment;
  • amendment of the section 19 report (as appropriate) for auditee responses, followed by provision of a final draft report to the Auditor-General for consideration/clearance;
  • the final performance audit report to be available for tabling date; and
  • audit wrap-up and lessons learnt meeting.

202. Role and responsibilities of Group Executive Directors

Policy

202.1 The GED shall participate in key stages of the planning and delivery of performance audits including in the initial selection of audit topics and in progress reviews held during the conduct of an audit.

Guidance

202.2 The GEDs manage the ANAO’s Performance Audit Services Group (PASG) and engage in key elements of the conduct of a performance audit.

202.3 The strategic priorities of the GED are set out in the Service Group plan.

202.4 The procedural steps relevant to GEDs are included in the PASG Workflow, which contains corporate templates for staff to use during an audit.

202.5 For high risk audits, the GED is expected to have greater involvement in audit planning and execution, and to review significant matters arising during the audit.

203. Roles and responsibilities of the Engagement Executive

Background

203.1 The Engagement Executive is the ‘lead assurance practitioner’ for the purposes of the Standards on Assurance Engagements. The Engagement Executive is the head of a single administrative Branch in the Service Group that is responsible for delivering multiple performance audits in a financial year that support the delivery of the ANAO’s Annual Audit Work Program.

Policy

203.2 The Engagement Executive shall take responsibility for the overall quality on the engagement, including:

  1. the engagement being planned, performed and documented in accordance with the ANAO Auditing Standards, this Manual any other relevant ANAO policy, legal and regulatory requirements;
  2. appropriate review of the engagement documentation before the date of the assurance report; and
  3. that appropriate consultation has been undertaken on difficult and contentious matters.

203.3 Where this Manual specifies a policy requirement, that policy does not require the Engagement Executive to be directly involved in the performance of that action unless the policy specifically requires the Engagement Executive to do so. However, as required by 203.2 above, the Engagement Executive retains overall responsibility that the audit has been conducted in accordance with this Manual. Further, as required by Chapter 8 of the Shared Volume of this Manual, the Engagement Executive also retains overall responsibility for suitable direction, review and supervision being provided to the members of the audit team performing these policy requirements.

Guidance

203.4 The responsibilities in the auditing standards which this policy places on Engagement Executives include the following:

  • the overall quality on each audit engagement to which the Engagement Executive is assigned;
    1. through their actions and appropriate messages to the engagement team, the Engagement Executive should emphasise the importance of compliance with ANAO Auditing Standards and quality management policies and procedures. In addition, the engagement team should have the ability to raise concerns without fear of reprisals. Quality is essential in performing audit engagements and the overall quality of the audit will be helped by ensuring that the audit is performed in a manner consistent with ANAO standard methodology;
  • the engagement team’s compliance with relevant ethical requirements including APES 110;
    1. these include the principles of integrity, objectivity (including independence), professional competence and due care, confidentiality and professional behaviour. The Engagement Executive should remain alert, through observation and making enquiries as necessary, for evidence of breaches of relevant ethical requirements by members of the engagement team. If matters come to the Engagement Executive’s attention through the ANAO’s system of quality management or otherwise that indicate that members of the engagement team have breached relevant ethical requirements, the Engagement Executive, in consultation with others in the ANAO, shall determine the appropriate action2;
  • the engagement team’s compliance with Chapter 5 Professional, ethical and independence requirements of the ANAO Audit Manual – Shared Content;
  • the appropriate planning of the engagement consistent with ASAE 3500 Performance Engagements;
  • being satisfied that those persons who are to perform the engagement collectively have the appropriate competence and capabilities, including having sufficient time to perform the engagement in accordance with relevant standards and applicable legal and regulatory requirements and enable an assurance report that is appropriate in the circumstances to be issued3;
  • the direction, supervision and performance of the engagement consistent with professional and auditing standards and regulatory and legal requirements. The Engagement Executive should document the extent and timing of their reviews. Refer to the policy Direction, Supervision and Review (ANAO Audit Manual - Shared Content, paragraphs 8.2-8.4) for further guidance;
  • that sufficient and appropriate audit evidence exists and is documented to support the conclusions reached and for the auditor’s report to be issued;
    1. in line with the ANAO policy Audit Documentation (ANAO Audit Manual - Shared Content, paragraphs 9.2-9.16) the only work that should be done after the issuing of the final report is that of an administrative nature.
  • following appropriate procedures for consultations and differences of opinion and in particular ensure compliance with the ANAO policy on Differences of Opinion (ANAO Audit Manual - Shared Content, paragraphs 8.64-8.68);
  • determine that an EQR has been appointed, as required by the ANAO Auditing Standards and ANAO policy (Refer to the Engagement Quality Review (ANAO Audit Manual - Shared Content, paragraph 8.42); and
  • enough involvement in the audit engagement at appropriate stages throughout the engagement including attendance at key meetings, discussions with the engagement team, EQR and ANAO Executive and the completion of review of the planning and completion procedures at the appropriate stages of the audit.

203.5 Engagement Executives are assisted by the Audit Manager and team allocated to the engagement in fulfilling these responsibilities, including assisting in:

  • consideration of whether the audit should be delivered through in-house or externally contracted resources;
  • ensuring all audit related documentation is filed;
  • planning the scope of work to produce the audit deliverables in the agreed timeframe;
  • planning the time taken to prepare for, conduct and close the engagement;
  • delivering in accordance with agreed timeframes;
  • monitoring the costs associated with the audit, including recommending a variation to the budget if required;
  • ensuring the quality of the audit deliverables;
  • resourcing the audit to ensure that the audit team has the requisite skills to undertake the audit;
  • the direction, review and supervision of audit team members;
  • communication within the ANAO, among audit team members, and the entity being audited, and more broadly other interested parties;
  • assessing and managing the operational and engagement risks associated with the audit;
  • the procurement of any specialist resources and any associated contract management;
  • documenting the agreed scope, timeliness and quality assurance arrangements in respect of any services required from SADA, FSASG or PSASG to contribute to a performance audit; and
  • ensuring all persons engaged in the audit complete the required independence documentation and action is taken to manage any declared conflicts as required.

203.6 Audit Managers are expected to regularly monitor progress against established audit milestones and complete the actual date that audit milestones are achieved in the ANAO’s time monitoring systems in a timely manner. The data held in those systems forms the basis of reports to ANAO senior executives.

203.7 When engaging with the entity being audited, especially on difficult or contentious matters, the Engagement Executive and audit team should ensure that a professional and productive approach is taken. This includes, for example, trying to understand the audited entity’s circumstances, operating environment and point of view.

203.8 The Engagement Executive should be aware of any risks to audit timeliness and budget, and escalate these as soon as practicable.

203.9 The procedural steps involving the Engagement Executive are included in the PASG Workflow. As outlined in the PASG Workflow, the level of responsibility differs for audits with different risk ratings.

204. Role and responsibilities of the SADA Executive

Background

204.1 This policy sets out the responsibilities of the SADA Executive in a performance audit or any other PASG assurance engagement that engages the specialist skills of the SADA Group.

204.2 A SADA Executive is allocated to a performance audit or other assurance engagement consistent with ANAO Audit Manual - Shared Content, paragraph 6.4.

Policy

204.3 The role and responsibilities of the SADA Executive shall include:

  • the direction, supervision and performance of the SADA component of the engagement;
  • reviewing key documents and working papers on the audit file, including:
    1. the AWP and documentation of the agreed scope of SADA involvement;
    2. those relating to significant IT risks, judgements and difficult or contentious matters; and
    3. components of the section 19 report and Report Preparation Papers relevant to SADA work undertaken during the audit;
  • having sufficient involvement in the engagement at appropriate stages including attendance at key audit meetings, discussions with the engagement team, and Engagement Executive (where appropriate); and
  • attendance at Audit Committees, where appropriate.

Guidance

204.4 The responsibility for the direction, supervision and performance of the SADA component of the engagement includes:

  • emphasising the importance of audit quality on each engagement;
  • tracking the progress and quality of the SADA component of the engagement;
  • considering the competence and capabilities of the SADA audit team assigned to the engagement;
  • ensuring that appropriate SADA procedures are planned and performed;
  • addressing significant SADA matters arising during the engagement and the impact on the planned approach;
  • ensuring the SADA work performed supports the conclusions reached and is appropriately documented; and
  • agreeing with the performance Engagement Executive any SADA services to be provided in a performance audit, the scope, timeliness and quality assurance arrangements for those services, as well as ensuring those services are appropriately resourced.

204.5 IT can pose specific risks to an entity’s internal control. Some examples of IT risks which may be relevant to performance audits are:

  • reliance on systems or programs that are inaccurately processing data, processing inaccurate data, or both;
  • unauthorised access to data that may result in destruction of data or improper changes to data, including the recording of unauthorised or non-existent transactions, or inaccurate recording of transactions. Particular risks may arise where multiple users access a common database;
  • the possibility of IT personnel gaining access privileges beyond those necessary to perform their assigned duties thereby breaking down segregation of duties;
  • unauthorised changes to data in master files;
  • unauthorised changes to systems or programs;
  • failure to make necessary changes to systems or programs;
  • inappropriate manual intervention; and
  • potential loss of data or inability to access data as required.

204.6 Attendance at key meetings should be determined by the nature of the audit and the level of SADA involvement. If SADA involvement has been minimal and/or there are no material findings arising from SADA work performed, it may not be necessary for the SADA Executive to attend entry/exit meetings with the auditee, progress reviews or the section 19 workshop. However, attendance at key meetings would be expected if there was extensive SADA involvement and/or material findings arising from SADA work performed.

Engagement performance — planning

Chapters 205 to 209

205. Understanding the entity and activity subject to audit

Policy

205.1 The audit team shall obtain sufficient understanding of the entity and activity to be audited. This will assist the audit team to identify and assess the risks that the activity is not economic, efficient, effective or ethical, and design and undertake evidence-gathering procedures.

Guidance

205.2 The collection of information about the entity and activity to be audited is a key element of initial planning for an audit.

205.3 Obtaining an understanding of the activity and its context is an essential part of planning and conducting a performance audit. It includes gaining knowledge of the entity that is responsible for the activity, and where relevant, the broader program of which the activity is part.

205.4 This provides the auditor with a framework to:

  • distinguish between the activity and its control systems;
  • develop and assess the suitability of criteria;
  • target sources of relevant evidence;
  • identify performance audit operational risks;
  • identify engagement risks and assess materiality;
  • identify whether there is the need for specialist skills or the work of an expert; and
  • estimate resource requirements.
Types of information to be collected

205.5 The types of information that it may be appropriate to collect about the entity subject to audit includes:

  • objectives of the entity;
  • external accountability relationships — who the stakeholders and clients are and what their interests are in the entity;
  • internal accountability relationships — such as organisational arrangements, delegations and committee structures;
  • resources — the physical, financial, human and information resources available to the entity;
  • applicable legal and policy frameworks / requirements for the activity
  • management processes including:
    • governance arrangements;
    • performance criteria used by management;
    • assessment of performance by management;
    • the nature and frequency of reporting on performance;
    • the use of performance results to assess entity operations and performance;
    • the systems and controls in place for controlling the entity’s resources and ensuring appropriate client service;
    • risk assessments used by management; and
    • the role of internal audit in performance auditing;
  • performance goals — their consistency with the entity’s legislation and governance framework;
  • methods of program delivery — intended outputs of programs and outcomes, delivery methods and constraints on effective delivery;
  • the external environment — factors that influence the entity’s operations, over which the entity may have little control, such as economic, social and political influences, with a particular focus on changes to that environment; and
  • other publicly available information on the program.

205.6 The types of information to be collected for a cross-entity audit may focus more directly on the subject matter of the activity to be examined in the audit rather than entity-wide information. For example, the subject matter could include fraud control arrangements, the corporate planning framework, project and contract management, internal audit operations, recordkeeping or human resource management.

Sources of information

205.7 As a starting point, information already available within the ANAO should be reviewed and discussions undertaken, as necessary, with other ANAO staff who have knowledge of the entity or activity to be audited. Background information may have already been collected as part of the ANAO’s annual planning process, as part of another performance audit, by FSASG as part of the audit of an entity’s financial statements, or by PSASG as part of the audit of an entity’s performance statements.

205.8 Background information can also be derived from public or other sources external to the entity such as:

  • the entity’s website;
  • enabling and program-specific legislation;
  • Cabinet submissions and decisions;
  • the entity’s Corporate Plan, Annual Report and Portfolio Budget Statements;
  • audit committee papers;
  • Senate Estimates and other Parliamentary hearings, parliamentary committee reports, and Second Reading Speeches;
  • media reports, Ministerial and entity press releases, newspaper and journal articles, and television and radio reports;
  • central entity policies, standards, directives and guidelines;
  • Government and review tribunal hearings and reports, such as those by the Commonwealth Ombudsman;
  • reports of other external scrutineers, such as the Inspector-General of Taxation;
  • the entity’s planning documents and organisation charts; and
  • Australian or overseas material from entities that have similar programs or experiences.

205.9 In addition, the audit team should hold discussions with the entity, including Internal Audit, and obtain and review documentation, including:

  • relevant policies, plans and procedures; and
  • reports on any evaluations or reviews.

205.10 Sources of information for cross-entity audits, in relation to the subject matter rather than specific entity information, include:

  • policies or guidance promulgated by central agencies;
  • audits conducted by the ANAO and other audit offices within Australia or overseas;
  • internal audits and reviews conducted by the entities included within the scope of the audit; and
  • professional and community organisations and standard-setting bodies such as Standards Australia.

205.11 The audit team may also hold discussions with one or more of the entities to be included in a cross-entity audit to gain a practical understanding of the subject matter of the audit. The decision to undertake such discussions and collect documents from entities will need to be decided on a case-by-case basis. Some factors to consider when making the decision include:

  • whether it is likely that the particular policy or practice is widely understood and is being implemented; and
  • the need for information to assist in developing a suitable audit objective, criteria and approach.

205.12 The information-gathering powers in the A-G Act can be used to obtain information and documents required for planning an audit. However, in practice, the information-gathering powers are used as ‘reserve’ powers and access to required information is almost always obtained through cooperation with entities. If an entity is not cooperating, consult with the Engagement Executive and responsible GED in the first instance to escalate the request. If necessary, they will consider possible use of either section 33 or requesting that the Auditor-General uses section 32 to obtain the necessary information. ANAO Legal Services can also assist with informing entities about the nature of the information-gathering powers.

206. Materiality and risk assessment and management

Policy

206.1 Materiality shall be considered when determining the nature, timing and extent of procedures.

206.2 Materiality shall be assessed in planning and reassessed if there is any indication that the basis on which the materiality was determined has changed.

206.3 Materiality shall also be considered when evaluating the effect of any identified findings, taken individually and in combination.

206.4 A matter shall be considered material if it is significant to the performance of the activity in relation to economy, efficiency effectiveness and/or ethics evaluated against the criteria. During the performance engagement, the assurance practitioner shall reassess the materiality of any matter if there is any indication that the basis on which the materiality was determined has changed.

206.5 The determination of materiality is a matter of professional judgement and the basis for the professional judgements made shall be documented. Materiality assessments are documented throughout the audit in the AWP and working papers related to the audit steps at the planning (paragraphs 207.8, 207.26, 207.41 and 209.7(g)) and execution (paragraph 220.4) stages.

206.6 Audit risk shall be assessed at planning and considered and addressed throughout the audit in order to reduce it to an acceptably low level. Audit risks shall be documented in the AWP and the audit risk assessment plan, including the risk ratings and any mitigation actions put in place.

206.7 At each progress review point (see paragraph 212.1 in Engagement Performance – Execution), the audit team shall review the risk assessment and management plan. This review includes identifying any new risks, assessing if the planned mitigation actions have been and continue to be effective and if any changes in treatments are required.

206.8 Major changes to the risk levels or mitigation actions and new identified risks shall be discussed with the Engagement Executive.

Guidance

Materiality

206.9 ASAE 3500 defines materiality as variations in performance of an activity evaluated against the identified criteria which have the potential to affect the economy, efficiency, effectiveness and/or ethics (added by virtue of paragraph 6(a) of the ANAO Auditing Standards) of the activity and be reasonably expected to influence relevant decisions of the intended users or the discharge of accountability by the responsible party or governing body of the entity.

206.10 As such, findings are considered to be material if they, individually or in the aggregate, could reasonably be expected to influence relevant decisions taken by intended users on the basis of the auditor’s report. The Parliament is considered to be the primary user of the ANAO’s performance audit reports. The audit team’s consideration of materiality is a matter of professional judgement. The audit team needs to also consider the aggregate effect of individually insignificant findings.

206.11 Materiality can also be understood as the relative importance of a matter to change or influence the decisions of users of the report, such as legislatures or the executive. Materiality is considered in the context of quantitative and qualitative factors, such as relative magnitude, the nature and effect on the subject matter and the interests expressed by intended users or recipients. In addition to monetary value, materiality includes issues of social and political significance, compliance, transparency, governance and accountability. Materiality can vary over time and can depend on the perspective of the intended users and responsible parties (GUID 3910).

206.12 Guidance in respect of applying materiality to findings to form a conclusion is at Forming the Audit Conclusion.

Materiality and audit risk

206.13 Materiality and audit risk need to be considered together as considerations of materiality will consequently impact performance engagement risk.

206.14 Audit risk includes performance engagement risk, which is the risk of expressing an inappropriate conclusion based on evidence that is not soundly based. This may include evidence that is improper or incomplete as a result of inadequacies in the evidence gathering process, misrepresentation or fraud. Audit risk also includes operational engagement risk, which is the risk that an audit will not be completed in accordance with the approved budget and timeframe and to the required quality (GUID 3910).

206.15 Assessing and applying the concept of materiality throughout the audit addresses performance engagement risk by driving examination of:

  • material areas where the performance engagement risk is high; and
  • material areas where the performance engagement risk is low, but any significant variations or deficiencies could have a material effect on the economy, efficiency, effectiveness and/ or ethics of the activity/subject matter.

206.16 Determining materiality and performance engagement risk as a team, with Engagement Executive involvement, is vital to avoiding under, or over-auditing. This can result where team members have different views on materiality and performance engagement risk.

206.17 Audit risk is assessed in planning and throughout the conduct of a performance audit. The assessment of audit risk requires the audit team to:

  • understand the entity and its environment;
  • assess risks to the audit conclusion; and
  • design and conduct audit procedures to reduce engagement risk to an acceptably low level.
Materiality considerations

206.18 The consideration of materiality is relevant to all aspects of performance audits. Therefore, the auditor needs to consider materiality when selecting the audit topics, determining the audit objective(s), questions and scope, defining the criteria, evaluating the evidence, documenting the findings and developing the conclusions and recommendations.

206.19 Professional judgement about materiality is made in light of surrounding circumstances but is not affected by the level of assurance. Materiality for a reasonable assurance engagement is the same as for a limited assurance engagement because materiality is based on the information needs of intended users.

206.20 Quantitative materiality factors relate to audit findings that are expressed or evaluated numerically. Generally, audit findings are more material where they relate to relatively larger numbers or values in the context of the audit. There is no standard threshold for quantitative materiality in performance audits and in each case the auditor exercises their professional judgement in determining what numerical value represents a finding that is important in the measurement of the activity’s performance. For example, an audit finding that $1 million out of $2 million in spending activity lacked documentation is likely to be material, given the relative value of the finding to the subject matter under audit. On the other hand, a finding that a high proportion of low value transactions contained an error might not be considered quantitatively material if those errors related only to rounding errors that were within the audited entity’s accepted error rates. The auditor needs to consider the aggregate effect of individually insignificant findings.

206.21 Qualitative factors affecting materiality may include such things as:

  • the number of persons or entities affected by the matter being audited;
  • the interaction between, and relative importance of, various components of the activity when it is made up of multiple components, such as a report that includes numerous performance indicators;
  • the wording chosen with respect to the activity that is expressed in narrative form;
  • the nature of a finding; for example, the nature of findings in respect of a control when the assurance report includes a statement that the control is effective;
  • whether a finding affects compliance with law or regulation;
  • in the case of periodic reporting on an activity, the effect of an adjustment that affects past or current activities or is likely to affect future activities;
  • whether a finding is the result of an intentional act or is unintentional;
  • whether a finding is significant having regard to known previous communications to users, for example, in relation to the expected outcome of the audit;
  • whether a finding relates to the relationship between the ANAO and the auditee, or their relationship with other parties;
  • when a threshold or benchmark value has been identified, whether the result of the procedure deviates from that value;
  • whether a particular aspect of the program or entity is significant with regard to the nature, visibility and sensitivity of the program or audited entity;
  • whether the health or safety of citizens is affected; and
  • whether a finding relates to transparency or accountability.
Performance engagement risk considerations

206.22 In the context of a performance audit, risk assessment is the identification and analysis of the key risks to the achievement of objectives concerning economy, efficiency, effectiveness and ethics, thus forming a basis for developing potential audit questions and determining the potential audit scope.

206.23 Performance engagement risks have the potential to adversely affect the auditee in some way. As well as failure to deliver a policy or program economically, efficiently or effectively, other performance engagement risks could include exposure to financial loss, loss of reputation of the entity or its Minister(s), or concerns regarding national security or commercial confidence.

206.24 Indicators that there is a high level of engagement risk may include:

  • highly complex entities with multiple programs or functions;
  • deficiencies in corporate governance;
  • significant business risks which impact on the economy, efficiency, effectiveness or ethics of a program or the entity as a whole;
  • poorly controlled, or changing, processes and systems;
  • frequent changes in key personnel, systems or programs which are not well managed; and
  • previous performance engagements may have reported on significant findings.

206.25 Relevant factors for consideration when determining performance engagement risk include:

Factors that may impact the assessment of performance engagement risk

Subject matter characteristics

The nature of transactions; for example, high volumes, large dollar values and complex transactions.

The nature, size and complexity of the activity/subject matter.

External environment

The economic, social, political and environmental impact of the activity/subject matter.

Internal factors

The extent of management’s actions regarding issues raised in previous performance engagements.

The complexity and quality of management information and external reporting.

The effectiveness of internal control.

The nature and degree of change in the environment or within the entity that impact on the activity or subject matter.

   

206.26 The risk assessment should be informed by the auditor’s understanding of the entity and activity subject to audit. Based on this understanding, risks could be identified and analysed by answering the following questions:

  • What can go wrong?
  • What assets are at risk and from what sources?
  • With whom does the risk lie?
  • What factors are/can be constraining performance (economy, efficiency, effectiveness, ethics)?
  • What could be the cause (including weaknesses in controls?
  • What could be the consequences or the impact, including on the entity’s reputation?
  • How could this risk be managed?

206.27 The risks identified should be closely examined in order to decide on the ones that are key (significant and relevant). The risk level of the key risks should be determined by assessing the likelihood and potential impact of each risk using the risk matrix in the Risk Assessment Template.

206.28 Audit responses to address the key risks identified should then be documented in the Risk Assessment Template.

206.29 The Risk Assessment Template provides a repository for all established information on the risks and serves as a resource when communicating risk information to stakeholders. Each risk is allocated to a member of the audit team or other ANAO personnel who has the responsibility to regularly review the risks and assess the effectiveness of any mitigation treatments.

207. The Audit Work Plan

Background

207.1 The Audit Work Plan (AWP) documents the planning activities for the audit which are required by the Auditing Standards (ASAE 3500).

Policy

207.2 An AWP shall be prepared for each audit.

207.3 Audit teams shall determine whether an examination of the performance of the entity or subject matter is a suitable focus for a performance audit.

207.4 Prior to preparing an AWP, the Group Executive Director shall seek the approval of the Auditor-General to ascertain whether the proposed audit topic is suitable for planning. This will usually occur quarterly and for multiple audits at one time via a batch approval.

207.5 The audit objective shall be rational, clearly defined and relate to the principles of compliance, economy, efficiency, effectiveness and/or ethics. It shall be expressed in terms that can be concluded against.

207.6 Suitable criteria, corresponding to the audit objective, shall be identified for each audit. They shall be reasonable quantitative or qualitative measures of performance against which the activity’s performance may be assessed.

207.7 Criteria shall be relevant, complete, reliable, neutral and understandable.

207.8 The AWP shall document materiality and engagement risk level.

207.9 The AWP shall include the estimated cost of the audit, milestones and target dates. Performance audit teams shall discuss proposed audits with SADA in the planning and conduct of individual audits.

207.10 The AWP shall include the audit team members, including the GED, ED, Audit Manager and the number and classification levels of all other team members.

207.11 The AWP shall be provided to FSASG, and PSASG, where there is a concurrent4 performance statement audit. In return, FSASG and PSASG shall provide the Audit Strategy Document, interim and final management letters (where issued) and the closing letter, related to the relevant financial statement or performance statement audit.

207.12 Performance audit teams shall discuss proposed audits with FSASG and PSASG (where relevant)5 and consider financial and performance statement audit findings in the planning and conduct of individual audits.

207.13 Any major variation from the details about the rationale and background to the audit outlined in the Annual Audit Work Program shall be clearly communicated to the Executive in the AWP, with a detailed explanation of the reasons underpinning the variation. In those cases where the audit topic was not included in the Annual Audit Work Program, the reasons for undertaking the audit shall be set out in detail.

207.14 All AWPs shall be provided to the Auditor-General for approval, with a copy provided at the same time to the Deputy Auditor-General.

207.15 Where the audit is not included in the Annual Audit Work Program or where there is a significant variation in the nature and intent of the audit compared with that set out in the Annual Audit Work Program, the AWP must be endorsed by the responsible Group Executive Director before submission to the Deputy Auditor-General, and Auditor-General.

207.16 Following approval of the AWP by the Auditor-General, any subsequent changes to audit criteria shall be reviewed by the responsible Engagement Executive and approved by the responsible GED and the Auditor-General.

207.17 The AWP for each audit shall briefly identify any significant engagement and operational risks confronting the audit, including key risks identified in the portfolio overview of the most recent Annual Audit Work Program, where relevant.

207.18 Where audit teams are likely to have intermittent contact during the course of the audit with vulnerable people such as Indigenous communities, people with disabilities, people from non-English speaking backgrounds or children under the age of 18 years of age, they shall indicate this in the AWP. Advice shall be sought from an appropriate Senior Director in the Corporate Management Group (CMG) to ensure that the appropriate support arrangements can be put in place, including whether audit team members are required to apply for a working with vulnerable people registration or a working with children check.

Guidance

207.19 Audit planning varies according to the size, nature and complexity of the audit. The resources to be used in planning an audit should be commensurate with the nature and complexity of each audit and the assessment of the risks to the audit.

207.20 The preparation of an AWP necessitates understanding the entity and activity subject to audit. Per section 205.12 of this manual, the information-gathering powers in the A-G Act can be used to obtain information and documents required for planning an audit.

207.21 An AWP template and instructions on planning are available as part of the PASG Workflow, which contains corporate templates for staff to use during an audit.

207.22 Each AWP should contain sufficient information to allow the GED and the Executive to make a fully informed decision on the conduct of the proposed performance audit. Once approved, the AWP provides the authority to conduct the audit.

207.23 In preparing the AWP, the audit team should refer to the criteria, approach, budget and timeframes for similar audits considered previously and as outlined in their AWPs.

207.24 In a small number of cases, it may not be possible to finalise elements of the AWP until the delivery phase of the audit has commenced. In these circumstances, the audit team should clearly outline in the AWP those aspects of the audit plan that may be subject to change and set a timeframe to confirm any changes. Approval will be needed to proceed with the audit while the details of the AWP are still being finalised. The audit team is responsible for progressing the audit in parallel with the refinement of the audit plan during the delivery phase.

207.25 The following paragraphs outline considerations for preparing specific elements of the AWP.

Rationale for undertaking the audit

207.26 The AWP outlines the rationale for conducting the audit. Usually, the rationale for undertaking a particular audit has been identified in the course of preparing the Annual Audit Work Program. This rationale will be included in the audit report in accordance with paragraph 224.4 of this manual, and hence should be clear and understandable to a broad audience, with reference to the relative importance of the subject matter from a stakeholder perspective. The following table illustrates examples that should be incorporated in a rationale.

Element

Description

Materiality

Qualitative: High public visibility of the program; importance of the program to particular client groups; strong Parliamentary or community interest in the performance of the program.

Quantitative: high value of assets, annual expenditure or annual revenue of the entity or the program, activity or function. Scope may be limited to items above a certain dollar threshold, e.g. procurement contracts greater than $xx.

Impact

Significant impact of the activity, even when it is undertaken by a small unit within an entity with low materiality.

Key area/issue presenting risks or challenges to Commonwealth administration

The program or activity being a government initiative that is directly related to a key area/issue presenting risks or challenges to Commonwealth public administration.

Potential benefits from the audit

More efficient business processes; greater accuracy in claims processing; better management of contracts; improved adherence to Commonwealth policies; greater accountability through accurate performance reporting; earlier detection of risks to good management or prevention of fraud.

Previous coverage

No previous ANAO performance audit coverage; very limited internal review of a significant program; possibility of a follow-up audit foreshadowed in a previous ANAO audit; a follow-up audit requested by a parliamentary committee.

Value for money

Multiple factors need to be taken into account when determining value for money. These include:

  • reducing costs/managing budget cuts;
  • contract management; and
  • digital transformation.

Refer to Developing a Value for Money Perspective for Performance Audits for details on applying a value for money perspective.

Auditability

Relates to the ability to carry out the audit according to the ANAO Auditing Standards. Although some areas may be significant, they may not be auditable for one or more of the following reasons:

  • the area is outside our mandate;
  • the audit team does not have or cannot acquire the required expertise;
  • the area is undergoing significant and fundamental change;
  • suitable criteria or approaches are not available to assess performance; or
  • the information or evidence required is not available or cannot be obtained efficiently.
   
Background to the audit

207.27 Each AWP should include background information regarding the entity, program or function to be audited. This background information reflects and generally builds on the material for the particular audit that was included in the Annual Audit Work Program.

Audit objective

207.28 The audit objective, outlined in the AWP, is a key statement that is intended to define the intention of the audit. The objective of a performance audit is to provide an assessment of specified elements of an entity’s operations. The assessment should address one or more of the following terms: effectiveness; efficiency; economy; or compliance.

These terms are defined in ASAE 3100 and ASAE 3500 as follows:

Term

Definition

Effectiveness

The performance principle relating to the extent to which the intended objectives at a program or entity level are achieved.

Efficiency

The performance principle relating to the minimisation of inputs employed to deliver the intended outputs in terms of quality, quantity and timing.

  1. See Special Considerations for Auditing Efficiency: Methodology and Guidance for further guidance about conducting audits focussed on efficiency.

Economy

The performance principle relating to the minimisation of the costs of resources, within the operational requirements of timeliness and availability of required quantity or quality.

Compliance

The assessment of adherence to the requirements, as measured by suitable criteria.

   

207.29 ’Ethics’ is defined in the ANAO Auditing Standards as the performance principle relating to the extent to which the proposed use of public resources is consistent with the core beliefs and values of society. Where a person behaves in an ethical manner it could be expected that a person in a similar situation would undertake a similar course of action. For the approval of proposed commitments of relevant money, an ethical use of resources involves managing conflicts of interests, and approving the commitment based on the facts without being influenced by personal bias. Ethical considerations must be balanced with whether the use will also be efficient, effective and economical. See ‘Methodology Guidance: Audits of Ethics’ for further guidance about conducting audits focussed on ethics.

207.30 The audit objective and the audit scope (see paragraph 207.37) are interrelated and should be considered together. The audit objective needs to be realistic and achievable and give sufficient understanding to the entity and other relevant parties about the focus of the audit. The audit objective also provides the basis for developing the audit criteria and the audit approach.

Audit criteria

207.31 Audit criteria are the specific measures used to assess the performance of the activity. In accordance with ASAE 3500 paragraph 16(d), the criteria are benchmarks used to evaluate the underlying subject matter. Audit criteria are reasonable and attainable standards of performance against which the extent of effectiveness, efficiency, economy, ethical or compliance aspects of an entity’s programs or activities can be assessed.

207.32 Audit criteria are important because they provide:

  • a common understanding between the audit team, the ANAO Executive and the entity regarding the standards against which the entity is to be assessed; and
  • a structure for the evidence-gathering phase of the audit.

207.33 Suitable criteria are those that are relevant to the subject matters being audited and appropriate to the circumstances. As outlined in ASAE 3500 the characteristics of suitable criteria include:

  • relevance: relevant criteria contribute to conclusions that assist decision-making by the intended users;
  • completeness: criteria are sufficiently complete when relevant factors that could affect the conclusions in the context of the performance engagement circumstances are not omitted. Complete criteria include, where relevant, benchmarks for presentation and disclosure;
  • reliability: reliable criteria allow reasonably consistent evaluation of measurement of the activity, including when used in similar circumstances by similarly qualified assurance practitioners;
  • neutrality: neutral criteria contribute to conclusions that are free from bias; and
  • understandability: understandable criteria contribute to conclusions that are clear, comprehensive, and not subject to significantly different interpretations.

207.34 Sources of suitable criteria and sub-criteria may include:

  • policy decisions or policy statements;
  • legislation and regulations;
  • published performance measures and internal measures;
  • policies and guidance developed by central entities, regulators or government;
  • standards of good practice, relevant benchmarks and relevant practice guides developed by professions, associations or other recognised authorities;
  • statistics, practices, benchmarks, performance standards or procedures developed within the entity; and
  • subject matter or general literature6.

207.35 Criteria may require interpretation and modification to ensure their relevance to the audit. Criteria may need to be modified or refined as the audit proceeds and more information becomes available. Should the criteria require substantial amendment and impact on the scope of the audit, the proposed changes should be discussed, in the first instance, with the Executive Director. If agreed, the proposed changes can then be considered by the ANAO executive at the formal review stages (see the PASG Workflow).

207.36 For further details refer to ANAO Audit Manual – PASG Specific Chapter 208 Generic audit criteria.

Audit scope

207.37 The audit scope defines the boundary of the audit. The audit scope may identify:

  • the part of the entity, management control system or organisational unit to be examined;
  • the matters subject to audit;
  • particular entity locations to be visited during the audit;
  • the time period being examined by the audit; and/or
  • any associated matters that are not within the scope of the audit and the reasons for their proposed exclusion from the audit.

207.38 The scope is usually established based on information gathered during the planning phase or obtained in previous audits.

207.39 In establishing the scope of an audit, it may be necessary to have a broad statement of scope at the outset and refine this during the course of planning the audit and in the early stages of conducting the audit.

207.40 When determining the scope of the audit, the AWP should make reference to any known relevant reports published by other external and internal scrutineers; for example, the Inspector-General of Taxation, Inspector-General of Intelligence and Security, Parliamentary committees or internal audit.

207.41 The audit team considers matters such as materiality, risks to successful program performance and/or service delivery and auditability when establishing the scope. These matters should be considered throughout the planning and conduct of the audit and, particularly, in developing the detailed audit criteria.

Audit method

207.42 The audit method sets out the extent of evidence-gathering procedures to be undertaken and the reasons for selecting them, and the means to be used to collect information relating to the audit criteria. The method explains the intended use of specific data collection tools such as sample surveys, case studies, meetings, formal interviews, document reviews, compliance and/or system control analysis and testing. The method should also consider whether testing is likely to be performed on a sample basis.

207.43 The audit method also specifies where and why particular fieldwork is to be carried out and lists the involvement of any external stakeholders.

Cost of an audit

207.44 The cost of an audit includes the estimated costs of staff resources and the engagement of contractors and experts, and the estimated costs of travel and report publication. The costs of the initial planning phase of the audit and scoping study, where undertaken, are also to be included.

207.45 The budget for the audit is derived from a consideration of:

  • the estimated hours required to be spent on the audit by the audit team, Engagement Executive and the Group Executive Director;
  • the cost of initial planning;
  • the cost of FSASG, PSASG and SADA staff, contractors, specialists and experts;
  • costs of travel, including attendance at any audit related conferences. Travel costs should include airfares, accommodation, travel allowance and taxi/car hire that will be incurred by all team members, including any contractors engaged to assist with the audit; and
  • the cost of publishing the audit report.
Audit milestones and target dates

207.46 AWPs are to include the target dates for the following audit milestones:

  • AWP discussion meeting;
  • Audit start date (Designation letter sent);
  • Progress Review 1;
  • Progress Review 2;
  • Report Preparation Papers to entity;
  • Section 19 workshop;
  • Section 19 report to entity; and
  • Audit tabled.

207.47 The PASG Workflow references a Ready Reckoner tool to assist with planning the audit phases and milestone dates.

The audit team

207.48 In determining the composition of the audit team, the following factors are taken into consideration:

  • the experience of the Audit Manager;
  • the number, level and experience of other team members;
  • the benefit of engaging FSASG, PSASG and/or SADA to assist in conducting elements of the audit;
  • the benefit of engaging specialists and/or experts (including from PSRG and CMB) to support the in-house team in addressing complex and/or technical issues for example, relating to methodology); and
  • the complexity and expected impact of the audit.

207.49 A specialist is an audit practitioner (either from ANAO or external) who specialises in auditing a particular subject area (e.g. IT Auditor).

207.50 An expert is a person or organisation whose expertise in an area other than auditing is used by the ANAO to assist in obtaining sufficient appropriate audit evidence. An expert would normally be external to ANAO but may include internal non-audit staff such as CMG or PSRG Legal Services staff.

207.51 Where gaps are identified in the skills necessary to conduct a particular audit, there are a number of options to address this, including:

  • undertaking training to obtain the skills;
  • obtaining advice and assistance from within PASG or other areas of the ANAO; or
  • engaging a contractor, expert and/or specialist.

207.52 As part of the AWP, the audit team are encouraged to seek the input of other PASG Audit Managers and Executive Directors who have conducted similar themed audits or audits of the same entity to assist in scoping the audit.

207.53 As part of the AWP, the audit team should outline a summary of the pre-audit consultation with FSASG, PSASG and SADA. This summary should include any potential or planned involvement of FSASG, PSASG and SADA in the performance audit, any risks identified from financial statement auditing, and other intelligence gathered from past audits that may be of relevance for the proposed audit.

Risk

207.54 A detailed risk assessment and management plan is completed and attached to the AWP that addresses each risk and its corresponding mitigation strategy (refer to ANAO Audit Manual – PASG Specific Chapter 206 Materiality and risk assessment and management).

Contributions to audit planning
External stakeholders

207.55 External stakeholders include people or organisations with an interest in the operations, activities, results or resources of an entity. The primary external stakeholders are members of the public, clients of the audited entity, and non-government organisations such as industry associations and special interest groups.

207.56 Increasingly, entities are expected to develop close links with interest groups such as consumer and industry associations, provider organisations and think-tanks. These relationships can go beyond the exchange of information and may involve more formal collaboration or negotiation about government decision-making.

207.57 The identification of external stakeholders — who and what their interests are in the audited entity or topic — should be completed during the audit planning stage.

Approval of the AWP

207.58 The approval processes for the AWP are outlined in the PASG Workflow, which contains corporate templates for staff to use during an audit.

208. Generic audit criteria

208.1 Generic audit criteria have been developed for the following four common types of performance audits conducted in the ANAO:

  • Grants administration (Post PGPA Act - 1 July 2014)
  • Procurement (Post PGPA Act - 1 July 2014)
  • Regulation (Post PGPA Act - 1 July 2014)
  • Contract Management (Post PGPA Act - 1 July 2014)

208.2 The purpose of each of the four criteria documents is to provide performance audit teams with a starting point for drafting criteria for these particular types of audits. Audit teams will still need to tailor specific criteria to their audits after identifying the time period and relevant legislative framework that applies (either prior to or post PGPA Act) and carefully considering the audit objective, the type of program or project being audited and the business risks of the entity being audited.

208.3 The generic criteria broadly describe key structures, processes and policies that would be expected to be in place within an entity. These criteria were developed using the ANAO’s Better Practice Guides (now withdrawn), relevant recent performance audits as well as Government directives. References are provided at the bottom of each document. Audit teams are encouraged to review this reference material when developing criteria for individual performance audits.

208.4 In relation to the criteria for Grants Administration, a tiered approach is warranted because the relevance of various parts of the Commonwealth Grant Rules and Guidelines often depends on the circumstances and parameters of the particular program.

209. Planning audit procedures

Policy

209.1 The audit team shall document the planned audit procedures including the nature, timing, extent and rationale for the planned procedures.

209.2 The planned audit procedures shall be updated as necessary throughout the audit and all changes in the planned procedures shall be clearly documented and explained.

209.3 At a minimum, an audit test program shall provide a link between the criteria, evidence, audit procedures performed and the results and findings.

209.4 The planning of an audit shall include an assessment of whether the audit team has adequate skills, competence and knowledge to undertake the particular audit.

Guidance

209.5 Combined, the AWP and audit test program document the planned audit procedures, including the nature, timing and extent of evidence-gathering procedures and the rationale for selecting the approach.

209.6 When determining the extent of time and resources required for planning, audit teams are to consider the:

  1. audit team’s experience with and understanding of the entity and the audit topic;
  2. size of the team;
  3. level of the audit team’s auditing experience;
  4. scope of the audit; and
  5. complexity of the audit criteria and proposed tests and evidence-gathering techniques.

209.7 The AWP and/or audit test program includes:

  1. the types and expected sources of audit evidence;
  2. the techniques planned to be used to gather evidence;
  3. the planned audit procedures, including timing and extent (e.g., target testing of a specified number or items with defined characteristics, random sampling);
  4. personnel and expertise requirements, including the nature and extent of the use of specialists or experts when applicable;
  5. the allocation of tasks to be performed by audit team members;
  6. a link between the criteria, evidence, audit procedures performed and the results and findings;
  7. materiality: and
  8. assessment of engagement risk.

209.8 Planning is not a discrete phase, but a continual process and the test program may need to be revised to reflect any changes in the planned approach. It is recommended that updates to the planned approach and rationale are documented to explain why the change was necessary.

209.9 In planning and conducting a performance audit, it is not expected that audit procedures will include directly assessing whether fraud or other wrongdoing is occurring in the program or activity subject to audit. It is not the auditors’ responsibility to prevent or detect fraud or other wrongdoing through the conduct of its audits. This is the responsibility of the entity itself. The ANAO is also not in a position to determine whether a fraud or other wrongdoing has actually occurred.

209.10 As part of planning and conducting a performance audit, the audit team is required to obtain an understanding of the entity and the program or activity subject to audit. This should include making an assessment of whether fraud, or related wrongdoing, may have a significant impact on the program or activity. Such an assessment could include: a review of the entity’s fraud control plan and related information and documentation, such as fraud plans of individual work areas; and a review of the entity’s systems and procedures relating to fraud prevention, investigations, prosecutions, and reporting. If the audit team concludes that the risk of fraud and related wrongdoing may have a significant impact on the program or activity subject to audit, the audit should assess the adequacy of the entity’s management of this risk.

209.11 The extent to which matters such as potential fraud and other wrongdoing are referred to in a performance audit report will depend on the individual circumstances. It is generally not necessary or appropriate to refer to individuals or specific instances in audit reports. In circumstances where such matters are systemic and have the potential to significantly impact the operations of the program or activity subject to audit, they may warrant specific audit coverage and reference in the audit report.7

Engagement performance — execution

Chapters 210 to 223

210. Designation: Communicating the terms of the audit

Background

210.1 The designation email provides notification of the Auditor-General’s decision to conduct an audit and mandate, states the section of the A-G Act under which the audit is to be conducted and advises that the audit report will be tabled in the Parliament as soon as practicable after the completion of the audit. It also provides specific details about the audit and audit team.

Policy

210.2 The ANAO shall issue a written designation message that communicates the terms of the performance engagement (audit or review) to each entity subject to the performance engagement (ASAE 3500). The designation shall include the legislation creating the Auditor-General’s mandate to perform the audit, being sections 17, 18, 18A or 18B of the A-G Act.

210.3 Where the scope of an audit is subsequently changed or extended to include more than one entity, the section under which the audit is conducted will change from section 17 to section 18 of the A-G Act. In these circumstances, a revised designation message shall be sent to the original entity and a section 18 designation message provided to the other entity(s).

Guidance

210.4 The ANAO issues a designation email that communicates the terms of the performance engagement to each entity subject to audit at the commencement of each audit before commencing audit fieldwork. A standard designation template is attached to the PASG Workflow, which contains corporate templates for staff to use during an audit.

210.5 The designation email is sent to the accountable authority of a Commonwealth entity (refer to PGPA Act, section 12 – normally the Chair of the accountable authority where the authority has more than one member) and to the Chair of the Board of Directors of a Commonwealth company.

210.6 The designation message provides notification of the:

  • Auditor-General’s decision to conduct and audit;
  • Auditor-General’s mandate;
  • section of the A-G Act under which the audit will be conducted;
  • audit focus, including the objective and scope of the proposed audit;
  • entity’s opportunity to provide written representations on the program or activity subject to the audit; and
  • ANAO contact details.

210.7 In addition, it may be useful to provide the entity with a scope diagram that includes the audit objective, high-level criteria and sub-criteria, and a separate list of the initial information requirements for the audit.

210.8 Section 17 of the A-G Act is limited to the audit of a single Commonwealth entity, company or subsidiary. Section 18 of the A-G Act does not limit an audit to an audit of single Commonwealth entity, company or subsidiary. It is necessary for audits conducted of more than one entity, company or subsidiary to be separated from audits of a single entity, company or subsidiary. Should the audit change or be extended to include more than one entity, company or subsidiary, it is to be conducted under section 18 of the A-G Act and the change shall be communicated to the original entity, company or subsidiary. Refer to paragraph 210.3.

211. Entry meetings

Policy

211.1 The ANAO shall conduct an entry meeting with each entity, unless the entity declines an entry meeting or it is impractical to do so; for example, where the audit involves a survey of a large number of entities.

211.2 Approval to not proceed with an entry meeting shall be obtained from the Engagement Executive and documented in the audit file.

211.3 A record of the entry meeting, including the outcomes and any significant decisions made, shall be documented in the audit file.

Guidance

211.4 Following the designation of an audit, the entry meeting serves a number of purposes, including introducing the audit team and allowing discussion of the audit objective, process and timing with the entity.

211.5 The purpose of an entry meeting is to inform the entity and establish the basis for a successful engagement by:

  • outlining the purpose of performance audits and opportunities to minimise the impact on the auditee;
  • drawing attention to information access powers under the A-G Act;
  • introducing to entity management the staff of the ANAO who will be involved in the audit;
  • explaining the background and objectives of the audit, discussing the audit criteria and responding to any questions the entity may have about these issues;
  • giving entity management the opportunity to ask questions about the audit process or any other relevant matter relating to the audit, including key milestones and expectations;
  • drawing attention to subsection 36(1) of the A-G Act concerning the confidentiality of information obtained during the course of an audit, particularly when an entity has had little previous exposure to performance audit requirements and processes;
  • allowing entity managers to bring to the attention of the ANAO any contextual matters that might influence the way the audit is conducted, particularly documentation that assists in explaining the program and/or relevant issues;
  • informing the entity about information the audit team is expected to require — specifically, classified and/or sensitive records and where arrangements need to be made to access systems together with applicable milestones — to help ensure an efficient and timely audit process; and
  • discussing the administrative arrangements surrounding the audit and establishing coordination arrangements with the entity.

211.6 It is advisable that the auditee(s) is informed at the entry meeting about the citizen contribution facility on the ANAO website and that the audit is open for stakeholders contributing information during the evidence collection stage. Also refer to the PASG Workflow.

211.7 ANAO officials are expected to approach entry meetings as an educative opportunity for audited entities and to not assume a high level of knowledge about ANAO processes.

212. Progress reviews

Policy

212.1 Progress reviews shall occur at:

  • 20 per cent into the allocated audit hours;
  • 50 per cent into the allocated audit hours; and
  • before or after the exit meeting (optional).

212.2 The third Progress Review (before or after exit meeting) shall occur when:

  • the ANAO Executive requests a meeting;
  • there is a significant audit issue arising from previous progress reviews remaining unresolved; or
  • the responsible Executive Director or Group Executive Director considers a meeting necessary.

212.3 The audit team shall prepare briefs on the audit’s progress in accordance with the appropriate templates provided by the service group, or as otherwise specified by practice management.

212.4 The Engagement Executive shall seek approval from the Auditor-General in those cases where there has been a change since the AWP was approved that affects the audit objective, scope or criteria.

212.5 The outcomes and all significant decisions made in key meetings and briefings with the ANAO Executive, responsible GED and/or Executive Director shall be documented in the audit working papers.

212.6 The relevant FSASG Executive Director or Signing Officer, as appropriate, shall be invited to attend the 2nd Progress Review meeting. Where there is a concurrent8 performance statement audit, the relevant PSASG Executive Director or Signing Officer, as appropriate, shall be invited to attend the 2nd Progress Review meeting.

212.7 In addition to the minimum progress review meetings outlined in paragraph 212.1, the audit team shall engage with the GED and the ANAO Executive at any other critical stages of the audit, and in relation to any significant issues.

212.8 The outcomes and all significant decisions made in key meetings and briefings with the ANAO Executive shall be documented in the audit working papers. Where notes or comments are made by the ANAO Executive on Progress Review briefing papers prepared by audit teams, these records shall be retained on the audit working papers.

Guidance

212.9 Performance audit teams are required to undertake a series of progress reviews with the Service Group Executive and ANAO Executive staff at key specified intervals during the conduct of the audit.

212.10 The audit team completes an initial progress review (PR1) in order to confirm that planning for the audit remains appropriate or to suggest a variation to the conduct of the audit or to discuss any emerging issues, such as difficulties in accessing information or data.

212.11 The initial progress review should explicitly consider the ongoing appropriateness of scope, budget and schedule. Where deficiencies are identified, treatments should be applied to support delivery on time and on budget. Audit Managers and Executive Directors should consider a range of treatments to support the timely delivery of the audit. If changes to budget, timeframe or scope are required, then a variation should be sought from the GED.

212.12 The audit team completes a second progress review (PR2) meeting to identify and distil the key issues arising from the audit and to consider the most effective structure for the presentation of the audit findings and potential recommendations in the proposed Report Preparation Papers.

212.13 Report Preparation Papers should be fully defined, findings and conclusions identified and Report Preparation Papers approximately 50% drafted when the second progress review meeting is conducted.

212.14 Prior to the commencement of the drafting of the section 19 report, the audit team can request a third progress review meeting to discuss the key issues arising from the entity’s response to the Report Preparation Papers and/or the exit meeting and to consider the most effective structure for the presentation of the audit findings and recommendations in the section 19 report.

212.15 Specific requirements for the briefing are included in the PASG Workflow, which contains guidance and corporate templates for staff to use during an audit.

212.16 Progress review meetings are an important opportunity to canvas and discuss issues such as:

  • key messages from the audit findings;
  • the proposed audit report structure, including headings for:
    • key findings in the report summary;
    • chapter headings in the report body; and
    • appendices;
  • potential sensitivities that may arise from the audit findings and the impact that could have on reporting of audit evidence, including those arising from legal advice received or security issues;
  • potential audit recommendations;
  • the linkages (or otherwise) back to the original request in the case of audits undertaken following a request from a Member or Senator of the Parliament of Australia;
  • outline of any significant proposed divergence from the audit objective and/or audit criteria set out in the approved AWP, with reason(s) for this divergence;
  • any difficulties envisaged in meeting stakeholder expectations;
  • estimated dates for the remaining milestones until the report is tabled, against approved milestone dates; and
  • the preliminary overall audit conclusion.

212.17 To assist in drawing on the experience of other audit teams, the responsible Executive Director may invite members of other audit teams to participate in progress review meetings on a case-by-case basis. As noted at 212.6 members of the FSASG team are required to be invited to attend the 2nd Progress Review meeting and may be invited to attend the 1st and 3rd Progress Review meetings.

212.18 Notwithstanding the formal requirements for briefings during key stages of the audit, the audit team would be expected to provide briefings to the responsible GED and the ANAO Executive that are commensurate with the likely impact of the audit.

212.19 Discussions with the Executive could occur:

  • during or at the completion of fieldwork so that any significant issues that are identified in the course of the fieldwork are discussed; and
  • after the completion of the exit meeting to discuss entity feedback, key findings and conclusions for the draft report.

212.20 These briefings are also an opportunity to bring the Executive’s attention to matters such as potential sensitivities, budget or timeframe pressures or potential difficulties in meeting stakeholder or user expectations, including for audits requested by Members and Senators of the Parliament of Australia.

213. Advising the entity of progress and significant issues

Policy

213.1 The audit team shall keep the management of the entity subject to audit informed in a timely manner of the conduct of, and significant issues arising from, the audit.

213.2 The audit team shall inform the entity of any deficiencies in controls and systems, and material findings on a timely basis to allow the entity sufficient time to investigate and respond to the findings.

213.3 The audit team shall consider, within the terms of the engagement or regarding relevant legislative requirements, whether any other matter has come to their attention that needs to be communicated with the entity.

Guidance

213.4 Explaining the audit process to the entity at all key points during the audit and keeping the entity informed is crucial to maintaining a sound professional relationship with the entity being audited and ensuring the free flow of information.

213.5 Informing the entity of matters arising from the audit would include providing entity management with early advice on particularly significant issues that are identified during the course of the audit so that the entity’s perspective on the issues may be obtained at an early date.

213.6 When advising the entity on significant issues identified during the audit, the audit team must consider:

  • whether there is sufficient certainty concerning the audit position to warrant the advice; and
  • including an appropriate caveat for any advice provided.

213.7 This is particularly important because the ANAO’s position may change during the audit review process.

213.8 The level and frequency of contact with entity senior management is determined by the audit team but would be commensurate with the expected impact of the audit, and sensitivity to the issues arising.

213.9 Keeping the entity informed provides the opportunity for the audit team to foreshadow future steps in the audit process and flag further information and assistance that will be sought during the remainder of the audit.

213.10 Informing the entity about the conduct of the audit could include advising the entity on just completed or planned future key steps in the audit process, for example, the results of a significant meeting, the commencement/completion of fieldwork, and the development of Report Preparation Papers.

213.11 Informing the entity about the conduct of the audit could also include significant changes to the audit process and/or timing, for example, caused by the unplanned absence of key members of the audit team.

214. Entity security requirements

Policy

214.1 When planning an audit, the audit team shall consider the level of security clearance that will be required to conduct the audit in an entity, and take appropriate action to obtain the necessary clearance (if not already in place for all audit team members).

214.2 All staff shall comply with the relevant requirements of the Australian Government’s Protective Security Policy Framework, as implemented by the entity subject to audit.

214.3 Where the audit team considers that entity security requirements are not reasonable, this shall be promptly brought to the attention of the responsible Executive Director and, as necessary, the responsible GED, and resolved through professional communication with the entity as soon as practicable.

214.4 Entity information about individuals shall only be accessed for a clearly defined audit purpose, and ANAO staff shall not attempt to access information from entity systems or records relating to themselves, relatives or acquaintances. If there is a risk of accessing such information in the course of an audit or review, staff shall bring the risk to the attention of their audit executive and consult with the relevant entity as necessary.

Guidance

214.5 When allocating resources to an audit, PASG Practice Management should have regard to the likely level of security clearance required and allocate staff accordingly where possible.

214.6 The audit team needs to have regard to the reasonable security requirements of the entity being audited.

214.7 For further information, the audit team can refer to the ANAO’s Personnel Security Policy, available on the Audit Central Protective Security page.

214.8 There are a number of factors that the audit team can consider to assess whether the entity security requirements are reasonable. These include whether the requirements:

  • impinge on access to documents and systems necessary to the completion of the audit;
  • introduce barriers to the efficient conduct of the audit that are contrary to the exercise of access powers under the A-G Act; and
  • do not provide for the same level of clearance for the audit team as the level of clearance for relevant entity personnel.

214.9 Entity ICT security protocols may be triggered in the event of inappropriate access. Inappropriate access may result in reputational damage to the ANAO as a trusted user of entity information and action against the individuals involved.

215. Gathering audit evidence

Policy

215.1 The audit team shall gather evidence that is sufficient and appropriate to address the audit’s objectives and support the audit’s findings and conclusions.

215.2 The audit team shall exercise professional judgement to obtain the sufficient quantity and appropriate quality of audit evidence.

215.3 This evidence shall be documented and stored appropriately to provide the basis for concluding that the audit was conducted in accordance with the ANAO Auditing Standards and other legal requirements.

215.4 All evidence gathered shall be stored appropriately in accordance with the ANAO’s Protective Security Governance Policy and ANAO Information Management Framework.

215.5 ANAO staff granted privileged (researcher) access to entity document management systems must ensure that all information searches relate directly to an active audit or review and are conducted for the purpose of gathering evidence.

215.6 If an entity is not providing evidence in a cooperative manner, the audit team shall consult with the Engagement Executive and responsible GED in the first instance to escalate the request. The ability to use section 32 powers to obtain information is not delegated and can therefore only be exercised by the Auditor-General. Any request for the Auditor-General to use section 32 powers shall be made through the responsible GED.

215.7 Any request to engage with a Minister or ex-Minister or their staff or former officials of an audited entity to obtain audit evidence shall be approved by the responsible GED.

Guidance

215.8 Audit evidence is information obtained and used to support audit findings and conclusions. The collection, analysis and use of evidence are essential elements of the effective conduct of a performance audit.

215.9 Decisions concerning the evidence gathering process begin at the planning phase of an audit. The exercise of professional judgement concerning sufficient and appropriate audit evidence then continues throughout the audit. The audit team must assess whether the evidence obtained has the sufficient quantity and the appropriate quality. This assessment will inform the decision to obtain additional or different evidence.

215.10 Performance auditors need to have discussions with the auditees about the available evidence at the planning or conducting phases. They should ascertain the nature of the evidence and how it will need to be collected and analysed and interpreted by the audit team.

215.11 It is important that the audit team obtain evidence from a variety of sources, as different perspectives and conclusions may be presented from multiple sources. It is necessary to continually identify potential sources of evidence during the conduct of the audit. This is because not all circumstances can be foreseen during planning.

215.12 The internal controls of the systems that generate the audit evidence also need to be assessed for accuracy and consistency.

215.13 It is likely that several techniques will be used to gather audit evidence in any performance audit, such as:

  • analytical procedures;
  • document review;
  • review of email records;
  • meetings;
  • formal interviews;9
  • conducting surveys (refer to ANAO Audit Manual – PASG Specific Chapter 218 Surveys);
  • compliance testing;
  • physical observation; and
  • data analysis.

215.14 The four main types of audit evidence and their sources are set out in the table below.

Types of evidence

Sources of evidence

Physical

Physical evidence is obtained by observing people and events or examining property. It can take the form of photographs, charts or maps and detailed written descriptions of observations made.

Verbal

Verbal or testimonial evidence is obtained in the form of statements in response to inquiries or meetings. Verbal evidence can be obtained from entity staff and clients, other stakeholders or experts. Meetings conducted to obtain verbal evidence can be structured or unstructured and may involve the use of formal questionnaires and sampling techniques.

Documentary

Documentary evidence in physical or electronic form is the most common form of evidence. It may be obtained from within or outside the entity and includes such things as: policy statements and legislation; reports; reviews and evaluations; letters and minutes, emails and phone records; procedures and guidelines; risk assessment plans; planning and budget documents; contracts and leases; performance results; client feedback; computer system records; personnel documents; and organisation charts.

Analysis of information and data

Analyses of data and other information can be obtained from the entity (and may need verification) or can be generated by the audit team. It includes the analysis of ratios and trends, comparisons of procedures and operations with standards or specified requirements, and analysis of substantive testing of transactions.

   

215.15 Appropriateness of audit evidence is attributed to both the relevance and reliability of the evidence. While the assessment of sufficiency and appropriateness is a professional judgement of the auditor in each case, auditors may find it helpful to consider the following generalised statements concerning evidence appropriateness:

  • documentary evidence is more reliable than verbal evidence, but the reliability varies depending on the source and purpose of the document;
  • testimonial evidence that is corroborated in writing is more reliable than verbal evidence alone;
  • evidence based on many meetings together is more reliable than evidence based on a single or a few meetings;
  • testimonial evidence obtained under conditions in which people may speak freely is more reliable than evidence obtained under circumstances in which people may feel intimidated;
  • evidence obtained from a knowledgeable, credible and unbiased third party is more reliable than evidence obtained from the management of the audited entity or others who have a direct interest in the audited entity;
  • evidence obtained when internal control is effective is more reliable than evidence obtained when internal control is weak or non-existent;
  • evidence obtained through the auditor’s direct observation, computation and inspection is more reliable than evidence obtained indirectly; and
  • original documents are more reliable than copied documents.10

215.16 The performance auditor may also find the following generalised presumptions with respect to the sufficiency of audit evidence helpful in exercising their professional judgement in assessing the evidence obtained:

  • the greater the audit risk, the greater the quantity and quality of evidence required;
  • stronger evidence may allow less evidence to be used;
  • having a large volume of audit evidence does not compensate for a lack of relevance, validity or reliability; and
  • more evidence is normally necessary when the audited entity disagrees on ANAO’s conclusions over activities subject to audit.11

215.17 When gathering evidence, it is important to remember that while individual pieces of data or information, such as date of birth, may not be sensitive or classified, combined with a person’s address and bank account details, the information in aggregate is likely to be more sensitive and will require appropriate handling and storage for privacy and/or security reasons.

215.18 Decisions concerning the retention in audit working papers of data and the results of analysis must balance the need to retain sufficient evidence to support the audit findings and conclusions with the need to reduce the risk of unnecessarily retaining sensitive or classified information. It may be possible, for example, to delete reference to one sensitive field, such as tax file number, without affecting the sufficiency of the audit evidence in the working papers.

215.19 Audited entities will often facilitate the ANAO’s work by giving ANAO staff privileged (researcher) level access to their document management systems. It would be inappropriate to conduct an information search with the purpose of monitoring an entity’s preparation of a response to ANAO Report Preparation Papers or a proposed Auditor-General report, or gathering additional information after an audit or review has concluded.

215.20 The information-gathering powers in the A-G Act can be used to obtain information and documents required as audit evidence. However, in practice, the information-gathering powers are used as ‘reserve’ powers and access to required information is almost always obtained through cooperation with entities. ANAO Legal Services can also assist with informing entities about the nature of the information-gathering powers.

216. Sampling and selecting items for testing

Policy

216.1 When selecting a representative sample to obtain audit evidence, the auditor shall use a statistically relevant sample size, and the items shall be selected randomly.

216.2 When representative sampling is the primary method used for selecting items for testing, the standard range of 90-95% confidence level and 5% confidence interval shall be applied to obtain reasonable assurance. A lower confidence level of 80-90% may be considered and applied only when the procedures performed to which the sampling approach applies are in addition to other audit evidence obtained through separate procedures that address the same criteria.

216.3 The sample size for a representative sample shall be based upon a reasonable estimate of the proportion of the population estimated to have/not have the attribute being tested. If the test results differ from the estimated proportion, then the auditor shall assess whether an increased sample size is required.

216.4 If the population being tested is relatively small (under 250 items) or is estimated to have a high expected error rate (greater than 40%), then a sampling approach other than representative sampling shall be applied unless otherwise approved by the Auditor-General.

216.5 In considering the alternative approaches to sampling, the auditor shall:

  • evaluate the relative efficiency of each approach;
  • evaluate whether the evidence expected to be obtained will be sufficient and appropriate for the audit; and
  • document the rationale for the approach chosen.

216.6 When choosing a method for selecting items for audit testing:

  • the planned approach shall be outlined in the AWP and updated in the audit’s Progress Reviews; and
  • where a non-representative testing strategy is employed, the audit report shall provide a description of the characteristics, or attributes, of the population being tested and not assert or imply that the items tested were representative or that the sample size was statistically valid.

216.7 Audit teams shall apply the sampling approach outlined in this document, unless the circumstances require the assistance of a statistical expert and approval is obtained from the responsible Engagement Executive. A statistical expert shall be used when the attributes of the population being tested are of a more specialised nature or are outside the scope of the sample size calculator. This includes, but is not limited to, measurement of relationships between variables and the conduct of surveys.

216.8 The ANAO performance audit sampling template shall be completed when determining and documenting the approach to selecting items to test in a performance audit.

216.9 Where, in the conduct of a performance audit, an engagement team intends to obtain audit evidence about the operational effectiveness of internal controls, the non-statistical sampling approach to testing controls applied under ANAO financial statements methodology12 may be applied instead of applying paragraphs 216.1 to 216.8 where the following conditions apply:

  • the operational effectiveness of the tested control is not a criterion of the audit;
  • the control has been assessed as being designed and implemented in a manner that meets a control objective relevant to the criteria of the performance audit; and
  • the engagement team does not expect to identify exceptions to the control in their samples or, where limited expectations are expected, the sample size has been suitably adjusted in accordance with the financial statement controls testing methodology.

216.10 Under the policy at 216.9, where the operational effectiveness of the tested control is a criterion of a performance audit, the financial statement controls testing sampling methodology that applies to integrated audits13 may be applied with approval from the Engagement Executive and in consultation with PSRG.

216.11 All judgements made in determining the planned audit approach for selecting items for testing, and any subsequent revisions to the approach, shall be documented in the audit file.

Guidance

216.12 When planning and conducting a performance audit, the auditor is required to obtain sufficient appropriate evidence on which to base audit conclusions. Audit sampling means selecting a representative sample for testing with a view to making statistically appropriate conclusions about the total population. It assists the auditor by obtaining evidence ‘to provide a reasonable basis for the auditor to draw conclusions about the population from which the sample is selected’. Representative sampling is one of several methods to select items for testing.

216.13 The test item selection methods available to performance auditors are:

  • testing all items in a population (also known as census testing);
  • targeted testing based upon items with specific attributes in a population (also known as target testing); or
  • testing a representative sample of the items in a population.

216.14 Testing all items in a given population will obtain the highest level of assurance but, especially when dealing with large populations, may not be a reasonable or efficient way to conduct audit testing.

216.15 Targeting items for selection can be an efficient means to obtain audit evidence. This requires targeting items based on some attribute that is important to the audit. This may be based on quantitative factors (such as all items over a relevant threshold) or qualitative factors. The performance auditor chooses the relevant attributes to focus testing on the areas within the population with the highest level of audit risk. The total number of items tested will be determined according to auditor judgement. The performance auditor can only draw conclusions on the targeted items that have been tested and not on the untested portion of the population, and consequently auditor judgement on appropriate selection size would consider if the audit risk arising from the untested population is acceptable in the circumstances of the audit.

216.16 A representative sample requires a sample size that is statistically appropriate and the sampled items to be selected randomly. This allows the testing to be representative of the total population. This can be a very efficient way to test large populations (over 800 items). For moderately sized populations (between 250 and 800 items) the relative efficiency of a representative sampling approach will depend upon the circumstances.

216.17 The planned approach of the audit for selecting items for testing, which may include a combination of methods, is based on professional auditor judgement. The planned method(s) for selecting items for testing needs to be clearly defined and documented. This assists in determining an audit approach that is effective, efficient and repeatable.

216.18 Selecting items for audit testing may include both targeted and representative item selection methods.

216.19 The ANAO Performance Audit testing approach is embedded in the Performance Audit Sampling template and further guidance is contained within the template and incorporated instructions.

216.20 The sample size calculator assumes that the method for selecting the sample items for testing is random sampling. Therefore, any representative sampling must be performed according to a random selection approach. This excludes a convenience14 or haphazard15 sampling approach where each item in the population does not have an equal chance of being selected.

216.21 The audit testing template also assumes that the population is homogenous. It may be appropriate to target test items in the population that are not homogenous or use stratification to split the population into separate strata with different characteristics. When stratifying populations, the sample size calculator is completed for each separate population and clearly documented in the template.

216.22 If an item selected for testing is not relevant to the test and should not have been included in the population, the item must be replaced by selecting a new randomly selected item. If an item selected is not available for testing and suitable alternative audit procedures cannot be performed, e.g. documentation relating to the item is lost, this item is treated as a deficiency or error and is not to be replaced.

Procedure following testing

216.23 At the conclusion of the audit test, it is necessary to compare the estimated proportion rate with the actual percentage of errors in the sample result. If the actual result is equal to or greater than the estimate, then the result is considered to be statistically valid and the sample size is appropriate to draw conclusions from. If the result was less than the estimate, then the audit team needs to consider a larger sample size.

216.24 For example, consider a population of 800, with a confidence level of 95%, a confidence interval of 10% and an estimated error rate of 20%. An actual test result that found 10% of the attributes being tested were incorrect would be considered statistically valid for a sample size of 58 items and no additional samples would need to be tested. For the same example, with an estimated error rate of 20% where the actual error rate was 35%, the sample size would not be statistically valid and an additional 21 items would need to be tested to obtain the statistically valid sample size of 79 items in order to draw valid conclusions at the same confidence level and interval.

216.25 If the additional sample required to achieve a statistically valid sample size is large, it may be more efficient to revise the testing strategy. If no additional sample is tested, the sampled items tested so far do not represent a statistically relevant sample and therefore cannot be relied upon as audit evidence by themselves. Consequently, the results cannot be included in the audit report in a manner that asserts or infers that the results are representative of the population. However, the audit team may consider the usefulness of the testing performed as corroboration of other audit evidence gathered and consider including the results in audit reports with suitable clear caveats about the inability to extrapolate the results of testing so far to the population.

216.26 Where the actual error rate in the original sample is significantly greater than the anticipated error rate (particularly where the actual error rate is also significantly greater than what would be considered a reasonable/acceptable error rate) this may be evidence that the process being tested has significant deficiencies related to how the process has been designed and implemented that the audit team had not detected. The audit team should re-examine these matters before determining the appropriate audit response.

Figure 1:

Figure 1
Sampling approaches to testing of controls

216.27 Performance audits may in some circumstances include gathering evidence that a control has been operating effectively. Audit evidence about controls effectiveness can be used by performance auditors to make relevant conclusions about the effectiveness, efficiency, economy and ethics of activities.

216.28 Where financial statement auditors have already performed audit procedures as part of their audit work, this policy allows performance auditors to rely on that work without dealing with the complexity of adjusting sample sizes to comply with the standard performance audit sampling methodology.

216.29 Alternatively, performance auditors may find it convenient to engage with financial statement auditors, or systems assurance specialists from SADA, to perform control testing procedures in accordance with the sampling methodology that they are experienced in using.

216.30 However, performance auditors may only apply the financial statement controls sampling methodology where the key assumptions underpinning this approach are appropriate to the performance audit purpose of the testing in accordance with the policy requirements specified in 216.9. These key assumptions include:

  • that the testing is intended only as supporting evidence about the conclusions being drawn by the auditor and not as evidence of the audit criteria itself (that is, controls testing is supported by additional evidence as to the audit criteria);
  • that the control being tested has been designed and implemented appropriately (that is, that the control, if applied consistently throughout the period relevant to the audit conclusion, would achieve the control objective on which the auditor intends to rely); and
  • that the auditor identifies no or, in rare cases, very low levels of exceptions in the control effectiveness in testing.

216.31 Under the financial statement methodology, exceptions identified in the testing will cause the auditor to not be able to rely on the control, except in circumstances where a very small number of exceptions are identified and additional samples are tested in accordance with the methodology prescribed in the Audit Guide. This may not be an efficient and effective manner of obtaining audit evidence so should be carefully considered before being applied.

216.32 While not relevant to the financial statement mandate of the ANAO, the financial statement controls sampling methodology provides approaches to test controls for the purposes of concluding on the effectiveness of the control itself as part of the audit scope. This would be relevant where control effectiveness is a criterion of the performance audit. Given that this is a rarely used feature of ANAO’s financial statement methodology, the policy requires technical consultation with PSRG before it is used.

217. Audit documentation: Protective marking and bulk collection of entity emails

Background

217.1 The A-G Act (section 33) provides for full and free access to any documents or other property. Documentary evidence can include material that has a ‘PROTECTED’ protective marking, including ‘PROTECTED CABINET’. The Auditor-General may also, by written notice, direct a person to provide information, produce documents and give evidence. However, most audit evidence is obtained through a cooperative approach with the audited entity.

Policy

Application of ANAO Protective Security framework

217.2 All staff shall comply with the ANAO’s Protective Security Governance Policy in the conduct of their audits.

217.3 When collecting, analysing, and documenting evidence (including the bulk collection and analysis of entity emails) and drafting the audit report, all audit staff shall comply with the ANAO’s Information Security Policy and Information, Classification and Handling Guidelines.

Dealing with Cabinet Material

217.4 When dealing with Cabinet documents,16 audit teams shall have regard to:

  • ANAO Protocol on use of Cabinet-in-Confidence Information in Audit Reports
  • Practice Note: Use of Cabinet Material in Performance Audit Reports; and
  • Access to Cabinet Documents Policy and Procedures.

217.5 In applying the requirements of 217.4, audit teams shall:

  • ensure that Cabinet documents are requested, handled and disposed in accordance with Cabinet requirements as they apply to the ANAO;
  • through the Engagement Executive and responsible GED, advise the Auditor-General when the audit team expects that the performance audit report will refer to Cabinet material. This will commonly occur through the section 19 workshop process but may occur separately where deemed appropriate by the Engagement Executive. However, this advice is communicated, the audit file shall document the Auditor-General’s involvement,17 including the audit team’s advice as to whether:
    1. it will be appropriate to refer to the material in the final report without explicitly identifying it as Cabinet information; or
    2. explicit discussion of Cabinet deliberations, processes and/or outcomes is likely to be required; and
  • ensure that the approval by the Auditor-General for the inclusion of Cabinet material in an audit file is documented in the audit file.
Bulk Collection of Data, including Emails

217.6 When a performance audit collects the emails of audited entities on a bulk basis18 the audit team shall ensure that auditee information is held, accessed, analysed and reported by the ANAO in a manner that meets the audit objective with the minimum reasonable risk posed to the confidentiality and privacy19 of individual auditee staff, contractors and other stakeholders.

217.7 This policy also applies to any other circumstance where bulk data is obtained for a performance audit where personal data is likely to be included, including other forms of electronic correspondence being collected on a bulk basis, such as instant messaging or telephone records.

Guidance

Application of ANAO Protective Security framework

217.8 The ANAO’s Protective Security framework, as specified in the Protective Security Governance Policy, applies to all data held by the ANAO, including evidence gathered in the process of planning, execution and reporting of performance audits.

Protocol for ANAO Use of Cabinet Material in Performance Audit Reports – Obtaining and Handling Cabinet Material

217.9 Sections 32 and 33 of the A-G Act provide the Auditor-General with broad information-gathering powers that extend to any relevant document or information, regardless of their classification or status. However, in practice, the information-gathering powers are used as ‘reserve’ powers and access to required information is almost always obtained through cooperation with entities, including Cabinet materials.

  • ANAO staff should seek to obtain information on a cooperative basis.
  • If an entity is not cooperating, consult with the Engagement Executive and responsible GED in the first instance to escalate the request. They will consider further options, including as necessary possible use of either section 33 or requesting that the Auditor-General uses section 32 to obtain the necessary information. ANAO Legal Services can also assist with informing entities about the nature of the information-gathering powers.
  • ANAO staff must at all times treat information in their custody appropriately and in accordance with the Protective Security Policy Framework as specified in the Information, Classification and Handling Guidelines.
  • Section 36 of the A-G Act ensures protection for any information gathered by making it an offence to disclose information gained in the course of conducting an Auditor-General function for any purpose other than performing that function.
  • The A-G Act also recognises that it may not be in the public interest for the Auditor-General to disclose some sensitive information.

217.10 In accordance with Cabinet material rules, CMG maintains documented processes for the requesting, handling and disposal of Cabinet materials which are documented on Audit Central.

Protocol for ANAO Use of Cabinet Material in Performance Audit Reports – Reporting on Cabinet Material

217.11 The ANAO has developed a protocol and practice note that summarises the circumstances in which the Auditor-General may decide to publish Cabinet material in performance audit reports or other audit products prepared under the A-G Act20 for tabling in the Parliament.

217.12 The A-G Act also recognises that it may not be in the public interest for the Auditor-General to disclose some sensitive information.21 When considering the publication of sensitive information, it is the Auditor-General’s role to determine whether the public interest is best served by disclosure.

217.13 As an independent officer of the Parliament, the Auditor-General has discretion in the performance or exercise of his functions or powers. In particular, the Auditor-General is not subject to direction in relation to whether or not a particular audit is to be conducted, or the way in which a particular audit is to be conducted.

217.14 The Auditor-General undertakes the following steps when considering the publication of Cabinet material:

  • as a matter of sound public administration, and with due regard to the long held Westminster tradition of respecting Cabinet confidentiality, the Auditor-General only publishes Cabinet material occasionally, and only when the material in question is fundamental to the conclusions of the audit;
  • ANAO officers provide written advice and consult with the Auditor-General about the potential need to include any such material;
  • the proposed inclusion of any such material is drawn to the attention of the auditee’s Accountable Authority during report preparation and when the Accountable Authority is provided with a draft final report for review and response; and
  • the Auditor-General seeks to avoid direct quotations from deliberative documents of the Cabinet and, to the extent that it is possible, publishes reports that attribute decisions or outcomes to the responsible minister or government rather than to the Cabinet.

217.15 Any explicit discussion of Cabinet deliberations, processes or outcomes is included only when, in the considered view of the Auditor-General, the conduct of those deliberations and processes, or the associated outcomes, led directly to a conclusion expressed in the final audit report.

217.16 To ensure consistency with the protocol, audit teams must follow these steps:

  • ensure at all times that any Cabinet material accessed or gathered during the course of an audit is treated and stored appropriately and in accordance with the Information, Classification and Handling Guidelines. If necessary, the ANAO Security Adviser (in CMG) can be contacted for advice and assistance;
  • when requesting access to Cabinet documents, which should be through the Cabinet Division of the Department of the Prime Minister and Cabinet (PM&C), please ensure the document numbers are identified (where possible) and the requested documents are relevant to the audit, as determined by the Auditor-General;
  • in preparing any such documentation, due regard must be given to security and information protection obligations. Audit teams should consult the Entity Security Adviser as necessary;
  • in most cases, references to Cabinet material in audit reports should use generic and minimal descriptions (i.e. ‘government’), other than when fundamental to audit conclusions; and
  • if a draft audit report explicitly identifies Cabinet material, then the relevant GED or Executive Director should inform their counterparts in the Department of the Prime Minister and Cabinet of any intended publication of Cabinet material.
Bulk Collection of Entity Emails

217.17 Entities’ email systems are used in the administration of programs and activities. As email systems and the information and records they contain are legally the property of the entity, the ANAO is entitled to access this material in the same way as other entity records and information. Email records almost inevitably contain personal communications. As a result, the ANAO’s access will generally involve access to both official and personal communications. This can cause heightened sensitivities and may, at times, lead to disagreement between the audit team and the entity that needs to be properly managed, often at a senior level.

217.18 Without appropriate care and attention, the ANAO’s collection of bulk email records also poses the risk that audit team members do not deal with personal information in an ethical manner or in a manner that is not seen to be ethical.

217.19 When bulk email records are collected audit teams should consider:

  • exploring ways of electronically searching records for relevant information in a way that limits access to personal information, and establishing internal team protocols to manage the risk that personal information is accessed and breaches to the need-to-know basis, including unreasonable access to personal information not related to the relevant audit questions;
  • that relevant email records may be archived and the retrieval may involve longer timeframes and additional costs compared with the retrieval of paper files. In this context, the ANAO does not accept responsibility for entity costs arising from the conduct of audits, as section 33 of the A-G Act provides for ‘full and free access at all reasonable times to any documents or other property’. Nevertheless, it is expected that audit teams would be conscious of the costs that may be incurred by entities in meeting requests for access to records, and would seek to minimise these, to the extent possible;
  • whether it may be appropriate to provide assurances to entity management that any personal information accessed will not be used in any way and will not be incorporated into the ANAO’s working papers; and
  • whether it is appropriate to limit access to entity bulk email records to nominated members of the audit team and/or establishing separate security arrangements for the: storage of bulk entity emails during the audit; and destruction of emails not used as key evidence by the ANAO.

217.20 Such operational protocols may also be considered necessary to provide assurance regarding certain paper records, such as records containing highly personal information pursuant to disclosure regimes.

217.21 A balance needs to be achieved between meeting the ANAO’s responsibilities, including conducting audits in an efficient and objective manner, and recognising the legitimate concerns of entity management. This, at times, may involve the audit team explaining, in more detail than would normally be the case, the ANAO’s audit and confidentiality responsibilities and the reasons why access to certain entity information and records is necessary to meet these audit responsibilities. Potential issues in accessing data should be escalated early.

218. Surveys

Policy

218.1 Audit teams shall document their intention to conduct a survey in the AWP, if known at that time, or relevant progress review brief.

218.2 The AWP or relevant progress review brief shall include details on the design and delivery of the survey, and the estimated costs of the survey in the proposed budget.

218.3 The audit team shall also discuss the planned survey with the auditee entity and document the outcomes of the discussion.

Guidance

218.4 From time to time, audit teams may seek to conduct a survey to gather information from a large population to inform the scope or direction of an audit, or to provide insights into the perceived performance of a program or entity.

218.5 Approaches to conducting a survey include:

  • the audit team designing, delivering and analysing the survey (subject to the audit team including members with the appropriate experience and expertise);
  • contracting an expert to design a survey, with the audit team delivering and analysing the survey; or
  • contracting an expert to design and deliver a survey, with the audit team analysing the survey.

218.6 Surveys can be conducted using an online tool or other means, such as a paper-based survey, telephone interviews or in-person interviews. Key considerations that audit teams should have when determining the survey approach include:

  • the information security associated with proposed online survey tools;
  • the information protection arrangements in place. At a minimum, any online survey tool used should offer SSL encryption to protect the privacy and integrity of collected data;
  • an awareness of and being transparent about the data storage location; and
  • any survey tool indemnity clauses that would require approval by a PGPA Act section 60 delegate.

218.7 Audit teams should compare and evaluate the options offered by various tools in terms of question design options, technical support and data reporting formats.

218.8 As for all procurements, audit teams should appropriately weigh up the costs and benefits of various options, ensure that they select a provider or tool that is fit for purpose and provides value for money, and clearly document the reasons for their decision. Any procurement plan should be endorsed by the PASG Practice Manager.

218.9 Audit teams should take the following steps to mitigate the risk of information leakage and manage stakeholder expectations of privacy and security:

  • inform potential survey respondents about the arrangements that are in place to protect and store their responses, and should disclose the involvement of any third-party provider(s);
  • minimise the collection of personal information, information about respondent organisations or contact details; and
  • provide participants with alternative survey completion options (e.g. offline or by telephone).

219. Verifying audit evidence

Policy

219.1 Audit teams shall adopt an attitude of professional scepticism in making independent judgements about audit evidence during an audit.

219.2 When using information produced by the entity to perform audit procedures and gather audit evidence, the audit team shall evaluate whether the information is sufficiently reliable for the purpose of the audit, including obtaining audit evidence about the accuracy and completeness of the information.

Guidance

219.3 Verifying audit evidence is the first step in analysing audit evidence to ensure that audit findings, conclusions and recommendations are based on sound evidence.

219.4 The risk, significance and sensitivity of the matter to be reported will determine not only the nature and amount of evidence to be collected but the extent of verification.

219.5 Assessing information obtained from entity records is particularly important in circumstances where management information, such as program expenditures or key performance indicators, is to be relied on. In these cases, it is essential that the completeness and accuracy of this information is assessed through appropriate audit testing to give the ANAO confidence that the systems and processes used to produce the information can be relied on. This will involve reviewing key system controls and testing a sample of transactions.

219.6 Audit testing is required to be undertaken in these circumstances irrespective of when such information is obtained. For example, if information is provided in response to a section 19 report that is significant in settling the final audit conclusions, it needs to be assessed. When management information is contextual and is not significant in the context of the audit findings and conclusions, it is not necessary to test its completeness and accuracy. However, in these situations it is expected that the audit report would indicate that the information has been sourced from entity records or is based on entity advice, as distinct from audit analysis.

219.7 The ANAO’s policy in relation to sufficient appropriate audit evidence also applies to any work performed by an expert. When using the work of an expert to support audit findings and conclusions, it must be adequate for that purpose, recognising that the responsibility for the findings generated or conclusions drawn from the work undertaken by an expert rests solely with the ANAO.

219.8 Professional scepticism (ASAE 3500 paragraphs 16(p), A57) means an attitude that includes a questioning mind, and being alert to:

  • conditions which may indicate possible deficiencies in administration;
  • audit evidence that contradicts other evidence obtained;
  • information that brings into question the reliability of documents and responses to enquiries;
  • conditions that may indicate systemic system deficiencies; and
  • circumstances that suggest the need for further analysis and enquiry.

219.9 The type of audit evidence gathered will determine the verification approaches available:

Type of evidence

Verification technique

Physical

The observation is recorded using, for example, image capture technologies.

Verbal/testimonial

The attendees confirm in writing the facts of the discussion.

A number of independent sources confirm the evidence; for example, meetings with staff, documentation and expert advice.

A survey provides valid results.

The facts of a discussion are confirmed from other sources. The verbal evidence is recorded.

Documentary

Documentary evidence is obtained from more than one source.

Documentary evidence is the final version and has the appropriate approvals and sign-offs.

The matters set out in the document, for example, performance results, are tested for accuracy.

Internal system controls are tested or electronic records are validated.

Adherence to procedures or plans is tested to determine that they are followed in practice.

Internally generated reports

Ensure that the report is a standard system report, unable to be changed by the auditee.

Technical re-performance using IT specialists.

Samples of data are tested for accuracy, completeness and validity.

Further guidance on these approaches is provided below.

   

219.10 There are a number of different approaches to test completeness and accuracy of internally generated reports, and each approach provides a varying level of comfort. Professional judgement is required to determine the most appropriate method in the circumstances based on an understanding of how the report is generated and the desired level of comfort.

Standard system reports

219.11 The approach which provides the highest level of evidence is where the auditee is utilising standard system reports. When the controls for producing the reports have been tested, then there is a reasonable expectation that any errors would have been identified by the system.

219.12 Typical considerations to document include:

  • auditee does not have a development environment, or access to such an environment is limited to external vendor accounts;
  • review of change management logs shows no changes except vendor management;
  • collaborative enquiry with different sources within the organisation confirms that the auditee does not undertake development/have access to the application source code;
  • ANAO understanding of the application based on knowledge and experience is that the application is ‘off-the-shelf’ and vendor supported; and
  • ANAO review of the standard user guides identifies the specific reports/controls to be part of the packaged application.

219.13 These are considerations only. It is important that the ANAO is comfortable that the standard reports are unable to be changed by the auditee and only by the vendor.

Technical re-performance

219.14 Technical re-performance involves reviewing the code (e.g. SQL query) which is used by the system to produce the report. This can only be performed when the auditee is able to access the code, and when the audit team has appropriately skilled team members who can understand the code.

Sample testing

219.15 Samples of data are tested for accuracy, completeness and validity. The procedure for this testing is detailed in the Sampling and selecting items for testing chapter of this manual.

219.16 With all approaches, there will typically be parameters applied to reports within a system. These parameters could be date ranges, product types, cost centres, etc. It is important that these parameters are reviewed as part of the process of obtaining comfort over the system reports.

220. Forming the audit conclusion

Policy

220.1 The audit team shall form the overall audit conclusion with regard to the objective of the audit, as evaluated against the criteria.

220.2 In accordance with the policy at paragraph 224.5 – 224.6, the overall conclusion is the first sentence under the Conclusion heading of the Summary and Recommendations chapter. It shall directly address the question of whether or not the audit objective has been met and, if not, be specific about the findings that resulted in exceptions to the conclusion.

220.3 The paragraphs following the overall conclusion shall be identical to the ‘conclusion’ section of the blue summary boxes in subsequent chapters. These conclusions shall directly address the question of whether or not the individual criteria have been met and, if not, be specific about the findings that resulted in exceptions to the conclusion.

220.4 Materiality shall be considered when evaluating the effect on the conclusion(s) of any identified findings, taken individually and in combination.

220.5 Only findings determined to be material shall be presented as exceptions to the conclusion(s) and included in the Conclusion section of the report. Other findings shall be presented in the supporting findings section of the report.

220.6 In accordance with the requirement at 224.2 for the report to be balanced, the conclusion section shall also include reference to evidence of good performance that has had a material impact on the conclusion; that is, the conclusion shall specify the material matters that resulted in the objective and/or criteria being met.

Guidance

220.7 The following diagram outlines the audit process and outcomes.

Audit process and outcomes
Audit findings

220.8 Audit findings are generated when the criteria (and sub-criteria) are compared with the audit evidence. Meeting or exceeding the criteria may indicate good practice leading to good performance. Failing to meet criteria would indicate that improvements are needed. It is, however, unrealistic to expect that the audited entity’s performance regarding economy, efficiency, effectiveness and ethics will always meet the criteria. This means that in addition to assessing whether the audited entity meets the criteria or not, the audit team also has to consider materiality and apply professional judgment in interpreting how this affects assessment of the entity’s performance.

220.9 The audit report should include both positive and negative points and give credit where it is due. Including positive aspects may lead to improved performance by other government organisations that read the report. It is important that the report contains all the information and arguments needed to satisfy the audit objective(s) and promote adequate and correct understanding of the matters and conditions reported.

220.10 Audits involve some type of analysis in order to understand or explain what has been observed. When analysing information collected, the auditor should focus on the criteria and objective. This will help to organise the data and also provide the focus for analysis.

220.11 While it is important to seek explanations for deviations from criteria, causes should be presented with caution. They have to be supported by sufficient and appropriate audit evidence. It is relevant to consider the audited entity’s views on reasons for performance problems or weaknesses. If such views are not supported by sufficient and appropriate audit evidence, the audit team cannot take for granted that they are relevant or correct.

220.12 The audit team should identify the possible effects of the criteria not being met. The effects could be identified either as what has already occurred or as possible future impact. The nature of the findings determines whether the audit team can present actual or potential effects.

Developing conclusions after considering the findings

220.13 Once the findings have been established and causes and effects considered, the audit team draws conclusions against the sub-criteria, criteria and objective respectively. Conclusions are statements informed by the findings. Since performance audits may point out performance relating to deficiencies in aspects of economy, efficiency, effectiveness and/or ethics, the conclusions have to specify the reasons why aspects of economy, efficiency, effectiveness or ethics may not have been fully met.

220.14 Audit conclusions clarify and add meaning to specific findings in the report. Conclusions present the opinion and go beyond merely restating the findings. Whereas the audit findings are identified by comparing ‘what should be’ according to the criteria with the audit evidence (including analytical evidence) on ‘what is’, the conclusions also reflect the auditor’s explanations and views based on these findings. Conclusions might include identifying a general theme or a certain pattern in the findings. An underlying problem that explains the findings may also be identified.

220.15 When drawing conclusions, it will often be necessary to revisit the data analysis and the audit findings to be sure that the conclusions are based on solid grounds. The analysis of data consists of combining results from different types of sources. The conclusions are based on the objective, criteria, evidence and findings.

220.16 The consideration of materiality is a matter of professional judgement. Materiality is considered in the context of qualitative factors and, when applicable, quantitative factors. The relative importance of qualitative factors and quantitative factors when considering materiality in a particular performance engagement is a matter for professional judgement. This judgement may be informed by whether the issue has had a substantive impact on the quality and/or cost efficiency of the services or policy advice provided – i.e., the primary objectives of the entity.

220.17 Concluding on the materiality of the findings identified as a result of the procedures performed requires professional judgement. For example: three criteria are identified for an audit. For one of these criteria, there are two sub-criteria. If sub-criteria one is satisfied, but the sub-criteria two is not, a finding will be reported in respect of sub-criteria two as it would be of interest to readers of the report. Professional judgement will be applied to determine the appropriate level of prominence for this finding in the audit report, given its actual or potential impact. That is, whether it will be presented within supporting findings or as an exception to the conclusion. This professional judgment will also consider the relative importance of this sub-criterion to the criterion to which it relates as well as the relative importance of that criterion to the other criteria and findings in respect of the other criteria.

220.18 In addition, professional judgement will consider whether there are factors that resulted in the satisfaction of sub-criteria one that are material to the conclusion that should be reported.

220.19 Findings are considered to be material to the overall conclusion if they, individually or in the aggregate, could reasonably be expected to change or influence the decisions of users of the report, such as the legislature or executive. Where a finding or findings is/are determined to be material to the overall conclusion, the conclusion will either be expressed in the form of the objective being met with exceptions or the objective not being met. For example, when there are findings in respect of one criterion only, it is likely that the objective would be found to be met with exceptions and the conclusion would be expressed as ‘the program was implemented effectively except for…’. Where the findings are so pervasive to the criteria and therefore objective of the audit, the conclusion would be that the objective was met and expressed as ‘the program was not implemented effectively because of …’.

221. Quality management

Background

221.1 A system of quality management is required to provide assurance that the audit team’s work has met the requirements of the ANAO Auditing Standards and complies with performance audit policies and related procedures.

Policy

221.2 There shall be a system of quality management to provide assurance that all audits have been conducted in accordance with performance audit policies and related procedures. Evidence that key elements of quality management have been undertaken shall be documented by relevant staff using the Performance Audit Report Policy Compliance Certification. The Performance Audit Report Policy Compliance Certification must be completed after the auditee’s response to section 19 report has been received and prior to the draft final report being provided to the Auditor-General for clearance.

221.3 Where work is delegated to members of an audit team, the Engagement Executive and/or the Audit Manager shall carefully direct, supervise and review the work undertaken to assist with the effective implementation of the audit. The Engagement Executive shall conduct a sufficient review of the audit team’s work to provide assurance that the audit has been properly performed and appropriate evidence-based conclusions reached.

221.4 Supervision and review of the work of experts shall also be undertaken to assess compliance with performance audit policies and related procedures. The audit team shall make enquiries and test the views expressed by the expert to the extent necessary to be satisfied that the findings and conclusions are supportable and appropriate.

221.5 The AWP, Report Preparation Papers and draft reports shall be subject to review, including that appropriate approvals were obtained and all significant issues were discussed with the Auditor-General.

221.6 The evidence contained in the working papers to support each of the key issues/findings shall be subject to review by the Audit Manager and the Executive Director.

Guidance

221.7 A system of quality management for performance audits is consistent with the ANAO Quality Assurance Framework and Plan.

221.8 Reviewing the audit coverage and findings provides assurance that all relevant issues and considerations have been addressed and that sufficient appropriate evidence was obtained for all issues that impact on the program or activity subject to audit and support the audit findings and conclusions.

221.9 A key element in the quality management system for performance audits is that the audit working papers should be subject to review. Working papers include all the planning documents, evidence collected and analysis undertaken during the course of an audit, and all documents relating to reporting for an audit.

221.10 The extent and nature of the review of the working papers should be sufficient for the reviewer to be satisfied that the audit findings and conclusions are supported by audit evidence, that they demonstrate an in-depth knowledge and understanding of the subject matter and that they are presented in a balanced, fair and constructive manner. Cross-referencing and filing all working papers facilitate the review of audit evidence and working papers.

221.11 The quality management system described above does not require all audit working papers to be reviewed by the Engagement Executive or more senior members of the audit team. The Engagement Executive may delegate review of some audit work to the Audit Manager or other more senior members of the team with suitable skills and experience, but the Engagement Executive retains overall responsibility for the review and supervision of the engagement. Ordinarily, the Engagement Executive reviews the higher risk aspects of the audit in addition to all those areas that specifically require Engagement Executive review in accordance with the ANAO Auditing Standards and this policy.

221.12 Reflecting that the entity subject to audit also has an important role to play in the audit process, audit working papers should indicate the extent to which the entity was consulted during the planning phase of the audit and evidence of the interaction with the entity about the audit findings, conclusions and recommendations throughout the audit.

221.13 Where the work of an expert is being used as evidence in an audit, the audit team retains full responsibility for the conclusions drawn from that evidence and presented in the audit report, recognising that it is ultimately the Auditor-General who signs each report and therefore will make the final judgement about the audit findings and conclusions.

222. Report Preparation Papers

Policy

222.1 Report Preparation Papers (RPPs) shall address the audit objective(s) and audit criteria. The RPPs shall be sufficiently detailed to allow the entity to understand the basis for, and facilitate meaningful understanding and/or discussion of, the preliminary audit findings, conclusions and any potential recommendations.

222.2 All RPPs shall be prepared in accordance with the Service Group’s current ‘PASG Report’ template style and format.

222.3 RPPs shall be provided to the entity in accordance with the PASG Workflow.

222.4 In circumstances where RPPs are not provided to the entity, for example, a short duration audit where the entity was formally briefed about the finding, preliminary conclusions and possible recommendations before issuing a section 19 report, the reasons for not providing the Report Preparation Papers shall be documented.

222.5 RPPs shall be provided to FSASG. Briefings shall also be provided to FSASG and/or PSASG on significant issues as they arise in the course of a performance audit and as Report Preparation Papers are prepared.

222.6 Draft RPPs shall be cross-referenced to the audit evidence and documented in the audit working papers.

Guidance

222.7 The audit team’s review and assessment of audit findings and conclusions, drawing on the audit evidence gathered and analysed in the delivery phase of the audit, form the basis of the Report Preparation Papers.

222.8 Specific procedures for the clearance and issuance of RPPs are included in the PASG Workflow, which contains corporate templates for staff to use during an audit.

222.9 RPPs, or equivalent working papers for audits of multiple entities, will generally be the first formal opportunity for the entity to consider the ANAO’s preliminary audit findings, conclusions and proposed recommendations. It is important that the entity understands that this is not the formal section 19 report. The audit team should explain to the entity the purpose of the RPPs, preferably in person to entity management, as well as in the correspondence that accompanies the RPPs.

222.10 RPPs can be prepared while audit evidence is being collected or towards the end of this phase when the majority of evidence has been collected. Drafting report preparation and evidence papers during the delivery phase and providing them to the entity for comment on an ongoing basis presents the opportunity to test the facts and findings at an early stage and to identify the need for more information where this is required.

222.11 RPPs, particularly when issued prior to the completion of fieldwork, can also be used to elicit further information and/or to seek further views from the entity on particular issues that are important to the audit.

222.12 RPPs provide the entity with the opportunity to correct any errors of fact or interpretation, either in writing or at the exit meeting. The entity can also provide additional information in response to this correspondence.

222.13 The iterative review of RPPs can be extensive for some audits. Audit teams are to make judgements regarding the number of reviewed versions of the RPPs that are required to be retained on the audit working papers. For example, drafts that contain minor editorial changes and comments only may not need to be retained. Any substantive edits or comments made by the GED or ANAO Executive are required to be retained.

222.14 Draft RPPs that contain substantive review comments and edits made by the responsible Executive Director and Group Executive Director are to be retained on the audit working papers. The comments and edits made by the ANAO Executive are required to be retained as evidence that the work performed by the audit team has been directed, supervised and reviewed, as well as to retain a documented record of the audit reporting process.

222.15 For audits that do not issue RPPs, a written summary of the key findings can be sent to the entity for discussion at the exit meeting. For efficiency, the summary of key preliminary findings can be based on the content of the Progress Review 2 briefing.

222.16 Preliminary findings, conclusions and proposed recommendations contained in the Draft RPPs should not be a surprise to the entity given sufficient and comprehensive verbal briefings throughout the audit.

222.17 In the case of cross-entity audits or audits containing information about third persons or entities, audit teams will need to consider the preparation of extracts of RPPs. The extract strategy should be agreed with the Engagement Executive and GED if necessary and should have regard to the objective of providing sufficient information to recipients to respond meaningfully to the RPP, while avoiding the release to third parties and maintaining the ANAO’s confidentiality obligations.

222.18 In determining what is a significant issue requiring communication with FSASG, PASG shall consider:

  • the potential to give rise to a material mistake in the financial statements;
  • any actual or suspected fraud;
  • any issues revealing a weakness in key financial controls; and
  • any other issues of which the audit team is aware that may be of interest to FSASG.

222.19 In determining what is a significant issue requiring communication with PSASG, PASG shall consider:

  • whether there is a current performance statement audit underway or being considered for the next audit cycle;
  • the potential to give rise to a material misstatement in the performance statements;
  • any actual or suspected fraud;
  • any issues revealing a weakness in key controls related to the setting of performance measures or the preparation, monitoring and reporting of the entity’s performance; and
  • any other issues of which the audit team is aware that may be of interest to PSASG.

222.20 The ANAO Communication Unit is available to assist with any queries regarding the writing of audit reports in terms of format and style. Audit teams can also refer to the ANAO Style Guide.

222.21 RPPs should only be sent to a single officer of the entity. If that officer is not the accountable authority, director or member of the governing body, that officer would be referred to in the designation email so that the accountable authority, director or member of the governing body is aware of who the single officer will be. This will reduce the risk of accidental disclosure and give the relevant officer control over who receives the RPPs. Other officers may be sent an email informing them that the RPPs have been provided.

223. Exit meetings

Policy

223.1 An exit meeting shall be held with each entity subject to audit unless the entity declines the invitation or it is impractical to do so. For example, in the case where the audit approach involved a large number of entities participating in a survey, it may not be practical to conduct an exit meeting with each entity.

223.2 RPPs, or their equivalent for audits of multiple entities, shall be provided prior to the exit meeting to allow the entity sufficient time to review and formulate a considered response to the preliminary audit findings, tentative conclusions and proposed recommendations.

223.3 A record of the exit meeting, including the outcomes, any sensitivities and all significant decisions made, shall be documented in the audit working papers prior to the finalisation of the proposed section 19 report.

Guidance

223.4 Exit meetings are an important step in the audit process and occur at the end of the period for the major collection of audit evidence, analysis of findings and identification of potential recommendations for the entity.

223.5 Where an exit meeting is to be held, it can be a forum to:

  • give the entity an indication of what the final report will include;
  • give the entity the opportunity to provide its perspective on the preliminary audit findings and to correct any errors of fact, interpretation or perception; and
  • provide entity management with the opportunity to challenge proposed recommendations. The recommendations may then be adjusted, removed, or new recommendations developed in light of the additional information provided; and
  • give the entity the opportunity to provide additional evidence, and assess whether any audit findings should be modified in light of the additional information provided.

223.6 ANAO staff attendance at the exit meeting, in addition to the audit team members, is to be determined on a case-by-case basis by the Engagement Executive.

223.7 Entities should be encouraged to invite senior representatives to exit meetings.

Engagement performance — reporting

Chapters 224 to 227

224. Proposed section 19 report

Background

224.1 The proposed report is referred to as the section 19 report, because it is required by section 19 of the A-G Act.

Policy

224.2 The audit team shall draft a section 19 report that addresses issues that are relevant to the audit objective and the audit criteria. Audit reports shall be comprehensive, convincing, timely, reader friendly, and balanced.

224.3 All audit reports shall contain a report summary that consists of: background (including key information about the audit, audit objectives and criteria); conclusion; supporting findings; recommendations (where appropriate); summary of the entity response; and key learnings for Australian Government entities.

224.4 The background section shall outline the rationale for undertaking the audit, the audit objective and high-level audit criteria, to provide the report user with information about the basis on which the audit findings and conclusions have been made. The high-level criteria included in the report summary shall be fleshed out in the main body of the report, including identifying the sources of the criteria (the framework against which the criteria were audited)22.

224.5 The overall conclusion shall be framed against the audit objective, and directly address the question of whether or not the audit objective has been met and, if not, be specific about aspects or matters that require improvement.

224.6 The overall conclusion shall:

  • be expressed in the first sentence under the Conclusion heading of the Summary and Recommendations chapter;
  • contain paragraphs identical to the ‘conclusion’ section of the blue summary boxes in subsequent chapters;
  • summarise areas where the entity could improve its performance; and
  • present the significant matters and issues in a concise and balanced way and link them in a logical manner—this avoids the appearance of presenting a list of unrelated matters.

224.7 The report shall not directly quote International Organization for Standardization (ISO) standards without explicit permission from ISO as this would carry a significant breach of copyright.

224.8 In the draft section 19 report prepared for review by the Engagement Executive, Group Executive Director and the ANAO Executive, the findings in the section 19 report shall be cross-referenced to the audit evidence.

224.9 The section 19 report shall also include the following:

  • methods of data gathering and analysis applied;
  • time period covered;
  • sources of data; and
  • limitations to the data used.

224.10 Audit teams shall retain evidence of review by the Engagement Executive, Group Executive Director or the ANAO Executive of the draft section 19 report as part of audit working papers.

224.11 There shall be a section 19 Workshop meeting for each performance audit. The Auditor-General, Deputy Auditor-General, responsible GED, Engagement Executive, SADA Executive and the Engagement Executives for the auditee’s financial and performance statement (where there is a concurrent23 performance statement audit) audits shall be invited.

224.12 A record of the section 19 Workshop meeting shall be included in the audit file, which will be the draft section 19 report annotated with changes made from the draft provided by the Engagement team for discussion and any other key judgements.

224.13 The section 19 report shall be forwarded through the Engagement Executive to the responsible GED, the Deputy Auditor-General and the Auditor-General for discussion at the section 19 meeting, agreement and approval.

224.14 All section 19 reports shall be reviewed by the Engagement Executive and provided to the responsible GED, Deputy Auditor-General and Auditor-General concurrently for clearance. Following the incorporation of any comments, the Engagement Executive and, as necessary, the GED, reviews the section 19 report for final approval and formal sign-off by the Auditor-General.

224.15 In circumstances where they are considered to require amendment, approval to amend the audit criteria shall be sought from the Auditor-General.

224.16 The section 19 report shall be sent from the Office of the Auditor-General - Performance Audit email address, using the Auditor-General’s signature block.

224.17 For performance audits of individual Commonwealth entities, Commonwealth companies, subsidiaries or Commonwealth partners under sections 17, 18A or 18B of the A-G Act, the complete section 19 report shall be sent to the auditee.

224.18 For cross-entity performance audits conducted under section 18 of the A-G Act, arrangements for providing the section 19 report (either a complete version of the report or relevant extract(s) only) shall be determined in consultation with the Auditor-General.

224.19 Legal and procedural fairness obligations shall be fulfilled for all audits. The ANAO shall consider written comments provided within 28 days by the entity, or persons or bodies with a special interest in the section 19 report or extract, before finalising the audit report (section 19(7) of the A-G Act).

224.20 Where a person or organisation has a special interest in all or part of the section 19 report, the Auditor-General shall determine whether to provide that person or organisation with a copy of the section 19 report, or relevant extract, for comment.

224.21 Where an extract is to be provided to a person or organisation with a special interest, the Auditor-General shall approve the approach taken by the team to determine the content of the extract to be provided. Written notice shall be provided to the Auditor-General, through the GED, describing the approach proposed, with a copy of the extract attached.

224.22 Extracts from section 19 reports shall include sufficient material to allow the person or organisation receiving it to understand and comment on the accuracy of the facts that are relevant to them, and the context in which they are presented, in an informed way. Unless approved by the Auditor-General, the extract shall not disclose:

  • Cabinet material;
  • security classified material;
  • briefing notes, advice or other material provided in confidence to Government;
  • confidential material of other parties; or
  • the audit conclusion.

224.23 To meet the requirements of the A-G Act, any written comments received on the section 19 report within 28 calendar days of the section 19 report being received by the recipient shall be given full consideration by the audit team in preparing the final audit report.24

224.24 Key learnings for other government entities shall, unless the contrary is approved by the relevant GED, be incorporated into the summary section.

224.25 Any substantive edits or comments made by the GED and ANAO Executive shall be retained. The comments and edits shall be retained to evidence that the work performed by the audit team has been directed, supervised and reviewed, as well as to retain a documented record of the audit reporting process.

224.26 The section 19 report shall be provided to FSASG. Briefings shall also be provided to FSASG and/or PSASG on significant issues as they arise in the course of a performance audit and as section 19 reports are prepared.

Guidance

224.27 The section 19 report provides the opportunity for the entity subject to audit, and any other person that in the Auditor-General’s opinion has a special interest in the report, to provide comments that must be taken into consideration in preparing the final report.

224.28 The PASG Workflow provides instructions on preparing and issuing the section 19 report.

224.29 To be comprehensive, an audit report needs to include all the information and arguments needed to address the audit objective(s) and audit questions, while being sufficiently detailed to provide an understanding of the subject matter and the audit findings and conclusions.

224.30 To be convincing, an audit report needs to be logically structured and present a clear relationship between the audit objective(s) and/or audit questions, audit criteria, audit findings, conclusions and recommendations. It also needs to present the audit findings persuasively, address all relevant arguments to the discussion, and be accurate. Accuracy requires that the audit evidence presented and all the audit findings and conclusions are correctly portrayed. Accuracy assures readers that what is reported is credible and reliable.

224.31 Being timely requires that an audit report be issued on time in order to make the information available for use by the legislature, management, government and other interested parties.

224.32 To be reader friendly, the auditor needs to use simple language in the audit report to the extent permitted by the subject matter. Other qualities of a reader-friendly audit report include the use of clear and unambiguous language, illustrations and conciseness to ensure that the audit report is no longer than needed, which improves clarity and helps to better convey the message.

224.33 Being balanced means that the section 19 report needs to be impartial in content and tone. All audit evidence needs to be presented in an unbiased manner. The auditor needs to be aware of the risk of exaggeration and overemphasis of deficient performance. The auditor needs to explain causes and the consequences of the problems in the audit report because it will allow the reader to better understand the significance of the problem. This will in turn encourage corrective action and lead to improvements by the audited entity.

224.34 Some key questions that should be considered when drafting the section 19 report include:

  • Are the sources underpinning the audit criteria / sub-criteria included in the report?
  • Do the audit findings and conclusions relate to the audit objective?
  • Do the recommendations flow from the findings?
  • Have the questions posed in the audit been answered?
  • Is the audit title appropriate?
  • Is the structure of the report logical and does it flow easily?
  • Does the report indicate the reliability of the audit evidence used to support audit findings and, in particular, whether evidence such as entity survey responses was corroborated by other audit evidence?
  • Does the overall conclusion comprise a succinct, balanced discussion of those matters that directly support the conclusion against the audit objective, with significant findings covered in the key findings section?
  • Do the recommendations focus on significant issues? Are they practical? Are they too long? Are there too many?
  • Is the report internally consistent? That is, is there consistency between the body of the report, the overall conclusion and the key audit findings?

224.35 Audit criteria and their sources must be identified in the audit report because the intended users’ confidence in the audit findings and conclusions depends largely on the audit criteria. In performance audits, a wide variety of sources can be used to identify audit criteria25.

ISO Standards

224.36 Any direct quotation of ISO standards without explicit permission from ISO would carry significant risk of breach of copyright, given ISO’s very restrictive approach. Given the potential legal implications, paraphrasing or referencing the standards is the appropriate approach.

Report summary

224.37 The general purpose, features and structure of the report summary of the section 19 report are outlined in the following paragraphs.

224.38 The report summary is an important part of the audit report. It should capture the essence of the audit, the key issues identified and the value added by the audit. The report summary is to be four to six pages in length, and aims to:

  • attract the report user’s attention and interest; and
  • provide a concise outline of the report and the key messages the report wishes to convey to the report user.
Background

224.39 The background is a short section that provides key introductory information about the audit.

224.40 The background section should cover:

  • context, with a brief overview of the program or activity being audited;
  • the service or program arrangements the audit addresses and their importance in terms of government policy, service delivery or governance;
  • a summary of key outcomes, outputs and key performance indicators (where available);
  • the dimensions of, or other insights into, the audit subject matter that are important to convey the significance of the audit topic;
  • the rationale for undertaking the audit;
  • the audit objective; and
  • the high-level audit criteria.

224.41 The drafting of the section 19 report provides an opportunity to revise the audit objective, criteria and scope, to ensure they reflect the audit coverage and audit findings.

Conclusion

224.42 The overall conclusion is likely to be of most interest to the majority of users of ANAO reports.

Supporting findings

224.43 The report summary includes ‘Supporting findings’, immediately following the conclusion. This section includes the various findings that are listed throughout the report in the grey boxes attached to each chapter. These various findings should be copied directly into this section.

Recommendations

224.44 It is important that audit recommendations:

  • address the cause of the issue or matter that requires improvement; for example, if the audit found that data quality was poor due to the absence of any quality management/assurance arrangements, the recommendation should be directed at the need to implement these arrangements, rather than simply recommending that data quality be improved;
  • focus on significant issues. It is not necessary that every issue addressed in a report results in a recommendation;
  • do not contribute to the ‘red tape’ burden and do not simply recommend an entity comply with an existing requirement;
  • should stand alone; that is, they are able to be understood without reference to supporting material elsewhere in the audit report; and
  • are realistic and achievable and have regard to the cost of implementation. In this context, it is important for the audit team to fully consider the cost and other implications (such as coordination and consultation requirements) of each recommendation and be satisfied that they are cost-beneficial in terms of improving performance and accountability.

224.45 Generally, recommendations should focus on what needs to be done, rather than how the entity should do it. When the means of implementing a recommendation is important, this should be addressed in the body of the report.

224.46 Even though an issue may not warrant a recommendation, a report should encourage the entity to address opportunities for improvement.

Key learnings

224.47 This section should not re-state the findings relating to the entity subject to the audit. It should include broad lessons for other government entities and inform improved business practices across the public sector which have come to light during this audit. It may relate to both areas for improvement and good practice identified in the audit. This information is also relevant to the quarterly audit insights publications and other avenues for disseminating observations from ANAO audit work.

Report length

224.48 The ANAO aims for reports of approximately 50 pages. The length may vary according to the substance of the audit and the audit conclusions reached.

Cross-entity reports

224.49 For cross-entity audits, a decision is usually made early in the audit (and advice given to the entities included in the audit) about whether the audit report will include findings, conclusions and recommendations for each entity, or will be written in generic terms.

224.50 Where the section 19 and final reports are to be written in generic terms, the audit team will draw out common audit findings, conclusions and recommendations and/or key findings from the report preparation or evidence papers that were developed for each entity included in the audit.

Procedural fairness

224.51 Procedural fairness refers to a set of rights that people derive under common law and have under the administrative law principles of natural justice.

224.52 Procedural fairness requires a decision-maker to hear a person before making a decision affecting the rights, interests and legitimate expectations of that person. In the case of performance audits, the decision is whether or not to include in the audit report certain information or opinions relating to a person or an entity.

224.53 The ANAO’s legal and procedural obligations to public sector entities and their employees will be satisfied by the normal consultation process followed during the conduct of a performance audit, including through the provision of reports under section 19 of the A-G Act. In the rare circumstances where the normal consultative process does not satisfy the ANAO’s procedural obligations, then it may be allowable for the relevant individual to be given the opportunity to comment on relevant extracts of the report.

224.54 Where third parties, such as contractors used by the entity subject to audit, could be identified in a report, they would normally be provided with an extract of the report that makes reference to them or allows them to be identified.

224.55 The decision on which party (or parties) should be provided with copies or extracts of a section 19 report rests with the Auditor-General. At the time the section 19 report is submitted to the Auditor-General for approval, details of the steps proposed to meet procedural fairness obligations should be outlined, including any proposed extracts strategy.

224.56 Specific legal advice may need to be obtained on how best to meet the ANAO’s procedural fairness obligations in particular circumstances, such as, where a number of individuals or bodies are able to be identified and it could reasonably be argued that the report could impact on their reputation.

Processing and issuing the section 19 report

224.57 The steps involved in processing the section 19 report for distribution are outlined in the PASG Workflow.

224.58 Where, following the section 19 meeting, the Auditor-General makes comment on, or requires changes to, the section 19 report, the Auditor-General will clarify whether a revised version of the report needs to be viewed and cleared or if the report can be sent to the entity on the basis that the agreed changes will be made by the Engagement Executive, or GED as necessary.

Persons or bodies provided with the section 19 report or extract

224.59 The section 19 report, or extract from the report, may be given to any person who, or entity that, the Auditor-General considers has a special interest in the report or the content of the extract.

224.60 Whether a person has a special interest and should therefore receive a full copy or extract of the section 19 report will need to be determined on a case-by- case basis. For example, where an audit of procurement has been undertaken, a full copy of the section 19 report might be provided to the Department of Finance because it has policy responsibility for procurement matters.

224.61 A person or organisation with a special interest in the report or content of an extract may also have cause to make a claim under subsection 37(1) of the A-G Act, for example under paragraph 37(2)(e) which relates to the inclusion in a public report of particular information that would unfairly prejudice the commercial interests of a person or organisation. Policies and guidance with respect to the operation of section 37 of the A-G Act are provided in Chapter 226.

224.62 The process for preparing extracts to be provided to a third party is a deliberate process of reviewing the report in its entirety and considering each part as to what to include/exclude in the extract. Audit teams should consult with GEDs and/or EDs with experience in this process.

224.63 Generally, the following arrangements for provision of the section 19 report should apply. The section 19 report of an audit of a:

  • Commonwealth entity performed under paragraph 17(1)(a) of the A-G Act should be provided to a single official who is, or, in the case of a corporate entity is a member of, the accountable authority of the entity;
  • Commonwealth company performed under paragraph 17(1)(b) should be provided to a director (usually the Chair) of the company; and
  • corporate Commonwealth entity or a Commonwealth company, or a Commonwealth partner respectively, performed under paragraph 17(1)(c) and under section 18B should be provided to a person who is, or is a member of, the governing body of the subsidiary or the Commonwealth partner.

224.64 For Commonwealth entities, section 19 reports should only be sent to the accountable authority,26 or in the case of Commonwealth companies, subsidiaries and Commonwealth partners (which do not have an accountable authority of their own), a director or member of the governing body of the entity. This will reduce the risk of accidental disclosure and give the accountable authority, director or member of the governing body control over who receives the section 19 report. Other officers (including internal audit and ANAO liaison) may be sent an email informing them that the section 19 report has been provided. Note that you may need to consult with the Auditor-General about provision of relevant papers to entities with unusual governance arrangements; for example, due to the diarchy arrangement the Auditor-General has consented to the provision of relevant papers to both the Secretary of the Department of Defence and the Chief of the Defence Force.

Comments sought on the section 19 report or extract

224.65 It is important to note that an extract from the section 19 report attracts the same protection regarding confidentiality and the associated penalties as the full section 19 report (section 36(3) of the A-G Act).

224.66 Senior members of the audit team should take an active role in communicating with the entity throughout the period that the entity has to provide comments on the section 19 report to assist in ensuring a timely response.

224.67 Discussing the audit with the entity in both formal and informal ways throughout the 28-day period can assist in identifying and resolving any issues that have the potential to delay the finalisation of the audit report. Every effort should be made to reach agreement, especially in relation to recommendations, without compromising the Auditor-General’s independence. Such efforts often include the involvement of senior management from the entity and the ANAO in further discussions during the 28-day response period. The responsible GED and ANAO Executive should be kept abreast of key developments.

224.68 Entities’ final comments on findings and recommendations should include an ‘agree’ or ‘disagree’ position on each recommendation.

Consideration of comments on the section 19 report or extract

224.69 Where substantive comments are received, it would generally be appropriate to have further discussions with senior management of the entity and to give consideration to making amendments to the final report. As a general rule, the quality of the final report will be enhanced if disagreements can be resolved and the final report amended as necessary. The responsible GED and ANAO Executive should be consulted on such proposed amendments.

224.70 Incorporating entity comments in the final report is discussed at paragraphs 226.22 to 226.25.

Retaining evidence of review of section 19 report

224.71 The iterative review of the section 19 report can be extensive for some audits. Audit teams are to make judgements regarding the number of reviewed versions of the section 19 report that are required to be retained on the audit working papers. For example, drafts that contain minor editorial changes and comments only may not need to be retained.

Liaison with FSASG

224.72 In determining what is a significant issue requiring communication with FSASG, PASG shall consider:

  • the potential to give rise to a material mistake in the financial statements;
  • any actual or suspected fraud;
  • any issues revealing a weakness in key financial controls; and
  • any other issues of which the audit team is aware that may be of interest to FSASG.

225. Key learnings

Policy

225.1 At the conclusion of an audit, the audit team shall document any key learnings and/or instances of good practice that were identified during the audit.

225.2 Key learnings, and instances of good practice, from other government entities shall, unless the contrary is approved by the relevant GED, be incorporated into the summary section of the section 19 and final reports.

Guidance

225.3 The audit team will identify and document the key learnings from performance audits and other engagement reviews. These include lessons from other government entities that may inform improved business practices across the public sector.

225.4 PASG practice management has circulated key headings to assist in grouping findings under relevant audit activity types. These key learnings will be used to inform the ANAO Insights products.

225.5 Also refer above to paragraph 224.24 and 224.47.

226. Final report

Policy

226.1 The final audit report shall convey a sufficient understanding of the entity and issues related to the audit objective, and be based on sound evidence and well-targeted analysis.

226.2 The audit team shall consider all material events relevant to the audit up to the date of issue of the final report. Following fieldwork, the effect of any subsequent events on the audit findings shall be considered up to the date of the final report.

226.3 Audit reports shall not include particular information that is sensitive if its disclosure would be contrary to the public interest in the opinion of the Auditor-General (section 37 of the A-G Act).

226.4 If, during the course of an audit, an entity indicates, either verbally or in writing, that they believe that including specific content in an audit report would not be in the public interest pursuant to section 37 of the A-G Act, the Engagement Executive shall inform the Auditor-General immediately. The Auditor-General may seek further advice as to whether the relevant content may be information of the type listed in section 37(2) of the A-G Act. The purpose of such advice would be to assist the Auditor-General to form an opinion under section 37(1)(a) of the A-G Act.

226.5 If the Auditor-General decides that information that is material to the conclusion is to be excluded from a section 19 or final report because it is considered by the Auditor-General to be particular information for the purpose of section 37 of the A-G Act, the Engagement Executive shall document, in writing through the GED, the content that is proposed to be excluded and the Auditor-General’s reasons for its exclusion.

226.6 If the entity’s response includes comments on a third party, the third party shall be provided the opportunity to confirm the facts and to reply to any adverse comments, unless otherwise approved by the Group Executive Director.

226.7 Any changes of substance made to the section 19 report in the final report shall be discussed with the entity.

226.8 As per 224.23, to meet the requirements of the A-G Act, any comments received on the section 19 report shall be given full consideration, in preparing the final audit report. This consideration and the response to the comments shall be documented on the audit file.

226.9 Where the ANAO does not agree with entity comments (and consequently has not changed the report), the reasons for not agreeing with the comments shall be documented and reported to the ANAO Executive as evidence that the entity’s views were fully considered.

226.10 Formal comments on the section 19 report, or extract of the report, shall be included in full as an appendix to the final report in accordance with section 19(8) of the A-G Act.

226.11 Where the Auditor-General or Deputy Auditor-General requests amendments to the draft final report, the amended report shall be provided to the Auditor-General, Deputy Auditor-General and the responsible Group Executive Director for final comments and clearance.

226.12 All final audit reports shall be formally signed-off by the Engagement Executive and approved for tabling in Parliament by the Auditor-General.

226.13 Any rejoinders made by the ANAO in the final report—to the entity’s response—shall be approved by the Auditor-General

226.14 Evidence of review of the final report shall be retained as part of the audit working papers.

226.15 The final audit report shall state that the audit was carried out in compliance with the ANAO Auditing Standards, and shall also include the full expected cost of the audit to the ANAO.

226.16 The audit working papers shall, if applicable:

  • include formal comments from the entity provided with the section 19 report or extracts of the report;
  • clearly outline changes made to the report as a result of entity comments; and
  • include a record of the Auditor-General’s, Deputy Auditor-General’s and Group Executive Director’s review and/or request to amend the report.

Guidance

226.17 The final audit report, setting out the ANAO’s audit findings, overall conclusion and recommendations, is tabled in the Parliament and is available to the public.

226.18 The PASG Workflow provides instructions on preparing the final report.

Structure and content

226.19 The final audit report should be objective, balanced and constructive in the presentation of the overall conclusion and the audit findings. Recommendations should be practical and cost-effective.

226.20 The structure and content of the final audit report should be substantially the same as the section 19 report.

Section 37 of the A-G Act

226.21 In providing any advice sought by the Auditor-General in respect of the disclosure of particular information under section 37 of the A-G Act, audit teams should take into consideration that, in administering the A-G Act, specifically in respect of section 37, the Auditor‐General’s approach is in favour of disclosure to the Parliament, in the public interest, unless the Auditor‐General is of the opinion that the public interest is served by not disclosing ‘particular information’ which is otherwise prohibited from public release (for example, information with a national security classification). There is always a balance to be struck between the public interest in not disclosing information and the broader public interest in reporting transparently to the Parliament.

Incorporating entity comments in the final report

226.22 The final audit report includes the entity’s response to the section 19 report in the report summary as the final section titled Summary of entity response. It should be no more than half a page in length. If the entity’s response is longer, the audit team may discuss with the entity their willingness to prepare a shorter summary, or the full text should be included as an appendix.

226.23 Providing the entity with the opportunity for the formal comments to be amended, when a change has been made to the final report in response to an entity comment, avoids the situation where the entity’s published response relates to aspects of the report including the audit findings and conclusions that have been amended as a result of consideration of comments received on the section 19 report.

226.24 A rejoinder is the ANAO’s response to entity comments. Where the ANAO includes a rejoinder to the formal response of the entity in the final report, it should be described as an ANAO comment and placed below the entity’s comments. The ANAO comment will be a statement of fact addressing the specific entity comment and drawing attention, where appropriate, to the amended section of the report.

Approval and processing of the final report

226.25 See the PASG Workflow for detailed instructions on approving and publishing the final report.

227. Publication and closure

Policy

227.1 Prior to the tabling of the final report, an embargoed copy shall be made available to the following stakeholders two days prior to tabling:

  • the Prime Minister and responsible Minister (for all audits);
  • the Finance Minister, the Prime Minister’s Office and the Department of the Prime Minister and Cabinet (for cross-entity audits conducted under section 18);
  • an official who is, or is a member of, the accountable authority of the entity (if the audit is of a Commonwealth entity);
  • a director of the company (if the auditee is of a Commonwealth company);
  • a person who is, or is a member of, the governing body of the subsidiary (if the auditee is of a subsidiary of a corporate Commonwealth entity or Commonwealth company);
  • to the extent that the report relates to the operations of a Commonwealth partner—a person who is, or is a member of, the governing body of the Commonwealth partner; and
  • to the extent that the report relates to the operations of a person who is not a Commonwealth partner—that person.

227.2 The audit report shall be tabled as soon as practicable after the audit is completed (as required by subsections 17(4) and 18(2) of the A-G Act).

227.3 The final audit report shall be transmitted to the Presiding Officers to enable tabling in the Parliament as soon as practicable after the completion of each audit.

227.4 For the purposes of the Final Report, the relevant stakeholders include:

  • the Prime Minister;
  • the responsible Minister;
  • the relevant Parliamentary Secretary, where they have direct responsibility for the matters subject to audit;
  • the Secretary of the Department of the Prime Minister and Cabinet; and
  • the responsible accountable authority of the entity subject to audit.

Guidance

227.5 The PASG Workflow provides procedures on the publication and closure processes. This includes several procedures to forward the Final Report to publications, close the audit e-hive files and complete a Continuous Improvement Report and a JCPAA Briefing. While the ANAO Communication Unit plays an important role in producing the final report, organising its presentation to Parliament, and publishing after tabling on the ANAO website, the audit team remains responsible for ensuring that the words and layout are correct and that the report is complete and accurate.

Final draft to stakeholders (‘A4s’)

227.6 Third parties that received an extract of the section 19 report do not, as a matter of course, receive a copy of the final draft of the report. If a third party requests an extract of the final report, the decision should be referred to the Group Executive Director for consideration.

Tabling of the audit report

227.7 If the final report is to be presented when the Parliament is not sitting, it will be presented under Senate standing order 166 which provides for the presentation of documents when the Senate is not sitting. Upon receipt by the President of the Senate, the document is deemed to have been presented to the Senate, is authorised for publication, parliamentary privilege is attached and the embargo is lifted. The audit report will then be tabled at the next sittings of the House of Representatives and the Senate.

227.8 In addition to those persons and/or bodies who must receive a copy of the final report, a copy of the final report, or an extract, may also be given to a person or body that has a special interest in the report or the content of the extract.

Parliamentary review

227.9 All performance audits are required to be examined by the Joint Committee of Public Accounts and Audit in accordance with sub-section 8(1)(c) of the Public Accounts and Audit Committee Act 1951. The committee conducts public inquiries into a selection of audit reports each year, following consultation with the ANAO. The ANAO may provide a private briefing to the committee and gives evidence at these inquiries.

227.10 The PASG Workflow states that the audit team prepares a JCPAA Briefing and saves a PDF version to the JCPAA Summary folder on the ANAO shared library.

227.11 From time to time, other parliamentary committees also conduct inquiries into performance audit reports or aspects of public administration that have been the subject of performance audit coverage.

Attachments

Attachment 1: Generic audit criteria for Grants administration

Audit Objective

To assess the effectiveness of the [entity’s] design, implementation and administration of the [grant program].

Criterion 1

Access to the Program

Criterion 2

Assessment of Eligible Applications

Criterion 3

Advice to the Minister and Funding Decisions

Criterion 4

Funding Distribution, Signing of Funding Agreements and Feedback to Unsuccessful Applicants

Criterion 5

Managing Funding Agreements and Achieving Outcomes

   

Criteria 1 – Access to the Program

  1. Were application and assessment methods chosen that promoted open, transparent and equitable access to grants?
  2. Were the grant program guidelines and any related published material clear, consistent and approved in accordance with the requirements of the grants administration framework?
  3. Did the published program material (such as the program guidelines) set out the way potential applicants are able to access the grant program, and any eligibility or threshold criteria that applicants must satisfy in order to be considered for funding?
  4. Did the approach taken reflect the CGRGs preference for competitive, merit-based selection processes, based upon clearly defined selection criteria?
  5. Did the eligibility criteria reflect the policy intent of the grant program and were they straightforward, easily understood and effectively communicated to potential applicants?
  6. Were all eligibility/threshold criteria grouped and clearly identified as threshold requirements in a discrete section of the program guidelines so as to provide a comprehensive checklist of minimum and mandatory requirements both for the benefit of applicants, and to focus the department’s eligibility checking?
  7. Did the threshold/eligibility requirements clearly specify the date by which applications will be required to be submitted and clearly state that applications not received within the required timeframe or through the required application process would not be accepted?
  8. Did the guidelines also define what would constitute a complete application and stipulate whether incomplete applications will be considered?
  9. Was a sound communications strategy with potential funding recipients and other stakeholders in place to attract high quality applications?
  10. Was the information obtained from applicants and other sources sufficient to properly assess applications and adequately inform funding decisions, without being unduly onerous on applicants?
  11. Were potential applicants sufficiently engaged in the design of the program, and provided with sufficient time to develop and lodge a compliant application?
  12. Did the agency systems (IT and other) in place effectively support the application lodgement process?
  13. Were applications recorded as they were received, in a robust and accountable manner?
  14. How many applications were received, and what are the key features of the population of applications?
  15. Was a comprehensive and robust eligibility/compliance checking process implemented?
  16. Were all ineligible/non-compliant applications excluded from further consideration for funding?
  17. What were the main reasons for applicants/applications being assessed as ineligible?
  18. What was the proportion of applications and applicants assessed as ineligible – was it in line with expectations during the design of the program? If the proportion is high either in terms of expectations or compared to other grant programs, does this reflect a shortcoming in the design of the program in that ineligible applicants and applications were not effectively discouraged?

Key issue to form conclusion on:

Was the grant application process accessible and likely to maximise the attraction and selection of high quality applications?

Criteria 2 – Assessment of Eligible Applications

  1. Do the program’s arrangements clearly delineate roles and responsibilities for the department and the Minister in assessing, recommending and approving funding applications?
  2. Did the published program guidelines (or other published material) clearly outline the assessment criteria that would be applied?
  3. Was there a clear link between the program objectives and the assessment criteria?
  4. Were the assessment criteria weighted and, if so, were the weightings published?
  5. Did the assessment process (including any short-listing of eligible applications) implemented accord with the processes set out in the promulgated guidelines?
  6. What assessment rating approach was adopted - did it involve ordinal or numerical scales? Did the rating approach enable the consistent combining of ratings against each criterion into an overall rating of individual applications, and allow conclusions to be formed in respect to the relative merits of applications?
  7. Was the assessment of applications against each individual assessment criterion robust and reliable?
  8. Did the assessment address all published merit criteria that were applicable to each eligible application, with no unpublished criteria applied? Were the results of this work, together with any assessment of risks, cost effectiveness/contestability/economic use of resources, used to identify whether each eligible application represented value for money (having regard to the duty to promote the proper use and management of public resources, as provided by section 15 of the PGPA Act)?
  9. Where an Advisory Committee/Panel is involved in the assessment of applications:
    1. Was the appropriate expertise within, or available to the Committee/Panel to assess the applications?
    2. Was the role of this Committee/Panel (vis-a-vis the departmental officials) clearly set out? In particular, was it clear whether funding recommendations would be made to the Minister by the Committee/Panel, the department or both?
    3. Was the Committee/Panel provided with sufficient information to inform its assessment work? Did this include the applications themselves and any supporting documentation, or only the department’s analysis of the applications?
    4. Were procedures established and adhered to that effectively managed any conflicts of interest?
    5. Were Committee/Panel meetings minuted in a comprehensive and timely way to record not only assessment decisions taken, but also the reasons for those decisions?
    6. Was the way in which the Committee/Panel assessed applications consistent with the program guidelines and any internal documentation of the assessment framework, and in accordance with the key principles for grants administration established by the CGRGs?
    7. Did the department provide adequate support to the Committee/Panel in the assessment of applications?
    8. Was the nature of the Committee’s/Panel’s conclusions consistent with the CGRGs requirement for a clear funding recommendation to be made to the Ministerial decision-maker?

Key issue to form conclusions on:

Did the grant assessment process involve an approach by which those projects most likely to contribute to the cost-effective achievement of the program objectives were consistently and transparently selected as being recommended to the decision-maker for funding? For competitive grant programs, this involves all eligible, compliant applications being assessed in the same manner against the same criteria, with the outcome of the assessments against the published criteria then being used to rank each application in priority order.

Criteria 3 – Advice to the Minister and Funding Decisions

  1. Before approving any grants, did the Minister receive agency advice on the merits of each proposed grant relative to the published program guidelines?
  2. Was there any evidence that the agency advice was changed or otherwise amended at the request of the Minister and/or his/her Office? Was only one advice provided, or was there a series of advices provided until such time that the Minister was happy to accept the advice he/she received?
  3. Did the advice outline the application and selection processes that had been employed, including the assessment criteria used in arriving at the funding recommendations? Was the approach outlined consistent with the program guidelines?
  4. Did this advice include a clear funding recommendation? In particular, was the relative ranking/priority of each eligible application clearly identified in the advice, with the advice clearly identifying those grants that were recommended for approval, and those that were not recommended for approval?
  5. Where the agency advice identified two or more funding options open to the Minister, did the agency advice clearly identify which option was recommended by the agency, and why?
  6. If the recommended applications included any that had not been assessed/scored as meeting the published assessment criteria to a high standard, was the basis on which those applications could be considered to represent an efficient, effective and economical use of public money clearly articulated?
  7. Did the advice outline to the Minister the basis for determining the recommended distribution of funding between funding streams?
  8. Did the advice clearly identify to the Minister that he/she was considering the approval of one or more grants, and therefore identify any related reporting requirements in circumstances where a grant was awarded in the Minister’s own electorate, or where a grant was awarded to an application that had not been recommended for funding?
  9. Did the advice clearly outline other requirements of the financial framework that apply to grants, including the requirement that the decision-maker record the basis for each grant approval, in addition to recording the terms of each approved grant?
  10. Did the Minister approve any grants in his or her own electorate? If so, were they reported to the Finance Minister as soon as practical after the decision was taken?
  11. Did the Minister approve any grants which the agency had not recommended be approved?
    1. If so, has the annual report to the Finance Minister for that calendar year fallen due and, if so, were these grants reported to the Finance Minister? If the report is not yet due, are effective arrangements in place to ensure that any such grants will be reported to the Finance Minister?
    2. If so, did the Minister record substantive reasons for the approval, having regard to the program’s guidelines?

Key issue to form conclusion on:

Was departmental advice to the Minister complete in order to enable the Minister to perform the responsibilities as a grant approver, and did the Minister act in accord with this advice?

Criteria 4 – Funding Distribution, Signing of Funding Agreements and Feedback to Unsuccessful Applicants

  1. How does the analysis of funding awarded (both in terms of numbers of applications and amount of funding) compare to the population of applications (in total, and only eligible applications)? Is there any evidence of bias in the distribution of funding (including in party political terms), and is this consistent with the program objectives/criteria/focus?
  2. Were robust funding/project agreements developed and signed in a timely manner so as to enable successful applications to proceed? Do the terms of the funding/project agreements (including the payment arrangements, paying particular attention to any large up-front payments) reflect value for money?
  3. Do the funding/project agreements clearly outline what each project is expected to achieve in contributing to the program objectives? Do the funding/project agreements include performance indicators and reporting/acquittal arrangements to enable actual outcomes to be captured?
  4. Were the arrangements relating to payments to the states and territories undertaken in accordance with the federal financial relations framework?
  5. Was timely and accurate feedback provided to unsuccessful applicants as to the reasons they were unsuccessful? Was an appropriate complaints mechanism in place?
  6. Were any lessons learned clearly identified (including through consultation with applicants and other stakeholders)?
  7. Has a strategy and plan been developed to evaluate whether the program achieves its objectives? Have program key performance indicators been established?

Key issue to form conclusions on:

Is the distribution of funding consistent with the program objectives, and are appropriate arrangements in place to manage the successful projects? What outcomes are expected to be achieved by the funded applications?

Criteria 5 – Managing funding agreements and achieving outcomes

  1. Are there arrangements in place to ensure consistent, high-quality, and appropriate frequency of monitoring of funding agreements? Are arrangements consistent with what is set out in funding agreements?
  2. Are risks and delivery issues escalated as appropriate and in a timely manner? Is corrective action taken, and recorded, where necessary?
  3. Is the achievement of contract milestones verified before associated payment is made? Are payments correct, appropriately authorised and made to the correct recipients?
  4. Are there arrangements in place to recover grants funds if a recipient has not complied with grant conditions?
  5. Are contract variations documented and approved by the appropriate delegate?
  6. Is the performance data needed to support monitoring of individual grants, and the program as a whole, been identified, collected and analysed?
  7. Is program performance against outcomes reported to program stakeholders? Is the Minister and departmental executive kept informed of program performance with accurate and up-to-date information?
  8. Is a Program Evaluation Strategy in place to examine measure and report the achievement of program outcomes?

Key issue to form conclusions on:

Are projects being actively monitored in terms of their milestone achievement and contribution to program objectives?

Attachment 2: Generic audit criteria for Procurement

Generic audit criteria diagram

Attachment 3: Generic audit criteria for Regulation

Generic audit criteria diagram

Attachment 4: Generic audit criteria for Contract Management

Generic audit criteria diagram

Footnotes

1 The ANAO for the purposes of the PGPA Act is a listed entity. An official of a Commonwealth entity that is a listed entity is a person who is prescribed by an Act or the rules to be an official of the entity. Under s38 of the A-G Act, the persons listed as officials of the Audit Office include persons engaged under contract as referred to in section 27.

2 ASAE 3000 paragraph 34.

3 ASAE 3000 paragraph 32(a).

4 Depending on the timing of the performance audit, this may include either a previous year or current year performance statement audit. In addition, the PSASG GED may request an AWP where a performance statement audit is planned for the following year in the same entity.

5 See paragraph 222.19 for circumstances in which it may be appropriate to consult with PSASG.

6 Refer to GUID 3910 Central Concepts for Performance Auditing paragraph 57 and GUID 3920 The Performance Auditing Process paragraphs 39-41 for additional guidance on the sources of audit criteria.

7 Refer to ANAO Audit Manual – Shared Content 10.32 for more information about the ANAO’s responsibilities for documenting and communicating about potential fraud or other wrongdoing.

8 Depending on the timing of the performance audit, this may include either a previous year or current year performance statement audit. See paragraph 222.19 for circumstances in which it may be appropriate to consult with PSASG.

9 For formal interviews conducted pursuant to the Auditor-General’s information-gathering powers, refer to the ANAO Audit Manual – Shared Content, paragraphs 2.5 to 2.21.

10 INTOSAI GUIDEGUID 3920 The Performance Auditing Process paragraph 75.

11 INTOSAI GUIDEGUID 3920 The Performance Auditing Process paragraph 76.

12 The methodology is described at ANAO Audit Manual – FSASG Specific Chapter 115, in sections 5400 and 6100 of the Online Audit Guide and in the financial statement controls testing template.

13 Integrated audits are financial statement audit engagements that also include a separate auditor’s opinion on the design, implementation and operating effectiveness of controls.

14 A convenience sampling approach includes any approach where the choice of samples is influenced by the convenience of the auditor or the auditee. For example, only choosing samples from particular locations so that the supporting evidence is easier to locate or only choosing samples likely to have particular characteristics.

15 A haphazard sampling approach involves auditors attempting to approximate random sample selection without using a random number generator, similar to drawing lots from a hat. Because of the population sizes involved in audit sampling. it is highly unlikely that auditors would be able to ensure that each item in the population has an equal chance of being selected in applying this method.

16 Cabinet documents are defined by the Cabinet Handbook and the Freedom of Information Act 1982. The Cabinet Handbook 14th edition defines Cabinet documents at paragraph 132. Paragraph 133 notes that Cabinet documents are to be marked ‘PROTECTED Cabinet’.

17 This may be achieved through the documented briefing papers or minutes of the section 19 workshop.

18 Bulk email collection involves the gathering of email records on a non-specific basis where, for any reason, ANAO is not able to identify and obtain from the auditee the specific email correspondence required to provide the evidence sought.

19 In accordance with Chapter 2 of the Audit Manual – Shared Content, while the Australian Privacy Principles under the Privacy Act 1988 do not apply to the ANAO, it is ANAO’s position that the spirit and intent of Act will be followed to the extent it is appropriate.

21 A-G Act section 37.

22ISSAI 3000 paragraph 122 (incorporated into the ANAO Auditing Standards) requires the auditor to identify the audit criteria and their sources in the audit report. Refer to paragraph 207.34 for sources of audit criteria.

23 Depending on the timing of the performance audit, this may include either a previous year or current year performance statement audit.

24 Under the Acts Interpretation Act 1901 (Cth), the 28-day period is based upon calendar (and not working) days but does not include the day the recipient receives the section 19 report, meaning that where a recipient receives the section 19 report on a Monday, the last day of 28 day period will be a Tuesday. Where the last day of the 28-day period is a Saturday, Sunday or public holiday, the period is extended so that the last day is the first day following the end of the period that is not a weekend or public holiday.

25ISSAI 3000 paragraph 123.

26 For corporate Commonwealth entities where the accountable authority is normally a board of directors or similar, it is normally sent to the chairperson or equivalent of that board.