Audit snapshot

Why did we do this audit?

  • Primary Health Networks (PHNs) are a delivery model for primary health care with the main objectives of: improving the effectiveness and efficiency of health services, particularly for those at risk of poor health outcomes; improving the coordination of health services; and increasing access and quality support for people.
  • The Department of Health and Aged Care (Health) is responsible for administering PHN grant agreements, including monitoring compliance and the performance of PHNs, and supporting PHNs to improve.

Key facts

  • PHNs were established on 1 July 2015, replacing Medicare Locals.
  • There are 29 PHN providers which are responsible for 31 PHN regions across Australia.
  • Populations of PHN regions range from 64,000 to 1.7 million people.

What did we find?

  • Health has been partly effective in its performance management of PHNs.
  • Health has established largely fit-for-purpose compliance and assurance arrangements for PHNs and the PHN delivery model.
  • Health’s performance measurement and reporting arrangements for PHNs and the PHN delivery model are partly fit-for-purpose.
  • Health has not demonstrated that the PHN delivery model is achieving its objectives.

What did we recommend?

  • There were eight recommendations relating to: ensuring PHN compliance with grant agreement requirements; improved PHN performance measures; PHN data assurance; improved PHN performance reporting; IT systems for PHN monitoring and reporting; and evaluation of the PHN delivery model.
  • Health agreed to seven recommendations and agreed in principle to one recommendation.

$11.6bn

grant commitments to fund PHNs from 2015–16 to 2022–23.

132

different PHN performance measures in grant agreements and a performance framework.

3

annual reports on PHN delivery model performance produced since 2015–16.

Summary and recommendations

Background

1. Primary Health Networks (PHNs) were established by the Department of Health and Aged Care (Health) on 1 July 2015 as a delivery model for primary health care. PHNs have two key objectives: improving the efficiency and effectiveness of health services for people, particularly those at risk of poor health outcomes; and improving the coordination of health services and increasing access and quality support for people. 1

2. PHNs are non-government organisations funded through Australian Government grants. PHNs work across seven priority health areas comprising mental health, Aboriginal and Torres Strait Islander health, population health, the health workforce, digital health, aged care, and alcohol and other drug services.

3. Health awards and manages grant agreements with the PHNs and is responsible for: the provision of guidance materials; facilitating regular communications and information sharing with PHNs; provision of data to support PHN needs assessments; monitoring the compliance and performance of PHNs; and supporting PHNs to improve.

Rationale for undertaking the audit

4. Since the establishment of PHNs in 2015–16, Health has committed $11.6 billion to PHNs in grants funding. The commissioned services and activities of PHNs contributed to seven programs across three outcomes in Health’s 2022–23 Corporate Plan. The audit was identified by the Joint Committee of Public Accounts and Audit as an audit priority of the Parliament in 2021–22 and 2022–23.

5. Auditor General Report No. 9 of 2019–20 National Ice Action Strategy Rollout examined the expansion of alcohol and other drug treatment services through PHNs. Health agreed with the ANAO’s recommendation to ‘finalise the PHN Quality and Assurance Framework, with appropriate actions to assess whether PHNs are operating appropriately across the commissioning cycle’.

6. The audit aims to provide assurance to the Parliament that Health is appropriately monitoring compliance and performance of individual PHNs, as well as the overall performance of the PHN delivery model.

Audit objective and criteria

7. The purpose of the audit was to assess the effectiveness of the Department of Health and Aged Care’s performance management of Primary Health Networks.

8. To form a conclusion against the objective, the ANAO examined:

  • Has Health established fit-for-purpose compliance and assurance arrangements for PHNs and the PHN delivery model?
  • Has Health established fit-for-purpose program performance measurement and reporting arrangements for PHNs and the PHN delivery model?
  • Has Health effectively evaluated the PHN delivery model to demonstrate that it is meeting its objectives?

Conclusion

9. The Department of Health and Aged Care has been partly effective in its performance management of Primary Health Networks.

10. Health has established largely fit-for-purpose compliance and assurance arrangements for PHNs and the PHN delivery model, although these arrangements were first established in 2021, almost six years after the implementation of the PHN delivery model. As at December 2023, there is a compliance and assurance framework comprised of a strategy, risk management plan, requirements in grant agreements, schedule of assurance activities, clear roles and responsibilities and oversight arrangements. Risks are identified and assurance activities are planned over individual PHNs and the delivery model as a whole. However, the link between identified risks and assurance planning is unclear, and some planned activities are not undertaken. Assurance is also weakened by record keeping practices and a lack of public transparency.

11. Health’s performance measurement and reporting arrangements for PHNs are partly fit-for-purpose. There is a PHN program logic to inform the development of outcomes-oriented performance measures, however it lacks clarity. There are a number of performance measures, however many assess PHNs’ compliance with grant agreements rather than providing information about the achievement of outputs and outcomes. The performance measures are largely aligned to Australian Government guidance for performance measurement, except that there are weaknesses in relation to relevance and data assurance. Public performance reporting is not timely or informative about overall PHN delivery model performance, and does not include information about individual PHN performance. IT systems for PHN performance reporting are partly fit-for-purpose.

12. Health has not demonstrated that the PHN delivery model is achieving its objectives. Health had no evaluation plans for the PHN delivery model after 2018. Health has not conducted a comprehensive delivery model evaluation. A 2018 early implementation evaluation was inconclusive about the achievement of objectives at that early stage in the delivery model’s implementation. A lack of baseline and relevant performance data impedes understanding of whether the delivery model has met its objectives.

Supporting findings

Assurance arrangements

13. The PHN delivery model was first implemented in 2015 and lacked an assurance framework until 2021, although specific guidance and tools to support assurance activity have been in place since 2016. An assurance strategy and schedule were established in 2022 and a risk management plan in 2023. The assurance framework establishes responsibilities for assurance activities including audits of individual PHNs. There are several oversight bodies, although an operational working group was disbanded in 2022. Grant agreements with PHNs appropriately establish governance, acquittal and performance reporting requirements. Health has not complied with Commonwealth Grant Rules and Guidelines requirements to publish on GrantConnect approximately $10 billion in grants awarded to PHNs. (See paragraphs 2.1 to 2.15)

14. Health established an assurance plan for individual PHNs in 2022, which includes activities under three ‘lines of assurance’. Health’s assurance schedule does not include activities to monitor compliance with grant agreement requirements relating to the publication of needs assessments and activity work plans. Health took no action in response to PHN non-compliance with grant agreement requirements to provide financial information for a 2020 review of operational expenditure. Line one assurance activities (environmental scans, PHN risk assessments and assessments of PHN milestone reporting under grant agreements) were not always completed as planned or are difficult to assess due to poor record-keeping. Line two assurance activities (including assessment of PHN performance reporting and reviews by oversight bodies) were largely undertaken as planned. Line three assurance activities (risk-based audits of PHNs) were undertaken as planned. Health tracks PHNs’ implementation of recommendations from audits. The PHN Assurance Framework states that it takes a risk-based approach to conducting assurance, however risk assessments of PHNs are not undertaken as planned. (See paragraphs 2.16 to 2.26)

15. Health assesses risks and ‘hot issues’ for the PHN delivery model as a whole. Governance bodies have oversight of PHN risks, however consideration of risk by governance bodies has not been consistent with Health’s assurance planning and the terms of reference for these bodies. Health plans assurance activities over the delivery model as a whole, which are set out in an Assurance Framework or Schedule, however it is unclear how risk assessments inform assurance plans. The planned activities to provide assurance over the PHN delivery model have been partly carried out. In 2022–23 the Program Board did not meet as frequently as required. Six management reviews have been undertaken since 2018, including one in 2022–23. Health tracks its implementation of recommendations from these management reviews and assesses 76 per cent of agreed recommendations to have been implemented as at July 2023, with some 2018 recommendations related to performance monitoring still outstanding. Planned reviews of the PHN Assurance Framework and the Program Performance and Quality Framework in 2022–23 were not completed. (See paragraphs 2.27 to 2.35)

Performance reporting

16. Health established a program logic for the PHN delivery model in 2015, which was revised in 2018 and 2023. The 2018 program logic contains most of the expected elements of a program logic and provides information about how the PHN delivery model will achieve outcomes. The 2023 program logic introduces new objectives for the PHN delivery model, lacks clarity and is not aligned with guidance on developing program logics. (See paragraphs 3.2 to 3.9)

17. With the exception of measures related to the mental health priority area, the majority of performance measures included in PHN grant agreements check compliance with grant agreement reporting and other requirements, and do not assess the quality of outputs or the achievement of outcomes. Performance measures contained in mental health grant agreements are useful and support the adoption of an ‘outcomes orientation’, as recommended by the Commonwealth Grants Rules and Guidelines. In addition to measures in grant agreements, there is a performance framework which sets out 39 additional PHN ‘performance’ measures and 15 ‘organisational’ measures. These measures are used to monitor individual PHNs’ performance and, when aggregated, to conclude on the performance of the delivery model as a whole. Of the 39 performance measures, 20 measures are assessing something other than compliance with grant agreement requirements. These measures provide a basis for assessment over time; represent a mix of qualitative, quantitative, effectiveness and output measures; and mainly have targets. However, the relevance to stated PHN delivery model objectives is sometimes unclear, and there is a reliance on unverified PHN-supplied data for some measures. Health planned to improve PHN performance measures by July 2023, however this was delayed to June 2024. (See paragraphs 3.11 to 3.30)

18. Health’s PHN performance reporting is not timely, complete or consistently useful. A publicly available PHN annual report has been produced for three of the five years that the PHN delivery model has been operational and an annual report has been required. As at December 2023 there is no performance information publicly available about 2021–22 or 2022–23. Annual reports have been published years after the time period to which they apply. Health does not report against all the measures it has established, and published reports do not include analysis of financial information, the efficiency of PHNs, comparative analysis across PHNs or information on individual PHN performance. Some measures rely on secondary data that is unavailable. Annual reports partly rely on PHN self-reported performance information that is not always provided. (See paragraphs 3.31 to 3.54)

19. Health’s IT systems for managing PHN performance reporting are not fully fit-for-purpose. A system was developed in 2019 called the Primary Health Networks Program Electronic Reporting System (PPERS). Health has a range of guidance to support PPERS users. PPERS has limited capability to validate data inputs, analyse data and generate compliance and performance reporting. (See paragraphs 3.55 to 3.61)

Evaluation

20. A fit-for-purpose evaluation plan for the PHN delivery model was developed in 2015, which planned for evaluation activities up to December 2017. Despite a departmental evaluation strategy which indicates the importance of ongoing program monitoring and evaluation for new and high value policies or programs such as the PHN delivery model, there was no evaluation plan for the PHN delivery model after December 2017. A 2018 evaluation focused on the early implementation of the PHN delivery model. The PHN delivery model has not been comprehensively evaluated to determine whether it is meeting its objectives. There have been 23 evaluations of pilot programs and time limited grants provided through PHNs, however Health has not undertaken a consolidated review of findings from these evaluations to reach a conclusion about the effectiveness of the overall delivery model. (See paragraphs 4.1 to 4.9)

21. An early implementation evaluation reported in 2018 was inconclusive about whether the PHN delivery model was achieving its objectives but stated there were early indications of progress towards achieving objectives. Although there has been some improvement between 2018–19 and 2020–21 in PHNs’ average performance against some measures, no baseline data for these measures and the lack of relevant performance measures means that it is not possible to conclude if the PHN delivery model has met its objectives. (See paragraphs 4.12 to 4.17)

Recommendations

22. This report makes eight recommendations to Health.

Recommendation no. 1

Paragraph 2.24

The Department of Health and Aged Care ensure that PHNs fully comply with transparency and accountability requirements established in grant agreements, including requirements to participate in and provide data and information for the purposes of evaluation.

Department of Health and Aged Care response: Agreed.

Recommendation no. 2

Paragraph 3.25

The Department of Health and Aged Care establish performance measures that are clearly aligned to the Primary Health Networks’ and delivery model’s objectives.

Department of Health and Aged Care response: Agreed.

Recommendation no. 3

Paragraph 3.28

Where there is a reliance on Primary Health Network-supplied data, the Department of Health and Aged Care establish a risk-based methodology for obtaining assurance over the data.

Department of Health and Aged Care response: Agreed.

Recommendation no. 4

Paragraph 3.35

The Department of Health and Aged Care report on Primary Health Network performance as soon as practicable following the year to which the majority of the performance information relates.

Department of Health and Aged Care response: Agreed.

Recommendation no. 5

Paragraph 3.47

The Department of Health and Aged Care publish individual PHNs’ performance data and analysis in annual reports.

Department of Health and Aged Care response: Agreed.

Recommendation no. 6

Paragraph 3.52

The Department of Health and Aged Care publicly report on performance measures:

  1. in compliance with the Primary Health Network performance framework by reporting all performance measures; and
  2. in a way that is consistent with the intended purpose of conveying information about performance in addition to compliance with grant agreement requirements.

Department of Health and Aged Care response: Agreed.

Recommendation no. 7

Paragraph 3.59

The Department of Health and Aged Care implement a fit-for-purpose IT system for administering Primary Health Networks that supports the accurate capture and reporting of compliance and performance information.

Department of Health and Aged Care response: Agreed in principle.

Recommendation no. 8

Paragraph 4.10

The Department of Health and Aged Care:

  1. develop an evaluation plan for the Primary Health Network delivery model; and
  2. evaluate the Primary Health Network delivery model to determine whether it is achieving its objectives.

Department of Health and Aged Care response: Agreed.

Summary of entity response

23. The proposed audit report was provided to Health. Health’s summary response to the audit is provided below and its full response is at Appendix 1.

The Department of Health and Aged Care (the Department) welcomes the findings in this report and accepts the majority of recommendations. The Department is committed to implementing the recommendations and is taking steps to address the areas identified for improvement.

It is pleasing to note that compliance and assurance arrangements for the Primary Health Network (PHN) Program are fit for purpose. The audit identified opportunities to improve performance measurement and reporting for the PHN Program. These findings build on work already underway with PHNs through the Governance of PHN reporting framework, to redesign the Performance and Reporting Framework to include outcomes focused performance measures that align to the PHN delivery model objectives.

The Department is committed to demonstrating the performance of the PHN Program and will evaluate the PHN delivery model. The Department agrees in principle to recommendation seven noting investment in a bespoke system will be complex and will require funding decisions by Government.

The Department appreciates the report’s recognition that the Department is continuously engaging and consulting with its partners and key stakeholders to improve the effectiveness and operation of the PHN Program, and will continue this approach in developing and implementing our response.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Group title

Performance and impact measurement

Key learning reference
  • The development of an evaluation framework that includes a program logic model, data strategy and timeline for evaluation is an essential design element when developing new government programs.
Group title

Grants

Key learning reference
  • Compliance with mandatory GrantConnect reporting requirements of the Commonwealth Grants Rules and Guidelines supports transparency over grant expenditure, including approval dates, award amounts and whether the grant opportunity was competitive or non-competitive.

1. Background

Introduction

1.1 Primary Health Networks (PHNs) were established by the Department of Health and Aged Care (Health) on 1 July 2015 as a delivery model for primary health care.2 There are 31 PHNs across all states and territories in Australia. Each PHN is responsible for the ongoing assessment of health needs in the PHN region, supporting health services and stakeholders, and commissioning and integrating health services at the local level3, to ensure that people can receive ‘the right care, in the right place, at the right time’.4 PHNs have two key objectives:

  • improving the efficiency and effectiveness of health services for people, particularly those at risk of poor health outcomes; and
  • improving the coordination of health services and increasing access and quality support for people.5

1.2 PHNs work across seven priority health areas comprising mental health, Aboriginal and Torres Strait Islander health, population health, the health workforce, digital health, aged care, and alcohol and other drug services. PHNs contributed to seven programs in Health’s 2022–23 Corporate Plan.6

1.3 Health awards and manages grant funding agreements with the PHNs and is responsible for:

  • the provision of guidance materials;
  • facilitating regular communications and information sharing with PHNs;
  • provision of data to support PHN needs assessments; and
  • monitoring the compliance and performance of PHNs and supporting PHNs to improve.7

Funding of Primary Health Networks

1.4 PHNs are non-government organisations funded through Australian Government grants. PHNs are registered with the Australian Charities and Not-for-profits Commission.

1.5 Health provides four streams of funding to PHNs: operational funding (for the administration, governance and core functions of PHNs); flexible funding (for PHN priority areas); program funding (for programs previously managed by Medicare Locals (see paragraph 1.8) which remained in scope for PHNs after Medicare Locals were abolished); and innovation and incentive funding (for new models of primary health care delivery, that may be introduced across PHNs if successful). The amount of funding provided to each PHN is dependent on the population size of the PHN region, rurality and socio-economic characteristics, among other factors.

1.6 Between 2015–16 and 2023–24, $11.576 billion in grant funding commitments were made to PHNs (Figure 1.1 and Figure 1.2).8 Of the total, $2.814 billion was committed to core funding agreements to fund PHN governance, operations, commissioning and coordinating activities, and $6.407 billion was committed to three of the seven PHN priority areas ($4.9 billion to mental health, $848 million for alcohol and other drugs and $682 million for Aboriginal and Torres Strait Islander health). There was also $2.104 billion committed to PHNs for various pilot and other specific programs. Separate expenditure on the population health, health workforce and digital health priority areas cannot be separately quantified due to the structure of Health’s financial information. Appendix 3 shows grant commitments for each PHN.

Figure 1.1: Grant commitments to PHNs by state and territory, 2015–16 to 2023–24 (as at August 2023)

A figure that shows the distribution of total grant commitments to Primary Health Networks disaggregated by state and territory from 2015–16 to 2023–24. The amount of grant commitments ranges from $178 million to the ACT to $3.34 billion to NSW.

Source: ANAO analysis of Health’s master financial spreadsheet, as at August 2023.

Figure 1.2: Grant commitments to PHNs by core funding and priority area, 2015–16 to 2023–24 (as at August 2023)

A figure that shows the distribution of total grant commitments to Primary Health Networks from 2015–16 to 2023–24, disaggregated by priority funding area. The figure shows that $2.81 billion was allocated to Primary Health Networks core funding, $4.87 billion to mental health, $848 million to Alcohol and other drug services, $681 million to Aboriginal and Torres Strait Islander health, $251 million for aged care, and $2.1 billion for pilots and other programs.

Note: Pilots and other programs comprise: a range of targeted programs in addition to funding for the Community Health and Hospitals Program; after hours support; the Commonwealth Psychosocial Support Program; continuity of support; mental health – bushfire support; National Mental Health and Suicide Prevention Agreement and Bilateral PHN Program; National Psychosocial Support measure; Partners in Recovery, and urgent care clinics.

Source: ANAO analysis of Health’s master financial spreadsheet, as at August 2023.

1.7 Commonwealth grants provided to PHNs are subject to the requirements of the Commonwealth Grants Rules and Guidelines, issued by the Minister for Finance under section 105C of the Public Governance, Performance and Accountability Act 2013.9

Establishment of Primary Health Networks

1.8 Between 2011 and 2015, ‘Medicare Locals’ fulfilled a similar function to PHNs. A total of 61 Medicare Locals were created in three tranches in July 2011, January 2012 and July 2012 in response to the 2011 National Health Reform Agreement.10 Medicare Locals were created to identify the gaps in primary health care and improve service delivery at a local level.

1.9 In 2013 the Minister for Health commissioned Professor John Horvath AO to conduct a review into Medicare Locals, including all aspects of their structure, operation and functions. The Minister for Health stated that the purpose of the review was ‘reducing waste and spending on administration and bureaucracy, so that greater investment can be made in services that directly benefit patients and support health professionals who deliver those services to patients’.11 The review’s March 2014 report found that:

  • the name ‘Medicare Local’ was confusing;
  • general practitioners were not appropriately involved in Medicare Locals;
  • a number of Medicare Locals were not fulfilling their intended role of integrating care for patients, who continued to experience ‘fragmented’ health care;
  • there was variability between Medicare Locals in expenditure on administration, levels of funds allocated to frontline services, and accounting practices, and inconsistencies between planned and actual budgets;
  • there was a lack of clarity in what Medicare Locals were trying to achieve, variability in scope and delivery and inconsistent outcomes;
  • reporting requirements were burdensome for Medicare Locals; and
  • performance measures were input and process driven rather than outcome focused.

1.10 The Medicare Local review made 10 recommendations, including that a network of ‘Primary Health Organisations’ (PHOs) be established through ‘contestable’ processes. The tenth recommendation was that ‘PHO performance indicators should reflect outcomes that are aligned with national priorities and contribute to a broader primary health care data strategy’.

1.11 In April 2014 the Minister for Health accepted the review’s recommendations. Health provided the Minister with two options for the establishment of PHOs: a new policy proposal outlining options for establishing PHOs by July 2016 (Approach 1), or providing funding in the May 2014 Budget to ‘fast track’ PHO establishment by 1 July 2015 (Approach 2). Health explained that Approach 1 had the advantage of consideration by Cabinet and the sector through consultation, a policy development process and the commissioning of design expertise, facilitating a ‘robust and viable PHO model’. Approach 2 had the advantage of reducing uncertainty for the sector. The Minister for Health chose Approach 2. The 2014–15 Budget indicated that PHNs would replace Medicare Locals on 1 July 2015 and cease Commonwealth funding to Medicare Locals.

1.12 Twenty nine providers received Australian Government grants on 1 June 2015 to establish 31 PHNs.12 PHNs were established to be closely aligned with state and territory Local Hospital Networks.13 PHNs represent populations ranging from approximately 64,000 people in Western Queensland to around 1.7 million people in North Western Melbourne.

Figure 1.3: Primary Health Network boundaries, 2022

Figure shows a map of Australia that includes the geographical boundaries of each Primary Health Network region. The figure includes a numerical legend to support the identification of Primary Health Network regions shown in the map.

Source: Health.

Rationale for undertaking the audit

1.13 Since the establishment of PHNs in 2015–16, Health has committed $11.6 billion to PHNs in grants funding. The commissioned services and activities of PHNs contributed to seven programs across three outcomes in Health’s 2022–23 Corporate Plan. The audit was identified by the Joint Committee of Public Accounts and Audit as an audit priority of the Parliament in 2021–22 and 2022–23.

1.14 Auditor General Report No. 9 of 2019–20 National Ice Action Strategy Rollout examined the expansion of alcohol and other drug treatment services through PHNs. Health agreed with the ANAO’s recommendation to ‘finalise the PHN Quality and Assurance Framework, with appropriate actions to assess whether PHNs are operating appropriately across the commissioning cycle’.

1.15 The audit aims to provide assurance to the Parliament that Health is appropriately monitoring compliance and performance of individual PHNs, as well as the overall performance of the PHN delivery model.

Audit approach

Audit objective, criteria and scope

1.16 The purpose of the audit was to assess the effectiveness of the Department of Health and Aged Care’s performance management of Primary Health Networks.

1.17 To form a conclusion against the objective, the ANAO examined:

  • Has Health established fit-for-purpose compliance and assurance arrangements for the PHNs and the PHN delivery model?
  • Has Health established fit-for-purpose program performance measurement and reporting arrangements for PHNs and the PHN delivery model?
  • Has Health effectively evaluated the PHN delivery model to demonstrate that it is meeting its objectives?

Audit methodology

1.18 The methodology involved:

  • examining Health records, including performance data and reporting;
  • high-level examination of relevant record management systems;
  • meetings with PHNs;
  • meetings with Health personnel; and
  • submissions to the audit from external stakeholders (31 submissions were received from individuals, primary health networks, industry groups, professional associations and health services providers).

1.19 Australian Government entities largely give the ANAO electronic access to records by consent, in a form useful for audit purposes. In April 2022 the Department of Health and Aged Care advised the ANAO that it would not voluntarily provide certain information requested by the ANAO due to concerns about its obligations under the Privacy Act 1988, secrecy provisions in Health portfolio legislation, confidentiality provisions in contracts and the Public Interest Disclosure Act 2013. For the purposes of this audit, the Auditor-General therefore exercised powers under section 33 of the Auditor-General Act 1997 to enable authorised ANAO officers to attend premises, and examine and take copies of documents. Health facilitated authorised officers attending the Health’s premises to examine and copy documents, however the requirement was extended by Health to all documents, including those that did not relate to Health’s obligations under legislation. Health advised that this type of information largely was not segregated in Health’s record-keeping systems and Health could not be certain, in providing documents through electronic means, that documents containing this type of information were excluded. To provide comfort to the Secretary regarding Health’s obligations under portfolio legislation, on 9 August 2023 the Auditor-General issued the Secretary of Health with a notice to provide information and produce documents pursuant to section 32 of the Auditor-General Act 1997. Under this notice, Health agreed to provide the information and documents requested through electronic means.

1.20 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $552,500.

1.21 The team members for this audit were Michael Commens, Lily Engelbrethsen, Katiloka Ata, Andrew Yam, Dale Todd, Alicia Vaughan, Elizabeth Robinson and Christine Chalmers.

2. Assurance

Areas examined

This chapter examines whether the Department for Health and Aged Care (Health) has established fit-for-purpose compliance and assurance arrangements for Primary Health Networks (PHNs) and the overall PHN delivery model.

Conclusion

Health has established largely fit-for-purpose compliance and assurance arrangements for PHNs and the PHN delivery model, although these arrangements were first established in 2021, almost six years after the implementation of the PHN delivery model. As at December 2023, there is a compliance and assurance framework comprised of a strategy, risk management plan, requirements in grant agreements, schedule of assurance activities, clear roles and responsibilities and oversight arrangements. Risks are identified and assurance activities are planned over individual PHNs and the delivery model as a whole. However, the link between identified risks and assurance planning is unclear, and some planned activities are not undertaken. Assurance is also weakened by record keeping practices and a lack of public transparency.

Areas for improvement

The ANAO made one recommendation aimed at improving compliance with a grant agreement requirement for PHNs to participate in reviews and evaluations through the provision of data and information. The ANAO suggested that Health strengthen its assurance arrangements over a grant agreement requirement for PHNs to publish needs assessments and activity work plans on websites.

2.1 PHNs receive Australian Government grant funding for their administration and operations under the core schedule to grant agreements. Funding for the priority health areas of mental health, Aboriginal and Torres Strait Islander health, population health, health workforce, digital health, aged care, and alcohol and other drug services is provided through different grant agreement schedules. Grant agreements may also cover specific projects.

2.2 The Commonwealth Grants Rules and Guidelines (CGRGs) state that officials achieve value with relevant money in grants administration by ongoing monitoring and management and adopting an active risk identification and engagement approach.14 This requires fit-for-purpose planning of assurance processes, assurance over individual grantees against grant agreements, and assurance over the collective performance of the grants program.

Is there a fit-for-purpose compliance and assurance framework?

The PHN delivery model was first implemented in 2015 and lacked an assurance framework until 2021, although specific guidance and tools to support assurance activity have been in place since 2016. An assurance strategy and schedule were established in 2022 and a risk management plan in 2023. The assurance framework establishes responsibilities for assurance activities including audits of individual PHNs. There are several oversight bodies, although an operational working group was disbanded in 2022. Grant agreements with PHNs appropriately establish governance, acquittal and performance reporting requirements. Health has not complied with Commonwealth Grant Rules and Guidelines requirements to publish on GrantConnect approximately $10 billion in grants awarded to PHNs.

Governance arrangements

2.3 The PHN delivery model is managed by the Primary Health Networks Branch (PHN Branch) in the Primary Care Division of Health. Since 2017–18 the PHN Branch has had approximately 40 full time equivalent (FTE) employees (Figure 2.1).

Figure 2.1: Primary Health Networks Branch full-time equivalent (FTE), 2017–18 to 2023–24

Figure shows the number of full-time equivalent employees working in the Primary Health Networks branch from 2017–18 to 2023–24. The figure shows that the number of full-time equivalent employees varied year to year but remained relatively stable across the time period ranging from 37 in 2019–20 to 49 in 2022–23.

Source: ANAO analysis of Health data.

2.4 In September 2023 the PHN Program Board (see paragraph 2.6) considered the growth of the PHN delivery model since its establishment, noting an annual funding increase from $0.8 billion per annum in 2015–16 to just under $2 billion in 2022–23. The Program Board further noted that PHNs originally involved four policy areas in Health and 14 funding streams, and in 2022–23 involved 43 policy areas across 10 Health divisions and 47 funding streams.

2.5 Between February 2018 and August 2023 Health commissioned external consultants to undertake assurance, performance management, and risk management design and delivery activities for the PHN delivery model. Health advised the ANAO that there were $4.3 million in outsourced advisory services for work specific to the PHN Program between February 2018 and August 2023. This figure does not include funding that was included as part of broader Departmental contracts.

2.6 Line management and oversight arrangements for the PHN delivery model include the following.

  • Executive Committee15 — The role of Health’s Executive Committee is to provide strategic direction and leadership to ensure the achievement of outcomes documented in Health’s Corporate Plan and Portfolio Budget Statements. The Executive Committee met 145 times between 27 November 2018 and 5 December 2023, and discussed PHNs 10 times over this period. The most substantive examples are: in April 2019 the Executive Committee noted the status of recommendations from a July 2018 internal audit (see Appendix 4); in July 2019 the Executive Committee discussed the outsourcing of grant service delivery through PHNs; in November 2020 the Executive Committee discussed PHNs in the context of telehealth16; and in October and November 2022 the Executive Committee discussed a September 2022 presentation that had been made to the Program Assurance Committee (discussed at Table 2.1).
  • Program Assurance Committee (PAC) — The PAC is a Health advisory body in place since 2018 that is intended to provide a strategic view of program design and delivery, guidance to assist business areas to improve program design and delivery, and assurance to the Secretary and Executive Committee on the effectiveness of program management. The PAC terms of reference state that it will meet approximately seven times each year. Membership of the PAC comprises Health senior executive officials, primarily at the Deputy Secretary and First Assistant Secretary level. The PAC discussed PHNs and programs involving PHNs five times between September 2018 and December 2023, including a September 2022 presentation discussed further at Table 2.1.
  • PHN Program Board — Health established a PHN Program Board in July 2018 to ensure a ‘strategic and integrated’ approach to PHN delivery model management. The PHN Program Board was established to monitor PHN delivery model performance, promote best practice program management, oversee the delivery of priority projects and manage program risks. It is chaired by the Deputy Secretary Primary and Community Care Group and its membership comprises Health senior executive officials who have policy responsibility for PHN grant agreement schedules and program activities. The PHN Program Board is required to meet between two and four times each year. The Program Board met 15 times between 6 July 2018 and 23 March 2023 (three times annually, on average).
  • PHN Operational Working Group — The first meeting of the PHN Operational Working Group took place in March 2018. Health established terms of reference in August 2019, which were last updated in December 2021. The terms of reference stated that the PHN Operational Working Group was the primary mechanism to progress the work program of the PHN Program Board. The PHN Operational Working Group was comprised of Health directors with responsibility for PHN grant agreement schedules. The terms of reference state that the group will meet every three weeks, or about 17 times each year. From 1 March 2018 to September 2022, the group met 41 times (once in 2018, five times in 2019, 17 times in 2020, 15 times in 2021 and three times in 2022), prior to being disbanded in September 2022.
  • PHN Branch — The PHN Branch within the Primary Care Division has primary oversight of PHNs and the PHN delivery model, manages PHN grants, communicates with PHNs, undertakes reporting and undertakes assurance.
  • Other groups and divisions within Health — Other policy areas within Health are involved in aspects of the management of PHNs. These include the Mental Health and Suicide Prevention Division, the Ageing and Aged Care Group, the Population Health Division and the First Nations Health Division. Responsibilities include policy decisions for relevant grant agreements, reviews of PHN deliverables after these are processed by the PHN Branch; and reporting on specific performance measures and engaging with PHNs on these measures.

2.7 Health advised the ANAO that the PHN Operational Working Group was replaced by an ‘informal’ committee of assistant secretaries that ‘meets as required to share information and collaborate on strategic priorities for the PHN Program’. Health developed a ‘PHN Service Offer’ in June – September 2023, which describes the roles and responsibilities relating to the PHN delivery model (including those set out in paragraph 2.6, except for the Executive Committee and PHN Operational Working Group, and including reference to a ‘PHN Program Assistant Secretary Committee’). The PHN Program Assistant Secretary Committee is described as Health’s ‘senior governance forum with oversight of program delivery, performance and assurance’.

Grant agreements with individual Primary Health Networks

2.8 In addition to the core grant agreements and priority activity schedules described in paragraph 2.1, grants are awarded to PHNs for specific projects, trials, commissioning of third-party service providers and other activities. Health maintains records of funding commitments and payments to PHNs, including a history of all funding allocations since 2015. Although Health maintains a consolidated record of funding commitments, it does not maintain consolidated information on grant award numbers (as would be listed on GrantConnect), and the types of grants awarded. When the ANAO requested this consolidated grant information from Health, Health was unable to provide it.

2.9 The CGRGs state that reliable and timely information on grants awarded is a precondition for public and parliamentary confidence in the quality and integrity of grants administration. The CGRGs require Commonwealth entities to publish individual grant awards within 21 days of the grant taking effect.17 Auditor-General report No. 31 of 2022–23 Administration of the Community Health and Hospitals Program identified non-compliance with mandatory GrantConnect reporting requirements and included a recommendation for Health to establish a quality assurance process to confirm and where necessary correct the accuracy of reporting on GrantConnect.

2.10 Health’s reporting on GrantConnect of PHN grants is inconsistent with Health’s records of funding commitments and payments to PHNs. As at May 2023 GrantConnect included 841 PHN grants valued at $2.4 billion. This compares with $11.6 billion in funding commitments and $10.3 billion in payments made to PHNs by Health as at August 2023. As information about each grant awarded to each PHN has not been reported accurately by Health on GrantConnect and is not maintained by Health in another consolidated form, the ANAO was unable to determine the total number of grants awarded, the number of grants awarded to each PHN and the types of selection processes followed in determining grants awarded to PHNs. The accurate reporting of grant awards on GrantConnect is a mandatory requirement of the CGRGs, which are legal requirements under finance law. Health advised the ANAO that it is aware of this issue and intends to correct GrantConnect reporting by May 2024.

2.11 The ANAO reviewed core grant agreements in detail for a targeted sample of six PHNs.18 Core grant agreements for the sampled PHNs appropriately established governance, acquittal and performance reporting requirements.

  • Governance requirements — grant agreements for each of the six sampled PHNs included requirements for:

    • maintaining appropriate governance structures, including a skills-based board, General Practitioner (GP)-led Clinical Councils and Community Advisory Committees;
    • maintaining governance arrangements to appropriately manage conflicts and related party involvement;
    • participating in and allowing access to premises, materials and documents, for the purpose of any audit as directed by Health;
    • publishing key documents including needs assessments19 and activity work plans20 on PHN websites; and
    • supporting the monitoring and evaluation of the PHN through the provision of documents and surveys when requested by Health.
  • Acquittal and performance reporting requirements — grant agreements for each of the six sampled PHNs included requirements for:
    • milestone reports21 to be submitted before payments are made, and which include annual needs assessments and annual activity work plans;
    • annual performance reporting against the 2018 PHN Program Performance and Quality Framework (discussed further at paragraphs 3.19 to 3.21); and
    • annual audited financial statements for core operations and activities, priority area activities, and pilots and other ad hoc projects (such as grants for mental health support for bushfire affected communities).

Assurance strategy, guidance and tools

2.12 Health first developed a documented strategy for assuring the performance of PHNs in 2021, five years after the PHN delivery model was implemented. The PHN Assurance Framework was developed in response to a recommendation from a 2018 internal audit which found that there were gaps in quality assurance processes. In 2021 Health engaged PwC to develop the assurance framework, which was to be operationalised from 2020–21 to 2023–24.

2.13 The PHN Assurance Framework states that it was developed to support the performance and operation of PHNs and provide greater assurance to the Australian Government over Health’s activities to ensure PHNs are operating appropriately, in accordance with their legal and financial obligations, and meeting objectives. The Assurance Framework states that it is designed to assure the performance of individual PHNs and the PHN delivery model as a whole, and that it takes a risk-based approach to conducting assurance over both. Assurance activities listed in the PHN Assurance Framework comprise: self-assessment checklists; desktop reviews; enquiries and walkthroughs; process reviews (including observation); deep dives and detailed reviews; risk based and random sampling; and independent assurance engagement.

2.14 The implementation of the PHN Assurance Framework is supported by two key documents developed in 2022 and 2023: an Assurance Strategy and a Risk Management Plan.

  • PHN Assurance Strategy — This applies to 2022 to 2024. The PHN Assurance Strategy states that it will assist Health to identify key risks to the PHN delivery model, identify gaps in assurance and mitigation strategies, and target assurance activities to the areas of greatest risk. There is a timeline for the implementation of assurance activities.
  • PHN Risk Management Plan — This was established in March 2023. It sets out the processes to identify, analyse, treat, monitor and manage risks and issues. Health maintains other guidance and templates to support risk owners and other officials to manage PHN risks, including a grant recipient risk tool for monitoring individual PHN risks.

2.15 Prior to and since the development of the 2021 PHN Assurance Framework and to support its implementation, Health has maintained guidance materials and tools for use by PHNs or Health officials tasked with assurance work. Forms of guidance materials have existed since 2016. Further development on materials was undertaken from 2019. Examples include the following.

  • In 2019 Health commissioned KPMG to develop process maps for PHN grants management, reporting, assurance and compliance processes.
  • In 2021 Health established guidance for the assessment of PHNs’ 12-month performance reports (discussed further at paragraphs 3.19 to 3.21) including an Assessment Process Guide; Compliance Checklist; Individual Performance Assessment template, PHN Program Electronic Reporting System ‘Indicator Guide’; and a ‘Performance Rubric’.22
  • In 2021 and 2022 Health revised guidance for reviewing PHNs’ needs assessments and activity work plans, including a revised assessment template for the activity work plan.

Are assurance arrangements for individual Primary Health Networks fit-for-purpose?

Health established an assurance plan for individual PHNs in 2022, which includes activities under three ‘lines of assurance’. Health’s assurance schedule does not include activities to monitor compliance with grant agreement requirements relating to the publication of needs assessments and activity work plans. Health took no action in response to PHN non-compliance with grant agreement requirements to provide financial information for a 2020 review of operational expenditure. Line one assurance activities (environmental scans, PHN risk assessments and assessments of PHN milestone reporting under grant agreements) were not always completed as planned or are difficult to assess due to poor record keeping. Line two assurance activities (including assessment of PHN performance reporting and reviews by oversight bodies) were largely undertaken as planned. Line three assurance activities (risk-based audits of PHNs) were undertaken as planned. Health tracks PHNs’ implementation of recommendations from audits. The PHN Assurance Framework states that it takes a risk-based approach to conducting assurance, however risk assessments of PHNs are not undertaken as planned.

Assurance planning for individual Primary Health Networks

2.16 The PHN Risk Management Plan states that risks are identified and managed through day-to-day activities and policies, including annual individual PHN risk assessments (discussed at paragraph 2.18 and Table 2.1).

2.17 Separately to the annual individual PHN risk assessments, Health engaged PwC to undertake ‘baseline maturity assessments’ of the PHNs, with the final report provided to Health in April 2022.23 The objective was to evaluate each PHN’s effectiveness, efficiency and economy of operations; identify risks and issues that would not be picked up during business as usual assurance activities; and provide Health with confidence that individual PHNs are operating appropriately, in accordance with their legal and financial obligations. Overall baseline maturity risk assessments (which ranged from ‘very low’ to ‘very high’) were derived from a quantitative financial risk indicator24 and qualitative risk indicators. 25 The 31 PHNs were assessed as having an overall baseline maturity risk of ‘very low’ (2), ‘low’ (12), ‘medium’ (7) and ‘high’ (10). None were assessed as having a ‘very high’ overall risk. The baseline maturity assessments included recommendations relating to sharing best practice and implementing recommendations from previous PHN delivery model reviews, as well as the revision of both the qualitative and quantitative risk indicators. The baseline maturity assessment included some commentary on areas of improvement for individual PHNs but did not include recommendations for individual PHNs. The PHN Program Board agreed that from July 2024 Health will commence a second round of PHN maturity assessments, following a review of the baseline maturity assessment methodology (which is scheduled to commence in March 2024).

2.18 Annual individual PHN risk assessments are documented using a template that contains some of the same risk indicators captured in the baseline maturity assessments, including PHN issues management, governance and performance management. However, the results of annual PHN risk assessments are inconsistent with the baseline maturity assessment. For example, two PHNs in the ANAO’s sample of six PHNs were assessed as an overall low risk in the baseline maturity assessment but were assessed as high risk in the annual PHN risk assessment (last updated for one of the PHNs in September 2022).26 Health advised the ANAO that the baseline maturity risk assessments ‘utilised different criteria’ and had ‘lower tolerance levels’ than the annual PHN risk assessments.

2.19 Health first established a PHN Assurance Schedule for activities targeting the PHN delivery model and individual PHNs in 2022–23. The PHN Assurance Schedule outlines the assurance activities for the implementation of the PHN Assurance Framework. The one-page document sets out the timeframes (by month, over one year) for the achievement of milestones and assurance activities. ‘Next financial year’ activities are also summarised. The PHN Assurance Schedule categorises activities into three lines of assurance: first line (operational management), second line (oversight including risk management and compliance) and third line (internal audit).

2.20 The PHN Assurance Framework states that it takes a risk-based approach to conducting assurance. While the baseline maturity assessments have been used to inform the choice of PHNs to audit (a third line activity, see Table 2.1), there is no evidence that the annual PHN risk assessments or the baseline maturity assessments were used to inform the planning of Health’s first and second line assurance activities.

2.21 Grant agreements with PHNs require PHNs to publish annual needs assessments and activity work plans on PHN websites. Neither the Assurance Framework nor the Assurance Schedule includes compliance checks for these requirements. As at January 2024, out of 31 PHNs, the Northern Territory PHN website does not comply with relevant grant agreement requirements.

Opportunity for improvement

2.22 Health could include checks of PHN websites in assurance planning to ensure that PHNs publish up to date needs assessments and activity work plans on websites.

2.23 Grant agreements with PHNs state that PHNs are required to participate in and support evaluation including by providing financial and non-financial information and data to evaluators. A 2020 KPMG operational expenditure review (see Appendix 4), which was conducted in response to a recommendation of a 2018 evaluation, states that 26 per cent of PHNs did not respond to a survey seeking to quantify operational expenditure, and that other PHNs did not provide appropriate responses. The aim of the operational expenditure review was to identify the actual or ‘true’ operational expenditure of PHNs compared to what was reported, and identify unique and common cost drivers. As a consequence of PHN non-response, the review was unable to generalise findings across regional, remote and metropolitan regions, and comparative analysis included in the report was incomplete. Health advised the ANAO that it did not take any action in response to non-compliance with requirements to participate in the survey, including identifying which PHNs were non-compliant.

Recommendation no.1

2.24 The Department of Health and Aged Care ensure that PHNs fully comply with transparency and accountability requirements established in grant agreements, including requirements to participate in and provide data and information for the purposes of evaluation.

Department of Health and Aged Care response: Agreed.

2.25 The Department of Health and Aged Care requires Primary Health Networks (PHNs) to comply with the requirements of their grant agreements and will enhance existing assurance processes to further support compliance and follow-up where PHNs do not respond to requests for information.

Assurance activities over individual Primary Health Networks

2.26 The ANAO examined whether the individual PHN assurance activities planned for 2022–23 were undertaken in accordance with the line one, two and three activities listed in the PHN Assurance Schedule (see Table 2.1 and paragraph 2.19 for Health’s definition of the three lines).

Table 2.1: Assessment of Health’s implementation of individual PHN assurance activities against the PHN Assurance Schedule

Assurance category

Planned assurance activity

Completed?

Analysis

Line 1 — operational management

Quarterly environmental scans

Health did not undertake quarterly environmental scans in 2022–23.

Line 1 — operational management

Individual PHN risk assessments in August of each year

The ANAO reviewed annual PHN provider risk assessments for the six PHNs included in the ANAO’s targeted sample (see paragraph 2.8). Risk assessments for five of six PHNs had not been updated in August 2023 as required.

Line 1 — operational management

Assessment of PHN milestone reporting (including needs assessments and activity work plans)

Unable to fully assess

The ANAO attempted to review Health’s assessment of PHN milestone reporting covering 2019–20 to 2021–22 for the six PHNs included in the ANAO’s targeted sample. There was evidence of submission and approval of milestones reports as required under grant agreements. The ANAO could see evidence of PHNs being required to submit multiple versions of needs assessments prior to approval from Health and Health approving activity work plans. For example, the ANAO observed that Health had identified incorrect funding allocations in a milestone report, requiring the PHN to resubmit. There was evidence of Health providing feedback to PHNs on other deficiencies in milestone reports.

Information management standardsa and the CGRGsb require officials to maintain appropriate records. Although the ANAO found evidence that milestone reporting was being assessed, it was not possible to confirm whether this had been done appropriately due to difficulty in identifying the authoritative milestone documents (including the activity summaryc, budget summaryd and Health’s assessments of the activity work plan and budget), for each activity work plan in the ANAO’s targeted sample. This was due to inconsistent naming conventions, multiple versions of documents, inconsistent file types being used by PHNs (despite requirements for PHNs to submit reports in a standard file type) and document references in the PHN Program Electronic Reporting System (PPERS, see paragraph 3.55) linking to folders containing multiple copies of documents without appropriate version control.

Line 2 — oversight

Individual PHN performance assessments

PHNs are required to submit a 12-month performance report each year. Financial and non-financial performance information included in the 12-month report is to be assessed by Health and documented in its performance assessment for each PHN. The Assurance Schedule includes Health assessments of 12-month performance reporting for each PHN.e The ANAO reviewed annual performance assessments prepared by Health (and associated PPERS data extracts) for the six PHNs in the ANAO targeted sample for 2019–20, 2020–21 and 2021–22.f

Annual assessment reports were prepared by Health and completed in accordance with the Performance Rubric. Health’s performance assessments for each PHN in 2019–20 and 2020–21 included an assessment against 42 out of 54 performance measures (see paragraph 3.19). Health’s performance assessments in 2021–22 included an assessment against a reduced subset of 32 performance measures, as reporting requirements were reduced by Health.g All six sampled and examined PHNs were ‘on track’ or ‘progressing’ against each of four themes.In 2021–22, the number of indicators met ranged from 23 to 26 out of 32. Consistent with Health’s assessment guidance, Health’s draft performance assessments were provided to PHNs for comment, with PHN comments and Health’s response to the comments included in the final assessment reports.

Line 2 — oversight

Program reviews by the Program Assurance Committee and Audit and Risk Committee

A September 2022 ‘Health Check of the PHN Program’ conducted by Synergy (see Appendix 4) was noted by Health’s Audit and Risk Committee on 29 September 2022, largely consistent with the Assurance Schedule. Audit and Risk Committee papers do not include minutes on any discussion of findings, recommendations or underlying risks identified in the Synergy Health Check.

The PHN delivery model was the subject of a scheduled review by the PAC on 15 September 2022. The PAC received updates on the implementation of the PHN Assurance Framework, compliance improvements, the ongoing development of the PHN Program Performance and Quality Framework and the revision of performance indicators.

Line 2 — oversight

PHN Australian Charities and Not-for-profits Commission self-assessments

The Assurance Schedule includes requirements for Health to engage with PHNs around the completion of the Australian Charities and Not-for-profits Commission (ACNC) self-assessment and checklist for conformance with ACNC governance standards. Health did not engage with PHNs on the completion of the ACNC self-assessment in accordance with the Assurance Schedule.

Line 3 — Internal audit

In June 2022, line three activities for 2022–23 comprised six commissioned audits of individual PHNs commencing or completed in 2022–23.

ANAO reviewed planning records for all six planned audits, draft and final reports for all six audits and final reports for two completed audits. All six audits were undertaken in accordance with Health’s PHN Assurance Schedule. Health’s choice of PHN to audit was based on the outcomes of baseline risk maturity assessments.h Five of six PHNs targeted for an audit were assessed as a high overall risk. One PHN was assessed as low overall risk but had below average qualitative assessments on governance, monitoring and reporting. The scope for all six audits included PHNs’ governance and decision-making processes; financial management, planning and reporting; organisational capacity and capability; and probity and commissioning practices.

Health maintains a tracker for documenting the findings and implementation status of recommendations from PHN audits (see Appendix 5).

       

Key:  Largely or fully implemented as planned Partly implemented as planned Not implemented

Note a: National Archives of Australia, Information Management Standard for Australian Government, available from https://www.naa.gov.au/information-management/standards/information-management-standard-australian-government [accessed 8 November 2023].

Note b: Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 12.6, p. 33.

Note c: The activity summary is a template report describing the various activities planned by PHNs and the alignment with assessed needs and government priorities. The summary includes planned expenditure and timelines for completion of each activity.

Note d: The budget summary has, for each priority area, the proposed expenditure and the anticipated variation between income and expenditure for the relevant activity.

Note e: The Performance Rubric contains three assessment standards: ‘On Track’ (‘PHN has met performance criteria for all indicators’), ‘Progressing’ (‘PHN has met performance criteria for half of the indicators and is working towards meeting the other indicators’) and ‘Initial’ (‘PHN has met performance criteria for one indicator’). Health assesses activities using a traffic light system where green is on track, amber is progressing, and red is not progressing.

Note f: The 2022–23 annual performance assessments were to be completed in December 2023.

Note g: Organisational capability indicators reported against in 2019–20 and 2020–21 were not required to be reported against in 2021–22.

Note h: North Queensland, Australian Capital Territory, South Eastern Melbourne, Adelaide, Northern Sydney and Northern Territory PHNs were subject to audits in 2022–23.

Source: ANAO analysis of Health documentation.

Are delivery model assurance arrangements fit-for-purpose?

Health assesses risks and ‘hot issues’ for the PHN delivery model as a whole. Governance bodies have oversight of PHN risks, however consideration of risk by governance bodies has not been consistent with Health’s assurance planning and the terms of reference for these bodies. Health plans assurance activities over the delivery model as a whole, which are set out in an Assurance Framework or Schedule, however it is unclear how risk assessments inform assurance plans. The planned activities to provide assurance over the PHN delivery model have been partly carried out. In 2022–23 the Program Board did not meet as frequently as required. Six management reviews have been undertaken since 2018, including one in 2022–23. Health tracks its implementation of recommendations from these management reviews and assesses 76 per cent of agreed recommendations to have been implemented as at July 2023, with some 2018 recommendations related to performance monitoring still outstanding. Planned reviews of the PHN Assurance Framework and the Program Performance and Quality Framework in 2022–23 were not completed.

Delivery model risk assessment

2.27 PHNs were established by Health on 1 July 2015 as a delivery model for primary health care. The PHN Risk Management Plan states that delivery model risks are identified and managed through an overarching delivery model risk register; a quarterly delivery model risk register review by PHN governance groups; and a quarterly review of risk by all program and policy areas. The ANAO did not review program and policy areas’ risk assessments as the focus of this performance audit was the management of the PHN delivery model, which is managed within the PHN Branch.

Delivery model risk register

2.28 Health maintains an overarching risk register for the PHN delivery model consistent with the Risk Management Plan. The ANAO reviewed risk registers, updated approximately every six months, covering the period October 2021 to September 2023. The September 2023 risk register documents eight risks (seven medium risks and one high risk relating to Health’s ability to manage the expansion of PHNs’ roles).

2.29 In the March 2023 and September 2023 risk registers, two moderate financial and fraud risks were accepted outside Health’s tolerance level, which is set at low for financial and fraud risks. Key controls for the fraud risk, such as audited financial statements and PHN internal controls, were described as fully effective. This is not consistent with analysis undertaken by the ANAO on financial reporting, or with financial governance and conflict of interest management findings and subsequent recommendations from PHN audits undertaken by Health (Appendix 5). The ANAO compared financial reporting in PHNs’ 12-month performance reports to audited financial statements required under grant agreements. For all six target sample PHNs, there were instances of 12-month performance reporting being inconsistent with the audited financial statements in 2019–20, 2020–21 and 2021–22. Health advised that it undertakes analysis to better understand inconsistencies in financial reporting and that it has established a dedicated finance team to work directly with PHNs to improve the accuracy of financial reporting.

2.30 Since 2019–20 Health has maintained a ‘hot issues’ register, documenting a range of issues including the development of compliant grant opportunity guidelines and the accuracy of PHN financial reporting. Hot issues registers also include issues relating to individual PHNs’ leadership and governance.

Quarterly review of delivery model risk register by governance groups

2.31 The Program Board discussed ‘hot issues’ or the PHN risk register in eight of 15 meetings held between July 2018 and March 2023. Since March 2023, when the Risk Management Plan was implemented, the Program Board met in March and in September 2023. The risk register was reviewed in the March and September 2023 meetings.

2.32 The ANAO reviewed 41 PHN Operational Working Group meeting records covering the period 1 March 2018 to its final meeting on 21 September 2022. The group discussed ‘hot issues’ at 36 of 41 meetings, including all meetings held after 13 November 2019, and was scheduled to meet on 16 December 2021 to review the PHN risk register, however the meeting was cancelled.

Assurance planning and activities over the delivery model

Assurance planning

2.33 Updates to the delivery model risk register did not feed into the 2021 Assurance Framework and the 2022 Assurance Strategy, which have not been reviewed or amended since their establishment.

2.34 There are some inconsistencies between the PHN Assurance Framework and the Assurance Schedule regarding delivery model assurance activities. The PHN Assurance Framework sets out meetings of the Program Board, ‘deep dives’ and process reviews as delivery model assurance activities. The Assurance Schedule also sets out Program Board meetings. Thematic assurance activities or ‘deep dives’ as described in the PHN Assurance Framework were not included in the 2022–23 Assurance Schedule. The Assurance Schedule indicated that a revised PHN Program Performance and Quality Framework, as well as an annual review of the Assurance Framework, would be undertaken in July 2023. Health advised that completion of this work has been delayed until June 2024.

Assurance activities

2.35 The ANAO examined whether the delivery model assurance activities planned for 2022–23 were undertaken in accordance with the PHN Assurance Framework and PHN Assurance Schedule (Table 2.2).

Table 2.2: Assessment of Health’s implementation of PHN delivery model assurance activities

Planned assurance activity

Completed?

Analysis

Meetings of the Program Board

The 2022–23 Assurance Schedule includes requirements to hold four meetings of the Program Board between July 2022 and June 2023. Health’s Program Board met three times between 1 July 2022 and 30 June 2023. ANAO analysis of Program Board meeting records between July 2018 and March 2023 found that it discussed findings and recommendations from thematic reviews and internal audits, including concerns with third party provider commissioning practices, PHN governance arrangements and data quality. For example, on 1 July 2022 the Program Board endorsed the Assurance Framework and Assurance Strategy, noted the baseline maturity assessment final report, and noted the maturity assessment ratings of individual PHNs.

Deep dives and process reviews

Since 2018 Health has commissioned six thematic reviews and internal audits targeting PHN delivery model requirements and activities, one of which (a ‘health check’) was undertaken in 2022–23. Appendix 4 sets out the scope, key findings, and recommendations of delivery model reviews between July 2018 and December 2023, and Health’s assessment of implementation of recommendations as at July 2023 (as at December 2023, this was the last time implementation status was monitored).

Health maintains a consolidated tracking spreadsheet to monitor the status of recommendations from delivery model assurance activities. The six thematic reviews made a total of 63 recommendations, of which 62 were agreed to by Health. Health assessed it had implemented 47 (76 per cent) of agreed recommendations as at July 2023. As at July 2023 the tracker indicated that four recommendations from the 2018 Evaluation of the PHN Program were still not complete. These relate to performance monitoring and PHN delivery model operations. Unimplemented recommendations from the 2020 Review of PHN Grants and Reporting also relate to performance management and performance data.

Annual review of PHN Assurance Framework

The Assurance Schedule indicated that an annual review of the Assurance Framework would be undertaken in July 2023. Health had not commenced this work as at December 2023. Health advised the ANAO that it is planning this review work and intends to include new programs introduced at the 2023–24 Budget.

Revised Program Performance Quality Framework

The Assurance Schedule indicated that a revised PHN Program Performance and Quality Framework was to be implemented from 1 July 2023. The revised due date, which was agreed by the Audit and Risk Committee, is 28 June 2024.

     

Key:  Largely or fully implemented as planned Partly implemented as planned Not implemented

Source: ANAO analysis of Health documentation.

3. Performance measurement and reporting

Areas examined

This chapter examines whether the Department of Health and Aged Care (Health) has established fit-for-purpose performance measurement and reporting arrangements for the Primary Health Network (PHN) delivery model.

Conclusion

Health’s performance measurement and reporting arrangements for PHNs are partly fit-for-purpose. There is a PHN program logic to inform the development of outcomes-oriented performance measures, however it lacks clarity. There are a number of performance measures, however many assess PHNs’ compliance with grant agreements rather than providing information about the achievement of outputs and outcomes. The performance measures are largely aligned to Australian Government guidance for performance measurement, except that there are weaknesses in relation to relevance and data assurance. Public performance reporting is not timely or informative about overall PHN delivery model performance, and does not include information about individual PHN performance. IT systems for PHN performance reporting are partly fit-for-purpose.

Areas for improvement

The ANAO made six recommendations aimed at establishing performance measures that are related to PHN delivery model objectives; developing a strategy to assure PHN-supplied performance data; releasing timely annual reporting; reporting on individual PHN performance; improving the usefulness and completeness of performance reporting; and implementing a fit-for-purpose IT system for capturing, validating and reporting on PHN compliance and performance.

The ANAO also suggested that Health could: improve a program logic for the PHN delivery model; consider the availability of secondary data when selecting performance measures and determining reporting timeframes; and include financial and efficiency analysis in annual reports.

3.1 Internal and external performance measurement and reporting facilitates good management, governance and decision-making. External reporting plays an important role in maintaining public trust.

  • The Commonwealth Performance Framework sets out requirements for Australian Government entities for measuring, assessing and publicly reporting on activities.27
  • The Commonwealth Grants Rules and Guidelines (CGRGs) require that ‘grants administration should have a performance framework that is linked to an entity’s strategic direction and key performance indicators.’28
  • The Commonwealth Evaluation Policy and Toolkit (which was established on 1 December 2021) emphasises the importance of a program logic model in developing performance information.29
  • The Archives Act 1983 makes Commonwealth agencies responsible for following record keeping requirements set out in the Information Management Standards. Principle Seven of the Information Management Standards states that business information is to be saved in systems where it can be appropriately managed, protecting the integrity of the information and supporting trusted and reliable use.30

Is there a fit-for-purpose program logic?

Health established a program logic for the PHN delivery model in 2015, which was revised in 2018 and 2023. The 2018 program logic contains most of the expected elements of a program logic and provides information about how the PHN delivery model will achieve outcomes. The 2023 program logic introduces new objectives for the PHN delivery model, lacks clarity and is not aligned with guidance on developing program logics.

3.2 The Australian Institute of Family Studies states that the purpose of a program logic is to:

… show how a program works and to draw out the relationships between resources, activities and outcomes … The logic model is both a tool for program planning — helping you see if your program will lead to your desired outcomes — and also for evaluation — making it easier to see what evaluation question you should be asking at what stage of the program.31

3.3 The Commonwealth Evaluation Toolkit includes a program logic template and guide. The program logic template suggests considering program objectives, inputs, activities, outputs, and causally linked short-term, medium-term and long-term outcomes.32

3.4 Health first established guidance (the Performance Measurement and Reporting Framework) on developing a program logic model in 2021. The guidance includes contradictory information as it refers to two different sets of program logic elements, one of which is consistent with the Commonwealth Evaluation Policy and Toolkit. In March 2023 Health finalised a set of evaluation fact sheets, one of which contains guidance on developing a program logic and a template that is consistent with the Commonwealth Evaluation Policy and Toolkit.

3.5 Health has established a program logic for the PHN delivery model.

  • The first model was developed in 2015 and was consistent with the Commonwealth Evaluation Policy and Toolkit.
  • The 2015 program logic was replaced in 2018. The 2018 program logic model focused on the core function of PHNs (primary care) and included a separate program logic for each of the seven priority areas (see paragraph 1.2 and Appendix 6).
  • Health established a new program logic in 2023 that consolidated the work PHNs do across core functions and all priority areas (see Appendix 6).

3.6 The 2018 program logic contains inputs, activities, outputs, and medium- and long-term outcomes that are consistent with the stated objectives (see paragraph 1.1) of the PHN delivery model. Longer-term outcomes are that ‘PHNs support local primary health care services to be efficient and effective, meeting the needs of patients at risk of poor health outcomes’ and ‘Patients in local region receive the right care in the right place at the right time’.33 There are five ‘intermediate’ outcomes.34

3.7 Although the 2018 core program logic contains most of the expected elements of a program logic, some of its characteristics reduce its effectiveness as a tool for program planning and evaluation. Outcomes are vague. For example, terms such as ‘quality care’ could be more defined. Three of the five ‘outcomes’ more closely resemble outputs.35 Anticipated outcomes are not supported by more specific timeframes for their achievement, other than being described as ‘intermediate’ and ‘longer-term’. Outcomes in the seven 2018 priority area program logics are more specific and describe changes in knowledge, skills and behaviour.36 On the basis of the program logics alone, the logical link between outputs and outcomes is not always clear.37 However, the 2018 program logic was supplemented by a ‘theory of change’, which provided context for the causal relationships between the elements.38

3.8 The 2023 program logic includes ‘objectives’ at the start of the logic chain, which the guidance defines as the ‘intended result of the work’. The 2023 program logic introduces five new ‘objectives’ for the PHN delivery model: ‘improved quality of care and population outcomes’, ‘enhanced patient experience of care’, ‘sustainable cost’, ‘improved provider experience’ and ‘advancing health equity’. Outputs and impacts are included but these are expressed as performance measures39, and as such are more narrowly formulated than in the 2018 program logic, and the logical connection with the five new objectives is unclear. The 2023 program logic is not supported by a ‘theory of change’.

3.9 Neither the 2015, 2018 nor 2023 PHN program logic models considered potential unintended outcomes or adverse consequences from the PHN delivery model.

Opportunity for improvement

3.10 Health could improve the 2023 program logic by aligning it with Australian Government guidance and including clear and causally linked inputs, activities, outputs, and short-, medium- and long-term outcomes (which describe changes in knowledge, skills and behaviour) that are aligned to the stated objectives of the PHN delivery model.

Are there fit-for-purpose performance measures?

With the exception of measures related to the mental health priority area, the majority of performance measures included in PHN grant agreements check compliance with grant agreement reporting and other requirements, and do not assess the quality of outputs or the achievement of outcomes. Performance measures contained in mental health grant agreements are useful and support the adoption of an ‘outcomes orientation’, as recommended by the Commonwealth Grants Rules and Guidelines. In addition to measures in grant agreements, there is a performance framework which sets out 39 additional PHN ‘performance’ measures and 15 ‘organisational’ measures. These measures are used to monitor individual PHNs’ performance and, when aggregated, to conclude on the performance of the delivery model as a whole. Of the 39 performance measures, 20 measures are assessing something other than compliance with grant agreement requirements. These measures provide a basis for assessment over time; represent a mix of qualitative, quantitative, effectiveness and output measures; and mainly have targets. However, the relevance to stated PHN delivery model objectives is sometimes unclear, and there is a reliance on unverified PHN-supplied data for some measures. Health planned to improve PHN performance measures by July 2023, however this was delayed to June 2024.

3.11 When transitioning from Medicare Locals to PHNs in 2014, Health had advised the Minister for Health that PHNs would have a small number of key performance indicators that they would be expected to achieve, with ‘a move away from counting inputs and outputs to an environment of achieving outcomes’. In addition to having an outcomes focus, the performance indicator approach was meant to minimise administrative burden on PHNs such that ‘funding to frontline services will be maximised’.

3.12 Performance measures for Primary Health Networks are found in grant agreements, and in a separate performance framework.

Performance measures in grant agreements

3.13 According to the CGRGs, officials administering grants should establish appropriate performance measures on which to evaluate grants. The CGRGs note that:

Performance reporting requirements and other information sought from grantees are key inputs used by officials in evaluating whether outcomes have been achieved and whether a particular grant activity achieved value with relevant money. In developing the performance reporting and information requirements for particular grant opportunities and grantees, officials should balance the amount of information sought and the associated costs to grantees of collecting and collating such information, against the obligation to perform due diligence in relation to grant evaluation processes. Officials, in close consultation with government and non-government stakeholders, should design performance information to show the extent to which grant activities contribute to government outcomes, as well as producing outputs.40

3.14 Grant agreements with PHNs include a total of 52 performance measures (see Appendix 7) associated with: activities in the core schedule to the grant agreement41; activities related to the aged care42, alcohol and other drugs43, and mental health priority areas44; and Aboriginal and Torres Strait Islander health activities.45

3.15 The 29 grant agreement performance measures in the core, aged care, alcohol and other drugs and Indigenous health grant agreements are primarily (82 per cent) measuring compliance with grant agreement requirements, such as reporting requirements). As grant agreement management measures, they provide little information upon which to base future decisions for designing, continuing or concluding grant opportunities46, and they provide little information about whether outputs and outcomes have been achieved.

3.16 The 23 measures included in the mental health grant agreements evenly comprise measures for grant agreement management, and measures that track performance against grant activity outputs and outcomes. Mental health grant agreement performance measures are supported by data sources, and they represent a mix of input, output and outcome measures. Of the 23 mental health performance measures, six have clear targets. The mental health grant agreement is also the only one to include measures of efficiency related to the cost and timeliness of services.

Performance measures in a performance framework

Performance measure framework

3.17 In a 2016 PHN Performance Framework, Health established a set of measures for PHNs that were intended to assist with performance monitoring of individual PHNs as well as evaluation of the PHN delivery model as a whole. The measures comprised the following.

  • ‘Organisational’ measures — 28 measures to monitor the establishment of PHNs and the development of their key organisational capabilities. These related to governance, financial management, stakeholder management, supporting general practice, delivery of contracted services and direct services.
  • ‘National headline’ measures — Four ‘measures that reflect the Australian Government priorities’. These outcome measures comprised: potentially preventable hospitalisations; childhood immunisation rates; cancer screening rates (cervical, breast, bowel); and mental health treatment rates (including for children and adolescents).
  • ‘Local’ measures — Approximately five to 15 for each PHN, to measure the performance of individual PHNs at addressing local health priorities. Local indicators were developed by PHNs, included in activity work plans (see paragraph 2.8), and approved by Health. The 2016 PHN Performance Framework contained guidance for PHNs on developing local indicators and there was variation across PHNs in the number, topic and quality of local indicators.

3.18 A July 2018 EY evaluation of the PHN delivery model (see Appendix 4) identified an opportunity to improve performance management. The evaluation report noted that ‘the performance indicators that had been developed at a national, local (of which there were too many) and organisational level are not as useful as they could be’. Concerns were raised about the failure of the 2016 Performance Framework to include new programs, ‘inconsistent and fragmented reporting’, PHN reporting ‘burden’, and a lack of clarity over ‘whether data collection for performance monitoring was intended to measure compliance, accountability and/or quality improvement’.

3.19 A 2018 PHN Performance and Quality Framework (PHN Performance Framework) was intended to ‘move the Program towards outcome-based reporting’ and to measure the PHN delivery model’s progress towards achieving its objectives. The 2018 PHN Performance Framework is publicly available on Health’s website.47 It included nationally consistent performance measures for the PHN delivery model for the first time since the establishment of PHNs in 2015–16. The measures provide the basis for 12-month PHN performance reports, discussed in the context of line two assurance activities at Table 2.1, and when aggregated underpin annual reports on the overarching PHN delivery model. The measures comprised the following.

  • ‘Performance’ measures — 39 measures, comprising 13 related to the core functions of the PHN delivery model (labelled ‘P’ in Appendix 8) and 26 related to the priority areas (See paragraph 1.2).
  • ‘Organisational’ measures — 15 measures based on those established in the 2016 PHN Performance Framework.
  • ‘Aspirational’ measures — 13 measures that were to be developed after the implementation of the PHN Performance Framework. These measures were meant to relate to cultural appropriateness of health services, digital health services, aged care, and long-term impacts of improvements to services. As at December 2023 the ‘aspirational indicators’ had not been developed.

3.20 These measures were selected as they reflected areas where PHNs could be expected to influence changes.48

3.21 Performance measure data sources comprise: data reported by PHNs to Health; and secondary data from the Australian Institute of Health and Welfare (AIHW), Australian Bureau of Statistics (ABS), Australian Digital Health Agency, and Services Australia (previously Department of Human Services).

Quality of performance measures

3.22 Section 16F of the Public Governance, Performance and Accountability Rule 2014 (PGPA Rule) contains requirements for an entity’s performance measures to be published in its annual report. The Department of Finance’s Resource Management Guide (RMG) 131 Developing good performance information outlines principles to assist entities with meeting the PGPA Rule requirements. Performance measures should be relevant; reliable and verifiable; unbiased; contain a mix of qualitative and quantitative measures; contain a mix of output, efficiency, and effectiveness measures; and provide a basis for assessment over time. Health’s 2021 Performance Measurement and Reporting Framework (see paragraph 3.4) provides guidance on developing performance measures that is aligned with best practice principles in RMG 131.

3.23 Of the 39 performance measures in the 2018 PHN Performance Framework, 1849 provide assurance over the execution of grant agreement requirements and would be better described as grant agreement management measures.50 The PGPA Rule and the RMG 131 do not disallow the use of such measures in external performance reporting, however, grant agreement management measures provide limited information about the achievement of delivery model outcomes (which was the stated purpose of the 2018 PHN Performance and Quality Framework).

3.24 Public submissions to this audit raised concerns with the quality of PHN performance measures, the alignment of measures to PHN activities and objectives, and the administrative burden associated with collecting and reporting against the measures. The ANAO undertook further assessment of the 20 performance measures that were not grant agreement management measures against the principles outlined in the RMG 131 (Table 3.1). The measures partly align with RMG 131 principles, with the main areas of weakness being their relevance to the PHN delivery model objectives, and the risk of inaccuracy and bias due to unverified data provided by PHNs for some measures.

Table 3.1: Assessment of 2018 Primary Health Network performance measures a

Principle

Description of principle

Assessment of 20 performance measures

Overall rating

Relevant to purpose and/or key activities of PHN delivery model

Each performance measure should be aligned with the objectives, functions or role of the PHN delivery model.

The ANAO assessed that 11 out of 20 performance measures do not relate directly to the purposes and key activities of the PHN delivery model. Examples of unrelated measures include ‘rate of general practices receiving payment for after hours services’ and ‘rate of regular uploads to My Health Record’, which PHNs have limited ability to influence. The 2021 baseline maturity assessment found that PHNs believed that the 39 mandatory performance measures did not target the activities required for PHNs to operate efficiently and effectively.

Health included tables in the 2018 PHN Performance Framework that matched the 39 core and priority area performance measures with outcomes expressed in the 2018 program logics. Core outcomes include concepts that the 13 core performance measures do not address. For example, none of the measures supporting the outcome ‘People in the PHN region receive coordinated, culturally appropriate services from local health providers’ address cultural appropriateness. The 2018 PHN Program Performance and Quality Framework states that an indicator on cultural appropriateness is ‘to be developed’. As at December 2023 this measure had not been developed. Health advised the ANAO that because provision of culturally appropriate care was a requirement for accreditation under the Royal Australian College of General Practitioners’ (RACGP’s) Standards for General Practices, and because accreditation is separately tracked through a different performance measure, the cultural appropriateness indicator was not required as it was addressed under the RACGP accreditation process.

Reliable and verifiable

Performance measures should be supported by reliable sources of information.

Secondary data are used for 13 measures. PHNs provide data for seven measures.

The CGRGs state that, in adopting an outcomes orientation, entities should be aware of the ‘common trap’ of framing performance indicators that are reliant upon data provided by the grantee, without validating the grantee’s capacity to produce accurate, reliable and complete data.b Health states in the 2018 PHN Performance Framework that: ‘The amount of information or data that a PHN provides against each indicator will vary depending on the activities it has undertaken to address prioritised needs in that area. For some indicators PHNs may have no or limited input.’

In relation to data quality, there is evidence that individual program officers at Health contact PHNs to verify information on a case-by-case basis. However, systematic data verification is not part of the Assurance Framework activities. The IT system for recording performance information also does not support data verification (see paragraph 3.56).

The ANAO observed problems with annual reporting for the six mental health-related measures that rely on data from the Primary Mental Health Care Minimum Dataset (PMHC MDS) as well as PHN reporting in 12-month performance reports.c The PMHC MDS and 12-month performance reports contain different results and annual reporting of the six indicators is based on an assessment approach that uses both data sources. For the 2021–22 performance reporting cycle, data for these indicators was solely derived from the PMHC MDS, in an effort to improve consistency of data and reporting.

In response to a recommendation of the 2018 EY evaluation (see Appendix 4), Health established a suite of guidance and templates for PHNs on their reporting requirements.d There are inconsistencies between the different guidance documents concerning reporting against the 13 contextual measures, which led to confusion among Health staff and PHNs.

Entities should avoid using vague language like ‘qualitative assessment’ but rather identify a specific methodology (such as accessing results from internal databases, or participant and stakeholder surveys). Precise parameters should be available.

The ANAO assessed whether Health identifies a clear methodology. Two of 20 indicators lacked a clear methodology for calculation, including use of vague language like ‘qualitative assessment’ for one, and in the other qualitative information is divided by a number to obtain a percentage. All performance measures are supported by ‘performance criteria’.

Methodologies should be applied consistently (both by different users at a given point in time and by similar users over different reporting periods). This allows for the processes followed in generating data for each measure to be validated.

Methodologies for 18 of 20 measures involve numerical calculations, which supports repeatability and allows results to be verified.

Qualitative and quantitative measures

It is expected that an entity will have both quantitative and qualitative performance measures to capture the multiple dimensions of performance.

Measures comprise two qualitative and 18 quantitative measures (of which 11 rely on secondary data and seven rely on PHN-supplied data).

Measures of outputs, efficiency and effectiveness

Performance measures should include measures of outputs, efficiency and effectiveness, if appropriate.

There are 15 output and five effectiveness measures. There are no efficiency measures.

Basis for assessment over time

Entity performance should be measured in a consistent manner over a period of time. Generally, trends in performance measured on a consistent basis over time will be more informative than standalone or discontinuous measurement of performance.

All 20 performance measures are supported by consistent data collection timeframes ranging from every three to every 12 months.

Targets

The PGPA Rule subsection 16E(2) specifies that targets should be set for performance measures where it is practicable to set a target. Targets help to establish the desired state, or a threshold for what is considered acceptable performance.

19 performance measures have targets (‘performance criteria’) that state the expected results in each reporting period. The performance criteria for one performance measure qualifies the type of evidence required from PHNs and is not a target.

       

Key:  Not aligned Partially aligned Half aligned Mostly aligned Fully aligned

Note a: This analysis is based on 20 of 39 performance measures that were measuring the achievement of outputs and outcomes rather than grant agreement management measures focused on the execution of grant agreement requirements, such as reporting.

Note b: Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 10.5, p. 27.

Note c: PHNs and commissioned service providers are contractually required to submit data for these indicators to the PMHC MDS. Measures comprise MH1 (‘Rate of regional population receiving PHN commissioned low intensity psychological interventions’), MH2 (‘Rate of regional population receiving PHN commissioned psychological therapies delivered by mental health professionals’), MH3 (‘Rate of regional population receiving PHN commissioned clinical care coordination services for people with severe and complex mental illness’), MH5 (‘Proportion of people referred to PHN commissioned services due to a recent suicide attempt or because they were at risk of suicide followed up within 7 days of referral’), MH6 (‘Outcomes Readiness - Completion rates for clinical outcome measures’), and IH4 (‘Proportion of PHN commissioned mental health services delivered to the regional Aboriginal and Torres Strait Islander population that were culturally appropriate’).

Note d: In addition to the 2018 PHN Performance Framework (which includes an appendix with performance measure specifications), the suite includes a PHN Support Manual, 12-month Performance Report Policy Guidance providing detailed instructions for completing the 12-month report, and an Excel template for the 12-month performance report available in Primary Health Networks Program Electronic Reporting System. The template comprises separate worksheets that capture information relevant to each grant agreement and priority area. A new 12-month performance report guide was released each reporting period.

Source: ANAO analysis of the RMG 131 principles and PHN performance measures from the 2018 PHN Performance Framework (excluding 18 grant agreement management measures).

Recommendation no.2

3.25 Department of Health and Aged Care establish performance measures that are clearly aligned to the Primary Health Networks’ and delivery model’s objectives.

Department of Health and Aged Care response: Agreed.

3.26 The Department of Health and Aged Care is redesigning the Performance and Reporting Framework to include outcomes focussed performance measures which are clearly aligned to the Primary Health Network delivery model objectives.

3.27 RMG 131 notes that performance measures should provide an unbiased basis for assessment of performance. There is a strong reliance on self-reporting from PHNs, especially for the 18 grant agreement management measures. Health does not verify the information provided by PHNs, creating the risk of positive bias in the compliance and performance results (see paragraph 3.56).

Recommendation no.3

3.28 Where there is a reliance on Primary Health Network-supplied data, the Department of Health and Aged Care establish a risk-based methodology for obtaining assurance over the data.

Department of Health and Aged Care response: Agreed.

3.29 The Department of Health and Aged Care is committed to maintaining high quality performance data and will develop a strategy for obtaining assurance over data supplied by Primary Health Networks.

3.30 As noted in Table 2.2, Health planned to revise the 2018 PHN Performance Framework and develop new measures with an expected implementation from 1 July 2023. Health did not develop indicators or commence implementation as scheduled. The new performance framework is to be implemented from 28 June 2024.

Has Health effectively reported on delivery model performance?

Health’s PHN performance reporting is not timely, complete or consistently useful. A publicly available PHN annual report has been produced for three of the five years that the PHN delivery model has been operational and an annual report has been required. As at December 2023, there is no performance information publicly available about 2021–22 or 2022–23. Annual reports have been published years after the time period to which they apply. Health does not report against all the measures it has established, and published reports do not include analysis of financial information, the efficiency of PHNs, comparative analysis across PHNs or information on individual PHN performance. Some measures rely on secondary data that is unavailable. Annual reports partly rely on PHN self-reported performance information that is not always provided. Reporting of results was less informative in the 2019–20 and 2020–21 annual reports than in 2018–19.

3.31 Health publicly reports on PHN delivery model performance through PHN annual reports and its corporate plan.

Primary Health Network annual reports

3.32 There was no requirement in the 2016 PHN Performance Framework for annual reports. The 2018 PHN Performance Framework indicated that Health would produce an annual ‘PHN Program Performance and Quality Report’.51 The reports are meant to provide ‘an overall assessment of the PHN delivery model’s performance in meetings its objectives’.52

3.33 As at December 2023, Health had published three annual performance reports for 2018–1953, 2019–2054 and 2020–2155, and, although required under the 2018 PHN Performance Framework, as at December 2023 there were no annual performance reports for 2021–22 or 2022–23. Annual reporting from 2018–19, 2019–20 and 2020–21 is summarised in Appendix 8.

Timeliness of annual reporting

3.34 Health published the 2018–19 report on 1 January 2020. Publication was aligned with the release of secondary data for the core performance measure P12 (‘Rate of potentially preventable hospitalisations’, see Appendix 8), which Health obtains from the AIHW. Health advised the ANAO that ‘Due to delays in receiving hospital data from states and territories, and having this data cleared by the [AIHW], there is currently a two-year lag from the end of financial year to report publication.’ This means that Health is allowing the availability of data for one measure to drive the timing of reporting on all measures. In addition to the lag associated with the AIHW data, there were further delays in annual reporting in subsequent years. The 2019–20 Annual Report was published on 23 February 2023 and the 2020–21 Annual Report was published on 1 June 2023. The delays in annual reporting are not consistent with public transparency over PHN performance.

Recommendation no.4

3.35 The Department of Health and Aged Care report on Primary Health Network program performance as soon as practicable following the year to which the majority of the performance information relates.

Department of Health and Aged Care response: Agreed.

3.36 The Department of Health and Aged Care is developing new performance indicators to measure the effectiveness of the PHN Program and leveraging data, including secondary data to improve the timeliness and completeness of reporting. The Department is also redesigning the annual report to make reporting more informative, timely and useful.

Completeness of annual reporting
Reporting of established performance measures

3.37 The 2018 PHN Performance Framework states that the annual report will include the assessment of all 54 delivery model measures (see paragraph 3.19), against an outcome theme, ‘to provide an overall assessment of the PHN program’s performance in meeting its objectives’.56 Annual reporting does not include and is not meant to include grant agreement performance measures.

  • ‘Organisational’ measures — The three annual reports include reporting of all 15 ‘organisational’ measures. From 1 July 2021 Health no longer required PHNs to report against 13 out of 15 ‘organisational’ measures in each PHN’s 12-month performance report. Health advised that this decision was based on the outcomes of the 2021 baseline maturity assessments conducted by PricewaterhouseCoopers (PwC) (see paragraph 2.17) The two measures that PHNs were still required to report against involved activity variance reporting and the submission of key documents such as financial statements.
  • ‘Performance’ measures — Of the 39 core and priority area performance measures, 36 were reported in 2018–1957, and 35 were reported in 2019–2058 and 2020–21.59

3.38 All measures not reported on were based on secondary data. Excluded measures varied from year to year, however two measures were not reported in any of the three published reports. Data for one of these measures (P11 ‘Rate of discharge summaries uploaded to My Health Record’) is not available at all. Health did not provide an explanation in any of the three annual reports for why the second measure (P13 ‘Number of health professionals available’) was not reported. For the other unreported measures, the 2018–19 Annual Report stated that ‘information was not collected for 2018–19’ for one measure. The 2019–20 and 2020–21 annual reports provided no information as to why measures were not reported; however, Health advised the ANAO that the data for these indicators was unavailable for the relevant reporting period.

3.39 Health states in the 2018 PHN Performance Framework that it intends to disaggregate three indicators by Aboriginal and Torres Strait Islander status.60 This disaggregation did not appear in any of the three annual reports. In 2018–19 Health reported that ‘data was unable to be sourced for this disaggregation’ for two indicators and did not comment on the disaggregation for the third indicator.

Opportunity for improvement

3.40 Health could consider the availability of secondary data when developing performance measures and determining timeframes for reporting.

3.41 For measures that rely on PHN-supplied data, there were also issues with the completeness of data.

  • The 2018–19, 2019–20 and 2020–21 annual reports note that reporting requirements were not met for nine measures over the three years, stating for example that ‘Five PHNs did not meet reporting requirements’.61 However, ANAO analysis of PHN reporting showed 14 PHNs did not provide evidence for at least one of 17 performance measures over the three years.
  • In addition to non-reporting against whole measures, PHNs did not provide all required information for the performance indicator DH2 (‘Rate of health care providers using specific digital health systems’) in 2018–19, 2019–20 or 2020–21. Health advised that this was due to errors in the reporting template in the Primary Health Networks Program Electronic Reporting System (see paragraph 3.55).
Efficiency and financial reporting

3.42 PHN delivery model objectives include to improve ‘the efficiency … of health services for people, particularly those at risk of poor health outcomes.’ There is no measurement of, or reporting about, the efficiency of services delivered by individual PHNs or through the PHN delivery model as a whole.

3.43 The 2018–19 Annual Report included the distribution of allocated funding by priority area. This information was not included in 2019–20 and 2020–21 annual reports. ANAO reviewed annual expenditure reporting for all six PHNs included in its targeted sample and found significant underspends (see paragraph 2.29). Annual reports do not discuss this or other aspects of PHNs’ use of grant funding.

Opportunity for improvement

3.44 Health could include relevant financial analysis in its published PHN annual reporting and develop efficiency measures.

Reporting of individual PHN performance

3.45 Reporting on the performance of individual PHNs provides assurance to stakeholders that each PHN is operating effectively. Health states in the 2018 PHN Performance Framework that it will not discuss individual PHN performance in the annual report but that individual examples of best practice in commissioning or achievement may be included with the agreement of the PHN. The rationale for not disclosing individual PHNs’ performance is not stated in the 2018 PHN Performance Framework or elsewhere.

3.46 In accordance with the 2018 PHN Performance Framework, annual performance reports do not include individual PHN performance results or comparative analysis of PHNs. There is some analysis of PHN types in 2020–21 (for example, by state or metropolitan versus regional). Annual reports provide some de-identified information on the number of PHNs meeting or not meeting a performance measure, however underperforming PHNs are not identified. Individual examples of best practice also have not been reported. ANAO analysis showed significant variation in performance between PHNs and that some PHNs’ performance had declined. This is not conveyed in annual reporting.

Recommendation no.5

3.47 The Department of Health and Aged Care publish individual PHNs’ performance data and analysis in annual reports.

Department of Health and Aged Care response: Agreed.

3.48 The Department of Health and Aged Care is redesigning the Performance and Reporting Framework as the first step to ensure performance measures are robust and fit for purpose. As part of this process, the Department will work with Primary Health Networks in the design process for publishing individual performance data that reflects the differing demographics and arrangements across the diverse network.

Clarity and usefulness of annual reporting

3.49 The 2018–19, 2019–20 and 2020–21 annual reports differed in terms of what information was conveyed about each 2018 PHN Performance Framework measure (see Appendix 8). Reporting was of four types, which varied in usefulness to form a view about overall delivery model effectiveness.

  • Type 1 — Reporting provided information about the number of PHNs that ‘met’ the measure plus some further information about the output.
  • Type 2 — Reporting provided information about the output (this form was more commonly used for indicators based on secondary data).
  • Type 3 — Reporting provided the number of PHNs that met the requirement, with no further information about the output.
  • Type 4 — Reporting provided the number of PHNs that provided evidence of meeting the requirement, with no additional information about whether they met the requirement or about the output.

3.50 The way in which each performance measure was reported depended in part on the nature of the measure. Figure 3.1 shows the way each measure was reported in each of the three annual reports. Although the performance measures remained largely the same across the three annual reports, Figure 3.1 shows that the nature of reporting changed.

  • The total number of reported organisational measures remained the same between 2018–19 and 2019–20 and 2020–21, while the total number of reported grant agreement management measures increased by one between 2018–19 and 2019–20 and 2020–21, and the total number of remaining measures decreased by two.
  • Reported grant agreement management measures in 2019–20 and 2020–21 contained less useful information when compared to 2018–19.
  • For the remaining performance measures, there was movement away from reporting information about the output (type 1 and type 2) towards reporting whether the PHN had ‘met’ the measure (type 3 and type 4) in 2019–20 and 2020–21.

Figure 3.1: Types of annual reporting by type of performance measure, 2018–19 to 2020–21

A figure that shows the types of annual reporting for the three years 2018–19 to 2020–21, disaggregated by the three categories of measures comprising organisational measures, grant agreement management measures and other performance measures. For each year and each category of measures, the data is further disaggregated by four types of reporting: Type 1 - Health reported the number of PHNs that met the measure plus information about the input; Type 2 – Health reported information about the output; Type 3

Note: Analysis is based on the number of reported measures in each year for the three types of measures. 2018–19: organisational (15); compliance (17); performance (19). 2019–20 organisational (15); compliance (18); performance (17). 2020–21: organisational (15); compliance (18); performance (17).

Source: ANAO analysis of PHN annual reports.

3.51 The 2019–20 and 2020–21 annual reports also reported some measures in a more ‘narrative’ style that did not identify the measure number. Health advised the ANAO that this was to ‘improve readability’ of the reports, however this approach impedes year on year comparisons of performance data.

Recommendation no.6

3.52 Department of Health and Aged Care publicly report on performance measures:

  1. in compliance with the Primary Health Network performance framework by reporting all performance measures; and
  2. in a way that is consistent with the intended purpose of conveying information about performance in addition to compliance with grant agreement requirements.

Department of Health and Aged Care response: Agreed.

3.53 The Department of Health and Aged Care is redesigning the Performance and Reporting Framework and annual report to ensure that all performance measures are reported against consistently.

Corporate Plan

3.54 Commonwealth entities must report in their annual report on actual results achieved against the performance measures published in their corporate plans and portfolio budget statements. This performance information must be presented as an annual performance statement in accordance with the Public Governance, Performance and Accountability Act 2013 and PGPA Rule. The PHN program contributes to seven programs included in Health’s 2022–23 Corporate Plan (Appendix 9). Auditor-General Report No. 16 Audits of the Annual Performance Statements of Australian Government Entities — 2022–23 includes analysis of Health’s performance statements and associated performance measures.

Has Health established fit-for-purpose IT systems for managing performance reporting?

Health’s IT systems for managing PHN performance reporting are not fully fit-for-purpose. A system was developed in 2019 called the Primary Health Networks Program Electronic Reporting System (PPERS). Health has a range of guidance to support PPERS users. PPERS has limited capability to validate data inputs, analyse data and generate compliance and performance reporting.

3.55 PPERS was developed in 2019 to support the PHN funding process and monitoring of PHN delivery. The PPERS User Manual includes guidance for PHNs on uploading and inputting performance information relevant to milestone reports, including needs assessments, activity work plans and 12-month performance reports. Health has also developed guidance for Health officials administering PPERS.

3.56 Health uses data extracts from PHN 12-month performance reporting in PPERS as input into its annual performance assessment of PHNs (discussed at Table 2.1), and in PHN annual reports (discussed at paragraphs 3.33 to 3.38). PPERS characteristics limit its usefulness for PHN performance data entry, verification and reporting.

  • Data entry — Performance information is captured primarily in free text fields, which limits the analytical usefulness of the data and decreases the ability of Health to verify PHN-supplied data. The ANAO identified data quality issues relating to inconsistency in the reporting of individual PHN names, lack of version control and inconsistent naming of schedules/funding streams.
  • Data verification — There is guidance to PHNs on what to include in free-text fields, however the lack of defined system parameters for data means that verification at the data entry stage is not possible for some measures. The range of data captured does not facilitate data verification. For example, for the Indigenous Health indicator IH2 (‘Types of organisations delivering [Integrated Team Care] services’), PHNs are required to provide the number and type of organisations delivering services, but not the names of the organisations, which could be useful to a verification process. Using business rules provided by Health, the ANAO could not verify the data in PPERS.
  • Reporting — PPERS does not have the capability to generate reports. Performance information is exported from PPERS for further analysis in other systems. While Health has established processes for extracting data from PPERS for further analysis, this is not a simple or timely process.

3.57 The ANAO received 31 submissions to the audit from individuals, PHNs, industry groups, professional associations and health service providers. PPERS and associated data issues were raised as an issue in 26 per cent of submissions. Since late 2019 Health has monitored issues with PPERS for PHN and Health users of the system. These include: problems with access and permissions; and uploading and revising needs assessments, activity work plans and financial reporting.

3.58 In May 2023 the division responsible for PPERS developed a business case seeking capital funding of just under $1 million for improvements to PPERS. Health approved funding to fix ‘fundamental bugs’ in the PPERS system to ‘enable processing to continue’ in May 2023. Health advised that no further enhancements were funded.

Recommendation no.7

3.59 The Department of Health and Aged Care implement a fit-for-purpose IT system for administering Primary Health Networks that supports the accurate capture and reporting of compliance and performance information.

Department of Health and Aged Care response: Agreed in principle.

3.60 The Department of Health and Aged Care will explore options to design and develop a fit for purpose IT system for administering the Primary Health Network Program. The recommendation is agreed in principle as implementation is contingent on funding decisions of government.

3.61 Beta testing for the Primary Health Operational and Commissioning Unified System (PHOCUS) was completed in September 2023. PHOCUS was developed by the WA Primary Health Alliance, the provider for the three Western Australian PHNs. Health advised the ANAO that PHOCUS is intended to replace the performance indicator reporting component of PPERS, but is not intended to replace other PPERS functions, such as financial reporting and milestone reporting.

4. Evaluation

Areas examined

This chapter examines whether the Department of Health and Aged Care (Health) has demonstrated that the Primary Health Network (PHN) delivery model is meeting its objectives.

Conclusion

Health has not demonstrated that the PHN delivery model is achieving its objectives. Health had no evaluation plans for the PHN delivery model after 2018. Health has not conducted a comprehensive delivery model evaluation. A 2018 early implementation evaluation was inconclusive about the achievement of objectives at that early stage in the delivery model’s implementation. A lack of baseline and relevant performance data impedes understanding of whether the delivery model has met its objectives.

Areas for improvement

The ANAO made one recommendation to establish a plan for, and conduct, an evaluation of the PHN delivery model.

4.1 The Commonwealth Evaluation Policy (Evaluation Policy) states that Australian Government entities are expected to deliver support and services for Australians by setting clear objectives and consistently measuring progress towards achieving those objectives.62 The PHN delivery model aims to: (1) improve the efficiency of health services for people, particularly those at risk of poor health outcomes; (2) improve the effectiveness of health services for people, particularly those at risk of poor health outcomes; (3) improve the coordination of health services; (4) increase access for people; and (5) increase quality support for people.63

4.2 Evaluation is supported by an evaluative culture and good governance, including a plan to conduct evaluation activities.64 The ANAO examined whether Health has a fit-for-purpose evaluation plan for the PHN delivery model, and whether it has conducted evaluation of the PHN delivery model in accordance with its plan. The ANAO also examined the existing evidence to determine whether the PHN delivery model can be said to be achieving its objectives.

Has Health appropriately evaluated the Primary Health Network delivery model?

A fit-for-purpose evaluation plan for the PHN delivery model was developed in 2015, which planned for evaluation activities up to December 2017. Despite a departmental evaluation strategy which indicates the importance of ongoing program monitoring and evaluation for new and high value policies or programs such as the PHN delivery model, there was no evaluation plan for the PHN delivery model after December 2017. A 2018 evaluation focused on the early implementation of the PHN delivery model. The PHN delivery model has not been comprehensively evaluated to determine whether it is meeting its objectives. There have been 23 evaluations of pilot programs and time limited grants provided through PHNs, however Health has not undertaken a consolidated review of findings from these evaluations to reach a conclusion about the effectiveness of the overall delivery model.

Evaluation planning

4.3 The Commonwealth Evaluation Policy65 and Department of Finance (Finance) guidance on conducting evaluation66 state that plans to conduct fit-for-purpose evaluation activities should be established before beginning any program or activity; and should include timeframes, resources, baseline data and performance information. As part of a tool kit of evaluation resources, Finance produces a template for an evaluation plan.

4.4 Health introduced a departmental Evaluation Strategy in 2017 to strengthen policy and program evaluation practice and increase the use of evaluation evidence for decision-making, planning and reporting. The departmental Evaluation Strategy applied to the period 2016 to 2019.67 It noted that ‘all policies and programs should be evaluated on a systematic basis to ensure they are appropriate, effective and efficient’. In its guidance for risk-based evaluation planning decisions, a ‘tier one’ program is described as one that is of strategic significance; involving a high level of funding, risk and public interest; or is new. The Evaluation Strategy indicates that a program with these characteristics indicatively uses external evaluation, multiple evaluation points, high quantitative data requirements, comprehensive qualitative evaluation and multiple evaluation reports.

4.5 The 2016–2019 Evaluation Strategy included a model for evaluating policies or programs in four lifecycle phases comprising development, planning, implementation, and transition. The Evaluation Strategy noted that in the planning phase it is essential to commence planning for evaluation, and that in the implementation phase (defined as the period in which a policy or program is operational), ongoing monitoring and evaluation is needed to determine whether the program is working as expected and is cost-effective. Health’s 2016–2019 Evaluation Strategy indicated that divisions are responsible for funding and conducting evaluations. Health internal guidance on grants administration states that Health’s Policy Division is responsible for the evaluation of grant programs.

4.6 In accordance with the principles later expressed in the 2016–2019 Evaluation Strategy, in 2015 Health had contracted EY to develop a PHN Evaluation Framework and to evaluate the PHN delivery model. The PHN Evaluation Framework was fit-for-purpose and included:

  • a statement of the PHN delivery model’s objectives68;
  • a timeline for the evaluation to be completed between July 2015 and December 2017;
  • plans for obtaining ethics approval;
  • a detailed data collection and methodological approach that specifies the use of existing data wherever possible;
  • evaluation questions aligned to the objectives of the delivery model, including to what extent PHNs are ‘fit-for-purpose’ and whether PHNs have increased the efficiency and effectiveness of medical services for patients, particularly those at risk of poor health outcomes;
  • a mapping of evaluation questions to data sources;
  • a consideration of baseline data 69; and
  • a risk analysis.

4.7 The PHN Evaluation Framework incorporated evaluation activities up to December 2017. Health did not develop a PHN evaluation plan that applied after this date and at December 2023 does not have a plan for future evaluation activities. A program logic model was updated in 2023.

Evaluation activities

4.8 Health published the report from the 2016 to 2017 EY evaluation in July 2018. The evaluation report, which is publicly available on Health’s website70, described its aims as: ‘(1) assessing how the PHN Program was implemented in local contexts; (2) understanding the extent to which the PHN Program had an impact and achieved its intended objectives; and (3) informing the ongoing implementation of the PHN Program.’ The ANAO did not assess the 2018 evaluation to determine if it was conducted in accordance with the PHN Evaluation Framework or its overall quality. The evaluation’s conclusions, main findings and recommendations are described in Table 4.1 and Appendix 4. There are no other evaluation reports of the PHN delivery model on Health’s website, and no delivery model evaluation was conducted after July 2018.

4.9 Although there was no evaluation of the delivery model after 2018, between October 2015 and March 2023, Health evaluated 23 pilots, trial measures and other grant-funded activities delivered by some or all PHNs (Appendix 10). Health has not analysed these to determine if there are any recurring themes or consistent findings relevant to the achievement of PHN delivery model objectives.

Recommendation no.8

4.10 Department of Health and Aged Care:

  1. develop an evaluation plan for the Primary Health Network delivery model; and
  2. evaluate the Primary Health Network delivery model to determine whether it is achieving its objectives.

Department of Health and Aged Care response: Agreed.

4.11 The Department of Health and Aged Care is committed to demonstrating the performance of the Primary Health Network Program. The Department will develop an evaluation plan for the Primary Health Network delivery model and conduct an evaluation to determine whether it is achieving its objectives.

Has the delivery model met its objectives?

An early implementation evaluation reported in 2018 was inconclusive about whether the PHN delivery model was achieving its objectives but stated there were early indications of progress towards achieving objectives. Although there has been some improvement between 2018–19 and 2020–21 in PHNs’ average performance against some measures, no baseline data for these measures and the lack of relevant performance measures means that it is not possible to conclude if the PHN delivery model has met its objectives.

Evaluation findings

4.12 The main overall findings of the 2018 evaluation are shown in Table 4.1. Against the objectives of increasing the efficiency and effectiveness of health services for patients, particularly those at risk of poor health outcomes, the evaluation concluded that ‘there were indications of progress’. Against the objectives of improvements to the coordination of care to ensure patients receive the right care in the right place at the right time, the evaluation concluded that there had been ’some progress in building the strong foundations required to improve regional coordination of care …’.

Table 4.1: Summary of 2018 PHN delivery model evaluation findings

Evaluation question

Summary of findings

To what extent are PHN functions fit-for-purpose?

  • PHNs were ‘on the way to cementing themselves as the pre-eminent primary health care organisations … ‘.
  • Most interviewed stakeholders felt that the overarching delivery model objectives were sound and that PHNs had a critical role in helping to deliver sustainable, integrated and safe primary health care in Australia.
  • PHNs were able to take a more agile and community inclusive approach due to providing a health system infrastructure that was separate from Health. The organisational design of PHNs was appropriate for achieving regional objectives.
  • While some PHNs were still evolving their governance arrangements, substantial progress had been made. PHNs were working collaboratively with each other.
  • PHNs were well-aligned with other primary health care reforms and the broader policy context, however one of the key challenges for PHNs would be encouraging Local Hospital Networks (LHNs), state and territory health departments and other agencies to engage in regional planning more actively, and support integrated service delivery at the local level.
  • The evidence base for the benefits of commissioning was mixed. Observed impacts were highly dependent on context and the understanding of commissioning, as it relates to PHNs, was evolving.
  • Health needed to develop a program framework that was set within the broader national primary health care strategy and clearly set out the longer-term strategy of the PHN delivery model, to assist PHNs and stakeholders to understand how the model was expected to operate and its intent.

Have PHNs increased the efficiency and effectiveness of medical services for patients, particularly those at risk of poor health outcomes?

  • PHNs’ use of and access to timely and granular data has been limited.
  • Given the timing of the evaluation, many of the data items referenced in the PHN program logic required for measuring outcomes were still in development. These included: consistent and useable measures of PHN performance relating to local and organisational outcomes; and national data for measuring and attributing changes in health outcomes.
  • The delivery model was showing ‘indications of progress in achieving early outcomes’.
  • The PHN commissioning model was developmental; PHNs were learning as they went and evolving their approaches, some faster than others.
  • PHNs had developed partnerships with LHNs to support better integration of services, but much of this relied on the ‘goodwill’ of individuals rather than being systematically embedded in program design.
  • Effective consumer engagement was an area for development.
  • PHNs’ influence on the efficiency and effectiveness of medical services was indirect through the provision of resource-intensive practice support.
  • The implementation of the 2018 PHN Program Performance and Quality Framework provided an opportunity to systematically measure the delivery model’s efficiency and effectiveness (based on a set of agreed outcomes and accountabilities) and thus inform its ongoing development. The successful collection of data both locally and at scale would underpin this.

Have PHNs led to improvements to the coordination of care to ensure patients receive the right care, in the right place, at the right time?

  • PHNs had ‘made some progress in building the strong foundations required to improve regional coordination of care to benefit patients and the health system as a whole’.
  • PHNs had established the ‘building blocks’ to effective stakeholder relationships.
  • PHNs had provided opportunities for Australian governments to implement policies supporting greater integrated care (particularly in New South Wales and Queensland).
  • PHNs needed to improve engagement with service providers, especially the Aboriginal Community Controlled Health Sector.

How are the information, advice and support needs of PHNs identified in relation to the national support function and how effective has Health been in providing support?

  • It was challenging for Health to balance its roles as funder, centralised support and performance manager for the PHN delivery model. Nonetheless, Health had built a generally trusting and transparent relationship with PHNs.
  • Support provided to PHNs was reactive at times and it needed to be more proactive, including identifying PHNs which needed a stronger focus on performance improvement.
  • Health needed to consolidate learnings from ongoing evaluations involving PHNs, enhance program design and implementation, reduce duplication and improve operational management.
   

Source: ANAO summary of 2018 EY evaluation of the PHN delivery model.

4.13 The evaluation provided 24 ‘opportunities for future development’ of the PHN funding model, across the themes of: governance; external collaboration and stakeholder engagement; commissioning; performance management; program guidance; departmental operations; PHN operations; funding model; national support function; Aboriginal and Torres Strait Islander health; and the use of data (see Appendix 4). The implementation of recommendations is tracked by Health. All but four recommendations were listed as complete as at July 2023. The unimplemented recommendations include to improve alignment between monitoring and evaluation processes.

Improved efficiency of health services

4.14 Health did not establish measures of efficiency in its 2018–19 performance framework. Reporting in the 2018–19, 2019–20 and 2020–21 annual reports did not demonstrate the efficiency of health services delivered through the PHN delivery model, either for users of those services or for the Australian taxpayer. Health’s PHN annual reporting does not include financial information such as the amount of funds committed to and expended by PHNs, and it does not include analysis of the financial performance of PHNs or the delivery model.

Improved effectiveness of health services

4.15 As discussed in Table 3.1, a lack of alignment between Health’s performance indicators and the PHN delivery model’s objectives impedes Health’s ability to demonstrate whether PHNs are meeting the key objective of improving the effectiveness of health services for people, particularly those at risk of poor outcomes.

4.16 Although the 2018–19 PHN Annual Report described the performance results for 10 measures as ‘baseline’ results, this baseline was not used for trend reporting. For two measures in 2019–20 and eight measures in 2020–21, annual reports included a brief comparison with the previous year’s result.

4.17 The ANAO examined trends for 13 of 14 quantitative measures71 based on PHN-reported data to identify any trends in compliance with grant agreement requirements and performance over the three years for which PHN performance information is publicly available (see Table 4.2 and Appendix 11 for a full description of the measures). Between 2018–19 and 2020–21, there was an upward trend in average PHN-reported performance for nine indicators72 (see Table 4.2). A lack of baseline data makes it difficult to determine if effectiveness has increased since the PHN delivery model was implemented.

Table 4.2: 13 performance measures, 2018–19 to 2020–21

Indicator

2018–19

2019–20

2020–21

Trend

AOD1

98%

100%

100%

Trend 01

DH1

100%

100%

100%

Trend 02

DH2

75%

88%

92%

Trend 03

IH1

962136

787617

991115

Trend 04

IH2

2

2

2

Trend 05

IH4

61%

77%

88%

Trend 06

MH1

.15

.15

.20

Trend 07

MH2

.36

.41

.49

Trend 08

MH3

.11

.10

.15

Trend 09

MH5

53%

64%

66%

Trend 10

MH6

29%

41%

43%

Trend 11

P3

79%

80%

82%

Trend 12

P13

1578

1625

1700

Trend 13
         

Source: ANAO analysis of Health data.

Appendices

Appendix 1 Entity response

Page one of the response from DHAC. A summary of the response can be found in the summary and recommendations chapter.
Page one of the response from DHAC. A summary of the response can be found in the summary and recommendations chapter.

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

  • strengthening governance arrangements;
  • introducing or revising policies, strategies, guidelines or administrative processes; and
  • initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether all have been appropriately implemented.

  • In February 2023 Health published the Primary Health Network (PHN) 2019–20 Annual Report (paragraph 3.34).
  • In March 2023 Health developed a risk management plan for the PHN delivery model (paragraph 2.14).
  • In May 2023 a business case to maintain and enhance the PHN Program Electronic Reporting System was submitted, and maintenance was funded (paragraph 3.58).
  • In June 2023 Health published the PHN 2020–21 Annual Report (paragraph 3.34).
  • In September 2023 Health established a ‘PHN Service Offer’, which described roles and responsibilities across Health for the administration of PHNs (paragraph 2.7).

Appendix 3 Grant funding commitments to Primary Health Networks

Table A.1: Grant commitments to Primary Health Networks, 2015–16 to 2023–24 (as at August 2023)

Primary Health Network

Core

($)

Mental health

($)

Alcohol and other drug services

($)

Aboriginal and Torres Strait Islander health

($)

Aged care

($)

Pilots and other programsa

($)

Total

($)

Australian Capital Territory

40,124,728

75,593,748

18,698,152

6,129,858

4,206,577

33,348,118

178,101,181

Adelaide

86,540,727

219,870,923

46,256,514

17,273,923

10,896,139

73,327,729

454,165,955

Brisbane North

71,049,364

144,289,531

47,065,962

15,894,861

8,403,480

98,312,993

385,016,191

Brisbane South

76,645,371

164,337,478

28,978,692

17,388,677

9,269,840

93,283,534

389,903,592

Central and Eastern Sydney

92,435,785

171,184,234

37,077,310

16,877,148

11,926,391

106,253,202

435,754,070

Central Queensland, Wide Bay and Sunshine Coast

100,356,435

173,938,294

24,367,067

27,086,203

9,610,745

96,606,424

431,965,167

Country South Australia

161,663,814

172,174,900

25,910,623

22,989,242

7,914,867

72,688,521

463,341,967

Country Western Australia WA

156,171,570

225,404,149

33,733,413

48,891,134

7,205,474

67,592,458

538,998,198

Darling Downs and West Moreton

76,854,262

123,518,329

20,038,372

21,003,987

7,663,374

41,145,442

290,223,765

Eastern Melbourne

91,481,304

171,762,866

32,560,154

7,142,767

11,265,527

119,869,274

434,081,893

Gippsland

61,958,484

70,418,104

10,072,459

6,071,826

5,243,775

40,318,595

194,083,242

Gold Coast

53,444,800

112,908,787

16,905,524

5,940,065

5,718,834

49,163,161

244,081,171

Hunter New England and Central Coast

124,975,945

241,050,690

33,501,354

54,870,750

13,139,517

113,967,060

581,505,316

Murray

126,678,514

148,520,465

19,872,087

16,162,452

8,901,000

65,692,349

385,826,867

Murrumbidgee

57,291,651

70,898,611

10,726,058

11,453,245

4,860,627

39,963,305

195,193,497

Nepean Blue Mountains

43,066,505

95,123,194

13,326,857

10,632,837

4,637,121

44,990,007

211,776,521

North Coast

90,110,874

155,165,071

25,891,834

24,631,117

7,605,858

51,039,815

354,444,569

North Western Melbourne

98,504,241

202,103,699

50,391,328

11,131,298

13,573,709

76,423,430

452,127,705

Northern Queensland

133,148,665

245,915,060

41,991,720

64,733,169

8,511,933

78,185,707

572,486,253

Northern Sydney

65,635,577

87,391,347

14,957,405

1,877,622

7,673,998

35,004,931

212,540,879

Northern Territory

96,113,822

200,924,154

44,468,369

89,200,614

4,492,690

92,625,307

527,824,956

Perth North

70,728,227

214,848,179

32,573,740

16,479,893

8,239,297

77,918,673

420,788,008

Perth South

77,250,695

147,843,045

22,869,270

28,480,720

8,431,371

60,197,297

345,072,398

South Eastern Melbourne

92,688,988

282,852,209

33,743,931

7,850,192

11,932,417

92,559,804

521,627,541

South Eastern New South Wales (NSW)

82,842,632

133,997,055

18,397,521

18,491,549

7,361,812

60,920,865

322,011,433

South Western Sydney

74,184,114

137,903,451

24,941,329

13,182,095

9,232,105

49,166,269

308,609,363

Tasmania

116,312,384

160,396,545

38,423,689

23,463,641

7,535,263

88,145,077

434,276,598

Western NSW

103,630,144

134,038,750

17,778,606

29,379,255

5,840,272

43,276,931

333,943,958

Western Queensland

99,640,134

56,829,893

13,272,251

22,910,934

3,067,197

20,382,762

216,103,171

Western Sydney

73,258,801

191,719,107

24,269,454

17,142,110

8,182,360

70,117,691

384,689,524

Western Victoria

119,123,356

144,296,224

25,044,655

6,952,728

8,527,409

51,894,217

355,838,589

Total

2,813,911,913

4,877,218,094

848,105,701

681,715,913

251,070,972

2,104,380,947

11,576,403,538

               

Note a: Pilots and other programs comprise: Community Health and Hospitals Program; after hours support; Commonwealth psychological support; continuity of support; mental health – bushfire support; National Mental Health and Suicide Prevention Agreement and Bilateral Primary Health Network Program; National psychosocial support measure; Partners in Recovery, and urgent care clinics.

Source: ANAO analysis of the Department of Health and Aged Care’s master financial spreadsheet.

Appendix 4 Delivery model reviews and internal audits, July 2018 to December 2023

Table A.2: Delivery model reviews and internal audits, July 2018 to December 2023

Review / audit

Scope

Key findings

Recommendations

Status according to Department of Health and Aged Care a

July 2018 —Evaluation of the PHN Program

(EY)

To what extent are PHN functions fit-for-purpose?

Has the PHN Program increased the efficiency and

effectiveness of medical services for patients, particularly those at risk of poor health outcomes?

Has the PHN Program improved the coordination of care to ensure patients receive the right care, in the right place, at the right time?

How are the information, advice and support needs of PHNs identified in relation to the national support function and how effective has the Department been in providing support?

  • See Table 4.1
  • Governance — two recommendations

  • External collaboration and stakeholder engagement — four recommendations

  • Commissioning — three recommendations

  • Performance management — two recommendations

  • Program guidance — one recommendation

  • Departmental operations — one recommendation

  • PHN delivery model operations — two recommendations

  • Funding model — two recommendations

  • National support function — one recommendation

  • Aboriginal and Torres Strait Islander health — four recommendations

  • Use of data in the PHN program — two recommendations

July 2018 —

Internal audit of administration and performance of PHN’s commissioning (Protiviti)

Considering and defining the structure, roles and responsibilities relevant to supporting the PHN Commissioning process.

Assessment of the PHN Branch’s governance arrangements which support the PHN Commissioning process, including oversight committees and stakeholder engagement functions.

Assessment of the risk management framework used by the PHN Branch that has been adopted for overseeing PHN Commissioning activities. Review of risk registers to assess the completeness, relevance, and accuracy of risks identified for commissioning.

Documenting and assessing the processes and controls relevant to commissioning activities, in particular ongoing monitoring and evaluation arrangements.

Documenting and assessing assurance type activities performed by the PHN Branch over commissioning such as compliance monitoring.

Departmental Governance Arrangements are not well defined and do not support the complex decision-making and accountability measures needed to manage the PHN Program.

Standard grants management approach not suitable for such a large and complex program.

Improvement of the performance framework to reflect Departmental requirements. There are gaps within the performance framework, specifically the reporting and quality assurance processes, which hinder the Department’s ability to monitor performance of PHN Commissioning.

There has been limited formal assessment of the information needs of internal and external stakeholders, and communication resources have been developed reactively.

  • Define roles and responsibilities for the PHN Program. This should incorporate all key internal stakeholders and confirm that the governance arrangements adequately support the management and oversight of the PHN Program.

  • Develop guidance/instructional material to support the implementation of processes for the management and oversight activities of the PHN Program.

  • Undertake oversight and management of the PHN Program to enable a ‘proactive’ phase of maturity. This should include clarifying and documenting: governance arrangements; reporting expectations and templates; monitoring and compliance processes, procedures and checklists; system support; and stakeholder engagement.

  • Develop a fit-for-purpose Performance and Quality Framework to address Department, program and policy performance information to facilitate oversight, monitoring and evaluation of the PHN Program.

  • Identify and implement activities to better support efficient performance data collection and information sharing. This may include use of electronic systems.

  • Develop a quality assurance framework to improve financial and performance accountability, including consideration of an audit program. This should be complementary to the new Performance and Quality Framework.

  • Health should develop a stakeholder communication and engagement management plan based on the internal and external needs assessments.

June 2020 — Operational expenditure review (KPMG)

The review aimed to identify the actual or ‘true’ operational expenditure for PHNs versus what is currently being reported and determine unique and common cost drivers across PHNs.

Data findings included that 74 per cent of PHNs responded to the survey (26 per cent did not respond) and the quality of data from those that did participate varied in detail and completeness. The incomplete data set meant that it was not possible to generalise findings and consider implications for sub-groups of PHNs.

Several PHNs are using operational funding from their schedules to support core business costs such as rent, utilities and IT costs.

Sixty per cent or more of operational spending was spent by PHNs on salaries and leave provisions, health systems improvement (not further defined) and other costs (not defined)

The current reporting framework does not provide the Department with granular information regarding operational expenditure.

Reported expenditure is different to the actual expenditure for Board costs, salaries, rent and utilities.

  • Review and agree on definition of ‘operational expenditure’ and update the PHN Program Funding Guidelines and all schedules to reflect this agreed definition.

  • Investigate basis for operational funding caps for each schedule, circulate to departmental stakeholders for transparency.

  • Develop framework for operational funding cap adjustments, circulate with policy areas.

  • Redesign the PHN Program reporting framework, considering the standardisation of the framework for reporting operational expenditure, clear division of funding streams and more granularity for operational expenditure costs.

  • Review operational funding caps for each schedule under the redesigned reporting framework, after collection of a comprehensive data set.

  • Adopt an alternative funding model from the five different models outlined in the report.

  • Communicate changes to definition and application of operational expenditure to PHNs.

June 2020 — Review of PHN grants and reporting (KPMG)

KPMG conducted this review over three phases:

Phase one of the review, completed in January 2020 focussed on the implementation of previous review and audit recommendations.

Phase two involved the mapping of PHN administrative processes.

Phase three of the report relates to the development of recommendations, and supporting a change management plan, to improve PHNs grants and reporting.

The method for this review was to engage with Health officials and PHNs to determine recommendations to move the program forward based on findings from the Phase two report.

PHNs lack effective governance arrangements and clear roles and responsibilities. Due to multiple departmental divisions involved in PHNs, there is no single point of authority or oversight for the end-to-end grants administration process.

Poor program communication both within the Department and externally with PHNs.

A lack of clarity and agreement from Health on the purpose of the PHNs.

The funding schedule process is inefficient and often delayed.

PHNs reporting processes are slow and inefficient, burdensome for PHNs, and are not always of value.

  • Revisit program governance structures.

  • Review and document program roles and responsibilities.

  • Improve communications to promote greater transparency and understanding of the PHN Program.

  • Clarify program purpose.

  • Streamline funding schedules.

  • Simplify program reporting requirements.

  • Review Performance and Quality Framework.

  • Develop data strategy.

  • Conduct initial PHN maturity assessment.

  • Establish risk-based program assurance framework.

June 2022 — PHN governance and efficiency review

(PwC)

Examine whether the governance arrangements placed on PHNs through the Core Funding Agreement support the efficient and effective operation of the PHN program

Examine the implementation of governance requirements, including how local factors have been considered and what processes exist for regular reviews of operations

Examine compliance with Governance Standards required by the Australian Charities and Not-for-profits Commission

Examine PHN commissioning models and their efficiency and effectiveness in commissioning services

Consider the role of PHNs in responding to emergency situations, including drought, bushfires and COVID-19

Identify governance and administrative barriers to the effective and efficient operation of PHNs, and make recommendations for opportunities and enhancements.

PHN governance and administrative arrangements could more effectively support the achievement of PHN program objectives.

Funding arrangements could better enable the effectiveness and efficiency of PHNs.

PHNs have consolidated and refined many commissioning capabilities, however more development is required to support consistency, effectiveness and efficiency in practice.

There are opportunities to improve collaboration with other commissioners, including state and territory health agencies, regional secondary care entities and within the PHN network.

PHNs have several unique attributes and capabilities that position them to support emergency management alongside all levels of Government — more support is required to enable PHNs to play a greater and more sustainable role.

  • Provide greater flexibility in relation to funding and implementing internal PHN governance arrangements.

Implement risk-based assurance.

  • Greater synchronisation and coordination role for PHN Branch.

  • More closely align with the Commonwealth’s key principles for grant administration through consideration of key factors influencing the success of PHN commissioning arrangements.

  • Provide greater forward visibility and certainty to PHNs regarding the anticipated commissioning pipeline.

  • Provide greater flexibility in funding arrangements, where possible.

  • Position the Departmental national support function and updated PHN Performance & Quality Framework in support of PHN efficiency and effectiveness.

  • Leadership by the PHN network to develop resources and guidance which support commissioning effectiveness and efficiency.

  • Enforce accountability for joint planning, collaboration and commissioning with increased flexibility and operational funding (per findings 1 and 2), further resources could be dedicated by PHNs towards joint commissioning and collaboration activities.

  • Flexibility to resource innovation (joint planning, collaboration and commissioning)

  • Increased cross-network collaboration and commissioning led by the PHN network.

  • Articulate the strengths, positioning and role of PHNs to build awareness among emergency management stakeholders.

  • Recognise and authorise PHNs to adopt a formal role in emergency management.

  • Provide fixed and variable funding for PHNs to take a formal role in emergency management.

  • Build the capacity and capability of PHNs to undertake new and formalised roles in emergency management.

September 2022 — Health check of the PHN program (Synergy)

The alignment of performance reporting to the achievement of the PHN Program’s strategic objectives.

The assurance mechanisms in place to provide confidence to stakeholders on the PHN Program’s performance and risk management.

Grant administration processes and their alignment to the relevant grant opportunity guidelines and the CGRGs.

The appropriateness of governance arrangements to provide strategic oversight of the PHN Program.

The administration of PHNs was largely effective, with current processes primarily enabling the PHN delivery model to effectively manage risks and deliver on its objectives. It was identified that PHNs are appropriately managing its governance activities, while opportunities to improve were noted regarding:

limited guidance provided in risk management documentation regarding the processes for undertaking risk identification and assessment.

Absence of evidence to support the value for money considerations and assessment procedures that inform the provision of money through individual grant rounds.

Lack of alignment between the Performance Reporting Framework and the strategic objectives of the PHN Program.

  • Further strengthen the Risk Management Plan to provide actionable guidance to staff on the iterative processes that should be performed for risk identification and risk assessment.

  • Formalise value for money considerations during the grants administration process, ensuring that there is well documented evidence justifying how expended funds are, or will be, providing value for money.

  • Update the Performance Reporting Framework to better measure how, as a Program, it is currently achieving against its purpose and strategic objectives.

         

Key:   Recommendations fully implemented Recommendations on track for completion by due date ✘ Recommendations behind schedule Recommendation not accepted by Health

Note a: The status of recommendations is based on the Department of Health and Aged Care’s tracking of the implementation of recommendations as at July 2023. The ANAO did not confirm the accuracy of Health’s self-assessment.

Note b: The 2018 Protiviti internal audit, 2020 KPMG Operational Expenditure Review and 2022 PwC PHN Governance and Efficiency Review included disclaimers relating to the quality and completeness of evidence and the scope and methods for the analysis. Reviews and audits stated that the evidence relied upon was provided by Health officials and PHNs, was not independently verified by the reviewers, and that data and information was often incomplete.

Source: ANAO analysis of Department of Health and Aged Care records.

Appendix 5 2022–23 Primary Health Network audits

Table A.3: Status of audit recommendations from individual Primary Health Network audits commenced or completed in 2022–23, at November 2023

Primary Health Network

Recommendations or draft recommendations

Status according to Department of Health and Aged Care a

North Queensland

  • Further development of commissioning process and procedure documentation and implement appropriate controls to manage adherence.

Capital Health Network

  • Confirm its audited financial acquittal report format, including the detail on other income, with the Department to validate compliance with the range of requirements across the applicable Schedules for 30 June 2023.
  • Report assets within the definition and threshold to the Department as part of its twelve month report in line with reporting requirements, seek approval for purchases of assets within the definition and threshold.
  • Document declaration of conflicts of interest.

South Eastern Melbourne

  • Undertake the planned external assessment of cyber security arrangements against the Essential Eight framework.
  • Actively consider the areas of need identified by the VMO review, the internal Board of Directors Skills Matrix Survey, and the planned annual Director performance assessments as a focus for the development of future training for individual Directors or the entire Board.
  • Revisit the Constitution’s arrangements for members in the next six months, based on consultation with key stakeholders
  • Review and update (if necessary) any policies and frameworks that have passed (or are approaching) designated review dates.
  • In line with the recommendation from Baseline Maturity Assessment, further consider its framework for demonstrating achievement of the Strategic Plan and in delivering its outcomes.
  • Review process for the effectiveness of its stakeholder consultation, specifically to address whether it has the capacity to capture concerns or issues about the PHN and its operations.

Tracking not yet commenced

Adelaide

  • In line with the Baseline Maturity Assessment, it is recommended that a performance monitoring and reporting framework be established as a matter of priority.
  • Develop an overarching Budget Framework with the inclusion of guidance principles, linkage to mandatory requirements, and clear articulation of its alignment to the Strategic Plan and Commissioning Framework.
  • Continue to develop the Data Governance Framework and seek final approval from the Board as a priority.
  • Identify and set its Essential Eight target maturity level (to be endorsed by the Board). APHN should also continue with a regular program of self-assessment against the Essential Eight maturity model, along with independent external assessment as a matter of priority in the next six months and then periodically, to ensure cyber resilience remains a focus.
  • Detail its plan for systems migration in a formal strategy document (endorsed by the Board) to ensure a comprehensive and systemised approach to change management across the organisation.

Tracking not yet commenced

Northern Sydney

  • Implement a process to ensure that where conflicts of interest are identified and required to be reported to the Department, they advise the Department using the Conflict Notification Form Template.
  • Following endorsement of the Strategic Plan 2024–2028, [Sydney North Health Network] may consider lodging the Plan with the Department for information. We note this recommendation is based largely on the high quality of other SNHN documentation, and the potential for many of SNHN’s governance documents to be used for benchmarking.
  • Consider implementing a formal review cycle for the Commissioning Evaluation Framework and Evaluation Toolkit in line with the review structure in place for other commissioning documentation.

Tracking not yet commenced

Northern Territory

  • Board should proceed with their planned comprehensive review of the NTPHN Constitution to ensure that it remains fit-for-purpose.
  • Board should ensure ongoing vigilance in relation to Conflict of Interest and continue to support their Directors through regular training and communication on how to effectively manage conflict of interest risk.
  • Finalise and seek Board endorsement of the Strategic Plan 2023–2028.
  • Following endorsement of the Strategic Plan 2023–2028, NTPHN should further consider its framework for demonstrating achievement of the Strategic Plan and delivering its outcomes.
  • Continue with its review and update of the Stakeholder Engagement Policy and related documentation with a view to obtaining Board endorsement to finalise.
  • Develop an overarching finance strategy or financial policy.
  • Finalise and seek Board approval for the new Accounts Payable and Payments Procedure.
  • Review and update Disaster and Emergency Management Plan and Business Continuity Plan.
  • Continue to develop a compliance framework with the necessary supporting documentation.
  • Complete the refresh of their commissioning policies and procedures as soon as possible.
  • Review its approach to direct procurement and consider whether the level of reliance on direct procurement is appropriate.
  • Consider and improve its record-keeping arrangements in relation to commissioning activities.
  • Continue to develop the ICT Strategy, Data Governance Policy and other documentation supporting the development of robust data security arrangements.
  • Update and seek Board endorsement for the Information and Communications Technology Security Policy.
  • Conduct either an internal or external cyber capability assessment.
  • Continue to prioritise implementation of the Essential Eight Model at an initial Maturity Level of One.
  • Detail its plan for systems migration in a formal strategy document.

Tracking not yet commenced

     

Key:  Recommendations fully implemented Recommendations on track for completion by due date Recommendations behind schedule

Note a: Status is based on the Department of Health and Aged Care’s tracking of recommendations as at 7 November 2023, which the ANAO has not independently verified.

Source: ANAO analysis of draft and final PHN audit reports and the Department of Health and Aged Care’s recommendations tracker.

Appendix 6 Program logics, 2018 and 2023

Figure A.1: Primary Health Network program logic, 2018

A figure that shows the structure of Health’s 2018 Primary Health Networks program logic model. The figure includes the Primary Health Networks’ objectives and a description of the theory of change. The figure also includes a flow chart covering program logic elements including inputs, participants, activities, outputs, and intermediate and longer-term outcomes.

Source: Department of Health and Aged Care.

Figure A.2: Primary Health Network program logic, 2023

A figure that shows the structure of Health’s 2023 Primary Health Networks program logic model. The figure includes a flow chart showing the relationship between Primary Health Networks objectives, inputs, activities and outputs with impacts

Source: Department of Health and Aged Care.

Appendix 7 Primary Health Network grant agreement performance measures

Table A.4: Primary Health Network grant agreement performance indicators

Grant agreement indicator

Core grant agreements

Activities have been undertaken in accordance with the approved Activity Work Plan as amended and agreed by the Department as appropriate.a

Your organisation reports on total Clinical referral pathways views and top five most viewed pathways.

Your organisation describes the training and education you have provided to local health practitioners in relation to Clinical referral pathways.

Your organisation reports on all of your published aged care pathways, including when they were published or last updated (whatever is most recent).

Your organisation reports on the count of total page views for each of your aged care pathways.

Your organisation reports on consultation activity to inform development and/or enhancement of dementia Clinical referral pathways.

Your organisation reports on information and education activity to support the use of dementia Clinical referral pathways.

Your organisation reports on the count of total page views (and increase in views) for each of your dementia pathways.

Your organisation reports (and provides links to) all of your published dementia consumer resources, including when they were published or last updated (whichever is most recent).

Your organisation reports on consultation activity to inform development and/or enhancement of dementia pathways.

Aged care grant agreements

Activities have been undertaken in accordance with the approved Activity Work Plan as amended and agreed by the Department as appropriate.

The [Primary Health Network (PHN)] to report on the number of participating [Residential Aged Care Facilities] in their region and the number which have had their virtual access capability services assessed.

The PHN to report on the number of participating [Residential Aged Care Facilities] in their region which have the appropriate facilities and equipment to access services virtually.

The PHN to provide a brief description of the training and support they have provided to [Residential Aged Care Facility] staff or practitioners in relation to virtual access services.

The PHN to report on the number of participating [Residential Aged Care Facilities] in their region and the number that have been assessed as to whether they have an after-hours action plan.

The PHN to provide a brief description of the training and support they have provided to [Residential Aged Care Facility] staff in relation to managing after hours care and maintaining resident’s digital medical information.

Report on the number of consumers who have participated in the commissioned intervention activities.

Report on the number of participants who sustained or improved their quality of life based on the International Consortium for Health Outcomes Measurement (ICHOM) older person standard assessment.

Mental health grant agreements

Proportion of regional population receiving PHN-commissioned mental health services — Low intensity interventions.b

Proportion of regional population receiving PHN-commissioned mental health services — Psychological therapies delivered by mental health professionals.b

Proportion of regional population receiving PHN commissioned mental health services — Clinical care coordination services for people with severe and complex mental illness.b

Average cost of PHN-commissioned mental health service — Low intensity mental health interventions.

Average cost of PHN-commissioned mental health service — Psychological therapies delivered by mental health professionals.

Average cost of PHN-commissioned mental health service — Clinical care coordination services for people with severe and complex mental illness.

Proportion of regional youth population receiving PHN-commissioned youth-specific mental health services.

Proportion receiving PHN-commissioned mental health services delivered to the regional Indigenous population where the services were culturally appropriate.b

Proportion of people referred to PHN-commissioned services due to a recent suicide attempt or because they were at risk of suicide followed up within 7 days of referral.b

Clinical outcomes for people receiving PHN-commissioned low intensity mental health interventions.

Clinical outcomes for people receiving PHN-commissioned psychological therapies delivered by mental health professionals.

Extent to which establishment and transition expectations as set out in Item B.3 subsection 3.1 have been met.

Proportion of PHN annual flexible mental health funding allocated to low intensity mental health services, psychological therapies and services for people with severe and complex mental illness.

Evidence of partnerships with other regional service providers to support integrated regional planning and service delivery.b

Extent to which client and sessional data, including client outcome measures has been reported to the [Primary Mental Health Care Minimum Dataset].

Extent to which governance processes are in place and being managed according to national, state and local standards, including the National Standards for Mental Health Services 2010.

Extent to which income and expenditure is managed in a financially appropriate manner that aligns with the Guidelines.

Number of Aboriginal or Torres Strait Islander people that received mental health services in PHN region through this funding.

Types of mental health services that has been provided to clients.

Average length of each episode of care.

Average number of services that clients received.

Number of referrals made to other complementary services.

Indigenous health (Integrated Team Care) grant agreements

Activities have been undertaken in accordance with the approved Activity Work Plan as amended and agreed by the Department as appropriate.

Alcohol and other drugs grant agreements

Organisations and projects have been commissioned to deliver drug and alcohol treatment services.

Activities have been undertaken in accordance with the AOD guidance materials, and the approved Activity Work Plan as amended and agreed by the Department, as appropriate.

 

Note a: This indicator (and variations of it, for example ‘Activities have been undertaken in accordance with the approved service model as agreed by the Department as appropriate’) is included in all grant agreement schedules, and for all activities. This indicator has been recorded once for each grant agreement in the table to avoid repetition. It appears nine times in the core schedule, including for all trial, pilot and emergency response activities that were included in the core grant agreements for some PHNs.

Note b: Indicator is also used as a delivery model indicator in annual performance reporting (see next section).

Source: ANAO analysis of PHN grant agreements.

Appendix 8 Primary Health Network delivery model performance measures

1. Table A.5 shows performance framework measures as well as the way the results against these measures was reported in three available annual reports. The ANAO notes that there are inconsistencies in reported results across years. The Department of Health and Aged Care advised the ANAO that data for some 2019–20 indicators were revised in the 2020–21 Annual Report ‘to improve the accuracy of the reporting and analysis’.

Table A.5: Performance measures and annual reporting for the Primary Health Network delivery model, 2018–19 to 2020–21

Indicator

Performance criteria

Data source

Category

2018–19 Annual Report

2019–20 Annual Report

2020–21 Annual Report

Primary Health Network (PHN) program (core)

P1: PHN activities address prioritised needs

100% of delivered activities address prioritised needs in PHN Needs Assessment and/or national priorities

PHN

Grant agreement management

100% of PHNs (3110 providers were commissioned)

All 31 PHNs demonstrated activities address prioritised needs as set out in PHN Needs Assessment and/or national priorities.

All 31 PHNs demonstrated activities address prioritised needs as set out in PHN Needs Assessment and/or national priorities. This is the same as the 2019–20 reporting period.

P2: Health system improvement and innovation

At least one example of a health system improvement, innovation or commissioning best practice

PHN

Performance against outputs or outcomes

100% of PHNs

All 31 PHNs have provided descriptions of a health system improvement, innovation, or commissioning best practice that has taken place in 2019–20.

All 31 PHNs have provided descriptions of a health system improvement, innovation, or commissioning best practice that has taken place in 2020–21. This is the same as the 2019–20 reporting period.

P3: Rate of general practice accreditation

Increase in rate of general practice accreditation.

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of accreditation

PHN

Performance against outputs or outcomes

The national average rate of general practices that are accredited is 78.4 per cent.

The median rate of accreditation was 84% (up from 78% in 2018–19), and the accreditation rate increased in 52% of PHNs.

The average general practice accreditation rate in 2020–21 was 82% (up from 77% in 2019–20), and the average accreditation rate increased in 71% of PHNs. Accreditation rates increased in 2020–21 in both regional and metropolitan areas compared to 2019–20. Accreditation rates were also higher in regional areas (89%) compared to metropolitan areas (78%).

P4: Support provided to general practices and other health care providers

PHN delivers a range of support activities to general practices and other health care providers

PHN

Grant agreement management

100% of PHNs

All 31 PHNs have demonstrated they provide a range of support activities to general practices and other health care providers within their region.

All 31 PHNs have demonstrated they provide a range of support activities to general practices and other health care providers within their region.

P5: Rate of regular uploads to My Health Record (MyHR or MHR)

Increase in the rate of regular usage by general practices and other health care providers

Secondary data

Performance against outputs or outcomes

13.8% General Practices (2017–18)

The median rate of general practices uploading documents to MyHR at least once a week was 23% (up from 15% in 2017–18, the latest year available at the publication of the previous report).

In 2020–21, the average percentage of general practices uploading documents to MHR at least once a week was 25%, up from 18% in 2019–20.

P6: Rate of general practices receiving payment for after hours services

Maintain the existing rate of general practices receiving payment for after hours services

*contextual information supplied by PHN will be used in assessing the performance criteria

Secondary data

Performance against outputs or outcomes

The average rate in the 2nd quarter 2018 was: 35.7%(level 1); 7.2% (level 2); 4.6%(level 3); 4.0% (level 4); and 13.7%(level 5).c

The median rate of general practices receiving PIP [Practice Incentive Program] after hours payments was 72% (up from 62% in 2018–19), with this rate having increased in 97% of PHNs.

The average percentage of general practices receiving PIP after hours payments was 67% (up from 60%in 2019–20) and remains similar in both metropolitan and regional areas.

P7: Rate of [General Practice] style emergency department (ED) presentations

A decrease in rate of GP style ED presentations*

*contextual information supplied by PHN will be used in assessing the performance criteria

Secondary data

Performance against outputs or outcomes

11.5% (2017–18)

In 2019–20, the rate per 1000 population of lower-urgency emergency department presentations across PHN regions increased slightly in-hours, with a median rate 3% higher than 2018–19.

Not reported

P8: Measure of patient experience of access to [General Practice]

Decrease in ‘GP not available’ or ‘waiting time too long’ as reasons for why patient attended ED*

*Contextual information will be considered in determining whether the performance criteria is met

Secondary data

Performance against outputs or outcomes

546,000 people (2016–17)

Not reported

Not reported

P9: Rate of [General Practice] team care arrangements/ case conferences

Increase in the rate of people diagnosed with chronic conditions who receive GP team care arrangement and case conferences*

*Assessment of this performance criteria will take into account the Health Care Homes trial where relevant

Secondary data

Performance against outputs or outcomes

28.4% People with chronic conditions receiving team care services (2014–15)

The number of services increased between 2014–15 and 2019–20 in all PHNs, with a median increase of 51%.

It increased 5% in 2020–21 from 2019–20.

P9: Rate of GP team care arrangements/case conferences (Aboriginal and Torres Strait Islander people)

Increase in the rate of people diagnosed with chronic conditions who receive GP team care arrangement and case conferences*

*Assessment of this performance criteria will take into account the Health Care Homes trial where relevant

Secondary data

Performance against outputs or outcomes

Data not collecteda

Not reported

Not reported

P10: Cross views of My Health Recorda

5% increase in general practice MyHR provider viewing a record authored by another (from separate HPI-Os [Healthcare Provider Identifier – Organisation]) annually 5% increase in pharmacy MyHR provider viewing a record authored by another (from separate HPI-Os) annually

Secondary data

Performance against outputs or outcomes

19.4 General practices

5 Pharmacies

Cross-views of MyHR (the viewing of a MyHR document authored in a different practice) increased by more than 5% from 2018–19 in all PHNs in general practices that were registered MyHR providers, and in pharmacy providers in 90% of PHNs.

Cross-views of MHR increased in all PHNs by more than 12% from 2019–20 in all general practices that were registered MHR providers. In pharmacy providers, cross-views of MHR increased in 97%of PHNs.

P11: Rate of discharge summaries uploaded to My Health Record

Increase in rate of discharge summaries uploaded to MyHR

Secondary data

Performance against outputs or outcomes

Data not collected

Not reported

Not reported

P12: Rate of potentially preventable hospitalisations (for specific chronic diseases)

Decrease in PPH [potentially preventable hospitalisations] rates*

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of PPH

Secondary data

Performance against outputs or outcomes

PPH 1% increase for chronic and 51% increase for vaccine preventable from previous year (2017–18)

Between the 2018–19 and 2019–20 FY, reductions in PPH were reported by more than 90% of PHN regions, declining on average by around 6%.

In 2020–21, there was an average 6%decrease nationwide in the rate of PPH per 100,000 population from 2019–20.

P12: Rate of potentially preventable hospitalisations (for people over 65)

Decrease in PPH rates*

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of PPH

Secondary data

Performance against outputs or outcomes

3,448,118 Total PPH>65 (2017–18)

Not reported

Not reported

P12: Rate of potentially preventable hospitalisations (Aboriginal and Torres Strait Islander people)

Decrease in PPH rates*

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of PPH

Secondary data

Performance against outputs or outcomes

Data not collecteda

Not reported

Not reported

P13: Numbers of health professionals available

There are a range of primary health care professionals available within the PHN region

PHN and secondary data

Neither

Not reported

Not reported

Not reported

Aged care

AC1: Rate of [Medicare Benefits Scheme] services provided by primary care providers in residential aged care facilities

Increase in rate of MBS services in RACF [residential aged care facilities]

Secondary data

Performance against outputs or outcomes

16.9 GP consultation

services/ RACF place

3.2 GP after hours care services/ RACF place

The median rate of services per RACF place was 21 (up from 20 in 2016–17), with a median increase between years of 5%.

In 2020–21, this rate returned to the 2018–19 figure of 17.8 services per resident per year after reaching 18.3 in 2019–20.

AC2: Rate of people aged 75 and over with a GP [General Practice] health assessment

Increase in rate of people aged 75 and over with a GP health assessment

Secondary data

Performance against outputs or outcomes

24.20% GP health assessments >=75 yrs. (2016–17)

The median rate of the PHN population at least 75 years old with a GP health assessment is increasing across all PHNs. In 2019–20 the rate was 30% (up from 23% in 2016–17), with the median increase being 29%.

This has slightly declined to 15.3%in 2020–21 from 15.6%in 2019–20 and 15.8% in 2018–19.

Aboriginal and Torres Strait Islander health

IH1: Numbers of ITC [Integrated Team Care] services delivered by PHN

Services are being delivered across the range of services allowed within ITC Guidelines

PHN

Grant agreement management

962,136 unique services

All 31 PHNs have provided evidence of delivering services across the range allowed by Integrated Team Care guidelines, including care coordination, supplementary services, and clinical services.

All 31 PHNs have provided evidence of delivering services across the range allowed by Integrated Team Care guidelines, including care coordination, supplementary services, and clinical services. This is the same as the 2019–20 reporting period.

IH2: Types of organisations delivering ITC services

A range of organisations are engaging in ITC program

PHN

Grant agreement management

100% of PHNs

All 31 PHNs have shown engagement with an appropriate range of Integrated Team Care services including Aboriginal Medical Services, mainstream organisations, and services delivered by the PHN itself.

30 PHNs have shown engagement with an appropriate range of Integrated Team Care services including Aboriginal Medical Services, mainstream organisations, and services delivered by the PHN itself. This is 1 less than in the 2019–20 reporting period.

IH3: Evidence that all drug and alcohol commissioned services are culturally appropriate for Aboriginal and Torres Strait Islander peoplec

PHN supplies evidence that commissioned drug and alcohol services are culturally appropriate

PHN

Grant agreement management

26 PHNs supplied evidence to ensure drug and alcohol commissioned services are culturally appropriate for Aboriginal and Torres Strait Islander people in order to meet the performance criteria. 5 PHNs did not meet reporting requirements for this indicator.

30 PHNs provided adequate evidence of the cultural appropriateness of these services, which is 3 more than in the 2018–19 reporting period.

All 31 PHNs provided adequate evidence of the cultural appropriateness of these services, which is 1 more than in the 2019–20 reporting period.

IH4: Proportion of PHN commissioned mental health services delivered to the regional Aboriginal and Torres Strait Islander population that were culturally appropriateb

At least 5% growth on proportion of previous year or where all services were culturally appropriate, maintenance of this

PHN

Grant agreement management

61.30% of MH services (Baseline)

18 PHNs reported at least a 5% increase in the proportion of these services that were culturally appropriate between the 2018–19 and 2019–20 reporting periods

18 PHNs met the growth target (at least a 5%increase) for the proportion of PHN commissioned mental health services delivered to Aboriginal and Torres Strait Islander people that were culturally appropriate in the 2020–21 reporting period. However, 23 PHNs improved on the proportion of services that were culturally appropriate compared to 2019–20.

IH5: ITC improves the cultural competency of mainstream primary health care services

PHN provides evidence that as part of ITC it is working to improve cultural competency of mainstream primary health care services

PHN

Grant agreement management

30 PHNs

All 31 PHNs have described sufficient activities undertaken to improve the cultural competency of mainstream primary health care services

All 31 PHNs have described sufficient activities undertaken to improve the cultural competency of mainstream primary health care services. This is the same as the 2019–20 reporting period.

IH6: PHN provides support for Aboriginal and Torres Strait Islander identified health workforce

PHN supplies evidence of support provided to Aboriginal and Torres Strait Islander identified workforce in its region

PHN

Grant agreement management

84% of PHNs

30 PHNs have supplied either or both descriptions of formal and informal support activities, and a workforce strategy addressing the capability, capacity, and proportion of the Aboriginal and Torres Strait Islander identified health workforce. This is 4 more than in the 2018–19 reporting period.

30 PHNs have supplied either or both descriptions of formal and informal support activities, and a workforce strategy addressing the capability, capacity, and proportion of the Aboriginal and Torres Strait Islander identified health workforce. This is the same as the 2019–20 reporting period.

IH7: ITC processes support Aboriginal and Torres Strait Islander people enrolled in the program to access coordinated care

PHN provides evidence of its ITC processes

PHN

Grant agreement management

100% of PHNs

All 31 PHNs provided satisfactory descriptions of the referral, intake, and discharge processes used in their ITC programs.

All 31 PHNs provided satisfactory descriptions of the referral, intake, and discharge processes used in their ITC programs. This is the same as the 2019–20 reporting period.

IH8: Rate of Aboriginal and Torres Strait Islander population receiving specific health assessments

Increase in rate of population receiving specific health assessment

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of receiving specific health assessments

Secondary data

Performance against outputs or outcomes

33.7% (2016–17)

In 2019–20, median performance was 24 health assessments performed per 100 Indigenous PHN residents (down from 30 per 100 in 2016–17).

The number of First Nations people who had health assessments slightly decreased by 1% in 2020–21 from 2019–20, and by 1% in 2019–20 from 2018–19.

Alcohol and other drugs

AOD1: Rate of drug and alcohol commissioned providers actively delivering services

Rate of drug and alcohol commissioned providers actively delivering services increases or remains the same

PHN

Grant agreement management

98% Commissioned providers (Baseline)

All 31 PHNs report that the rate of drug and alcohol commissioned providers actively delivering services has remained the same or increased from 2018–19 to 2019–20.

All 31 PHNs report that the rate of drug and alcohol commissioned providers actively delivering services has remained the same or increased in the 2020–21 reporting period.

AOD2: Partnerships established with local key stakeholders for drug and alcohol treatment servicesd

A range of organisations are involved in delivering drug and alcohol treatment services

PHN

Grant agreement management

29 PHNs demonstrated evidence of formalised partnerships and collaboration established with local key stakeholders.

All 31 PHNs have a satisfactory range of organisations involved in delivering drug and alcohol services.

All 31 PHNs have a satisfactory range of organisations involved in delivering drug and alcohol services.

Digital health

DH1: Rate of health care providers informed about My Health Recorda

100% of general practices are aware of and provided with access to MyHR education

PHN

Grant agreement management

99% General practices

100% Pharmacies

All 31 PHNs report 100% of general practices are aware of and provided access to My Health Record education

All 31 PHNs reported 100%of general practices are aware of and provided access to My Health Record education.

DH2: Rate of health care providers using specific digital health systems

Increase in the rate of health care providers using smart forms, e-referrals and/or telehealth Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing rate of using specific digital health systems

PHN

Performance against outputs or outcomes

PHNs reported that they do support health care providers to use digital health systems to improve patient care and communication.b

27 PHNs reported an increase in the rate of general practices, pharmacies, and allied health service practices using smart forms, e-referrals, and telehealth

30 PHNs reported an increase in the rate of general practices, pharmacies, and allied health service practices using smart forms, e-referrals, and telehealth

DH3: Rate of accredited general practices sharing data with PHN

At least 5% growth on rate of accredited general practices sharing data with the PHN each year Where the rate is over 60%, the performance criteria is to maintain the existing rate

PHN

Grant agreement management

76% general practices (Baseline)

25 PHNs reported at least a 5% increase in the rate of accredited general practices sharing data with them (or where the baseline rate was over 60%, maintenance of that rate).

23 PHNs reported at least a 5% increase in the rate of accredited general practices sharing data with them (or where the baseline rate was over 60%, maintenance of that rate).

Mental health

MH1: Rate of regional population receiving PHN commissioned low intensity psychological interventionsb

At least 5% growth in number of people accessing Low Intensity episodes compared with previous year, or where service capacity has been reached, a maintenance of the previous year’s rate

PHN

Performance against outputs or outcomes

108 people per 100k (Baseline)

20 PHNs reported at least 5% growth in the number of people accessing low intensity episodes from 2018–19 to 2019–20.

17 PHNs reported met the growth target (at least 5% growth) for an increase in the number of people accessing PHN-commissioned low intensity psychological interventions in the 2020–21 reporting period compared to the 2019–20 reporting period.

MH2: Rate of regional population receiving PHN commissioned psychological therapies delivered by mental health professionalsb

At least 5% growth in number of people accessing Psychological Therapy episodes compared with previous year, or where service capacity has been reached, a maintenance of the previous year’s rate

PHN

Performance against outputs or outcomes

median rate was 355 people per 100k (baseline)

17 PHNs reported at least 5% growth in the number of people accessing psychological therapy episodes from 2018–19 to 2019–20.

17 PHNs met the growth target (at least 5% growth) in the number of people accessing PHN-commissioned psychological therapies in the 2020–21 reporting period. 20 PHNs improved on rates of access to PHN-commissioned psychological therapies compared to 2019–20.

MH3: Rate of regional population receiving PHN commissioned clinical care coordination services for people with severe and complex mental illnessb

At least 5% growth in number of people accessing Care Coordination episodes compared with previous year, or where service capacity has been reached, a maintenance of the previous year’s rate

PHN

Performance against outputs or outcomes

Median rate 61 persons per 100k (baseline)

14 PHNs reported at least 5% growth in the number of people accessing care coordination episodes from 2018–19 to 2019–20.

15 PHNs met the growth target (at least 5% growth) in the number of people accessing PHN-commissioned care coordination services in the 2020–21 reporting period compared to the 2019–20 reporting period.

MH4: Formalised partnerships with other regional service providers to support integrated regional planning and service deliveryb

Comprehensive regional mental health and suicide prevention plans to be jointly developed with LHNs by mid 2020

PHN

Grant agreement management

30 PHNs

30 PHNs had comprehensive regional mental health and suicide prevention plans being jointly developed with LHNs.

All 31 PHNs, in collaboration with their respective state and territory government-funded commissioning bodies, and other stakeholders, delivered their joint foundational regional mental health and suicide prevention plans.

MH5: Proportion of people referred to PHN commissioned services due to a recent suicide attempt or because they were at risk of suicide followed up within 7 days of referralb

100% of people referred are followed up within 7 days

PHN

Grant agreement management

55.50% of people (baseline)

1 PHN reported 100% of episodes where suicide risk was identified at referral received follow-ups within 7 days. There are limitations to this performance measure and the Department is working with PHNs to improve the data definitions and methodology used to calculate the KPI.

No PHNs met the target indicator of 100% follow up of clients at risk of suicide. However, 68% improved on the number of clients followed up compared to 2019–20.

MH6: Outcomes Readiness — Completion rates for clinical outcome measures

70% of completed episodes of care have recorded valid outcome measures at Episode Start and Episode End

PHN

Grant agreement management

25.50% of episodes (baseline)

In 2019–20 6 PHNs reported the required 70% of episodes of mental health care as having valid outcome measures taken at the start and end of the episode.

In 2020–21, 5 PHNs met the 70% target rate of episodes of care that recorded outcome measures at episode start and episode completion. The national median proportion of episodes with outcomes collected was 42% in 2020–21.

Population health

PH1: Rate of children fully immunised at 5 years

95% national immunisation target or increase in immunisation rate for region

Secondary data

Performance against outputs or outcomes

93.5% of 5yr olds (2016–17)

35% of PHNs met the national immunisation target of having 95% of children 5 years of age fully immunised, while 10% of PHNs had improved the fully immunised rate for children 5 years of age but did not meet the target.

In 2020–21, the average percentage of 5-year-olds fully vaccinated increased to 95.28% from 94.87% in 2019–20 and 94.98% in 2018–19.

PH2: Cancer screening rates for cervical, bowel and breast cancer

Increase in rate of specified population participation rates in cancer screening

*Where the rate has been stable for at least 3 years, the performance criteria is to maintain the existing participation rate

Secondary data

Performance against outputs or outcomes

55.3% decline cervical cancer (2015–16)

42.4% increase bowel cancer screening (2017–18)

55.0% increase breast cancer screening (2016–17)

Not reported

Note on cancer data: the latest bowel and breast cancer screening data is from 2019–20 and is not yet available for the 2020–21 period.

The number of patients receiving bowel cancer screening has been steadily increasing over the past 5 years.

The number of patients receiving breast cancer screening decreased 5% in 2019–20.

The latest cervical cancer screening data spanned 2 years, from 2018–20, and was not available for the 2 years prior (2016–18). The number of patients receiving cervical cancer screenings in 2018–20 was the same as in 2015–16.

Workforce

W1: Rate of drug and alcohol treatment service providers with suitable accreditationc

All specialist drug and alcohol treatment service providers have or are working towards accreditation

PHN

Grant agreement management

Data not collected

26 PHNs report all specialist drug and alcohol treatment service providers have or are working toward accreditation.

30 PHNs report all specialist drug and alcohol treatment service providers have or are working toward accreditation.

W2: PHN support for drug and alcohol commissioned health professionalsc

PHN supplies evidence of support provided to drug and alcohol commissioned health professionals

PHN

Grant agreement management

24 PHN supplied evidence that they are supporting drug and alcohol commissioned health professionals to meet the performance criteria. 7 PHNs did not meet reporting requirements for this indicator.

26 PHNs supplied adequate evidence of support provided to drug and alcohol commissioned health professionals, which is 5 more than in the 2018–19 reporting period.

28 PHNs supplied adequate evidence of support provided to drug and alcohol commissioned health professionals. This is 2 more than in the 2019–20 reporting period.

W3: PHN Commissioning Framework

The PHN has a Commissioning Framework which includes strategic planning, procuring services and monitoring and evaluation phases, with cultural appropriateness and stakeholder engagement considered throughout the process

PHN

Grant agreement management

17 PHN regions have Frameworks that include strategic planning, procuring services and monitoring and evaluating phases, with cultural appropriateness and stakeholder engagement considered throughout.

All 31 PHNs have Commissioning Frameworks including strategic planning, procuring services, and monitoring evaluation phases, with cultural appropriateness and stakeholder engagement considered throughout the process. This is 14 more than in the 2018–19 reporting period. This is the indicator where the greatest positive change has occurred

All 31 PHNs have Commissioning Frameworks including strategic planning, procuring services, and monitoring evaluation phases, with cultural appropriateness and stakeholder engagement considered throughout the process.

Note a: Measure in the My Health Record Expansion grant agreement schedule.

Note b: Measure in the Mental Health and Suicide Prevention grant agreement schedule.

Note c: Measure in the Drug and Alcohol Treatment Information Strategy.

Source: ANAO analysis of 2018 PHN Performance and Quality Framework and PHN annual reports from 2018–19 to 2020–21.

Appendix 9 Corporate plan measures contributed to by Primary Health Networks

Table A.6: Corporate plan measures contributed to by Primary Health Networks

2022–23 program

Activity and measure

2022–23 target

2022–23 result

1.1 Health research, coordination and access

The rate of avoidable readmissions to public hospitals reduces over time.

Reduced rate of avoidable readmissions compared to 2021–22 baseline.

Data not available

1.2 Mental health

Primary Health Network (PHN)-Commissioned mental health services used per 100,000 population.

Annual increase on 2021–22 numbers.

Not met — 6337 per 100,000. A decrease from 6552 services in 2021–22

1.3 First Nations health

Finalise and commence implementation of the National Aboriginal and Torres Strait Islander Health Plan 2021–2031 (Health Plan) and the National Aboriginal and Torres Strait Islander Health Workforce Strategic Framework and Implementation Plan 2021–2031 (Workforce Plan).

By 2031, increase the proportion of First Nations babies with a healthy birthweight to 91%.

Develop accountability and implementation arrangements for the Health Plan and the Workforce Plan.

Not met — Discussions with our Aboriginal and Torres Strait Islander stakeholders did not occur in 2022 as planned.

Percentage of First Nations babies who attend First Nations primary health care services, increase the number of those that have

a healthy birthweight.

89.6%

85.7%

1.5 Preventative health and chronic disease

Improve overall health and wellbeing of Australians by achieving preventive health targets.

a. Percentage of adults who are daily smokers.

b. Percentage of population who drink alcohol in ways that put them at risk of alcohol related disease or injury.

c. Percentage of population who have used an illicit drug in the last 12 months.

Increase the level of cancer screening participation.

a. National Bowel Cancer Screening Program.

b. National Cervical Screening Program.

c. BreastScreen Australia Program.

Progressive decrease

Data not available

1.6 Primary health care quality and coordination

The number of PHN regions in which the rate of potentially preventable hospitalisations is declining, based on the latest available Australian Institute of Health and Welfare longitudinal data.

26 PHNs

Met — 30 PHNs

3.1 Access and information

Maintain efficiency of My Aged Care assessments as demonstrated by the percentage of:

a. High priority comprehensive assessments completed within 10 calendar days of referral acceptance for community setting.

b. High priority comprehensive assessments completed within 5 calendar days of referral acceptance for hospital setting.

c. High priority home support assessments completed within 10 calendar days of referral acceptance (community setting only).

>90% for all three assessment types

Substantially met — ‘High priority comprehensive assessments completed within 10 calendar days of referral acceptance for

community setting’ was not met 77.6% and a decline from 95.5% in 2021–22

The percentage of surveyed users who are satisfied with the service provided by the:

a. My Aged Care Contact Centre.

b. My Aged Care website.

> 95%(a)

> 65%(b)

Substantially met — My aged care website not met 48.4%. No improvement from 2021–22.

3.3 Aged care quality

Percentage of care givers providing feedback via a survey who report an improvement in confidence when managing Behavioural and Psychological Symptoms of Dementia, following an intervention from the Dementia Behaviour Management Advisory Service (DBMAS) or the Severe Behaviour Response Teams (SBRT).

> 90%

Met — 94%

       

Source: Department of Health and Aged Care Annual Report 2022–23.

Appendix 10 Evaluations of Primary Health Network activities, pilots and trials

Table A.7: Evaluations of Primary Health Network activities, pilots and trials

Evaluation report

Date

Number of Primary Health Networks

Evaluator

Primary Care Living with COVID

March 2023

31

Alan Jeffery

Evaluation of the Improving Health System Responses to Family and Domestic Violence Primary Health Network (PHN) Pilots

February 2023

6

Sax Institute

Evaluation of the Aged Care Temporary Allied Health Measures: Final Report

January 2023

11

Nous Group

The Way Back Support Services Evaluation

December 2022

27

Nous Group

Initial Assessment and Referral (IAR) Decision Support Tool for Mental Health Care

December 2022

Not specified

Primary Health Tasmania

Evaluation of the National Headspace Program

October 2022

31

KPMG Australia

Social Policy Research Centre of the University of New South Wales (UNSW)

Evaluation of the Wound Management Pilot

October 2022

3

Nous Group

Evaluation of the Health Care Homes Trial

July 2022

10

Health Policy Analysis Pty Ltd

UNSW

Centre for Health Economics, Research and Evaluation (University of Technology Sydney)

Evaluation of Head to Help and Adult Mental Health Centres final evaluation report

April 2022

6

Nous Group

University of Sydney

Evaluation of the PHNs’ Improved Access to Psychological Services in Aged Care Facilities initiative

March 2022

31

Australian Healthcare Associates Pty Ltd

Evaluation of the Improving Social Connectedness of Older Australians project pilot

March 2022

2

Centre for Health Service Development (CHSD)

Australian Health Services Research Institute (AHSRI) (University of Wollongong)

Evaluation of the Greater Choice for At Home Palliative Care Pilot Measure

November 2021

11

Deloitte Access Economics

Evaluation of National Psychosocial Support Programs

July 2021

31

Nous Group

Evaluation of the Farmers Trial and the Fishers Trial

May 2021

9

CHSD

AHSRI

Implementation Review of the National Initial Assessment and Referral Guidance

March 2021

9

University of Melbourne

PHN Initial Assessment and Referral for Mental Health Care — State of Play report

February 2021

31

Australian Government Department of Health and Aged Care

Primary Health Network After Hours Evaluation Report | Australian Government Department of Health

January 2021

29

Health Policy Analysis Pty Ltd

National Suicide Prevention Trial

December 2020

11

University of Melbourne

Evaluation of the Early Psychosis Youth Services — Final Report

August 2020

 

The George Institute for Global Health

The University of Sydney

EY

Headspace Pilbara evaluation

March 2020

8

Telethon Kids Institute

Review of Care Coordination within the Integrated Team Care Program

July 2018

Not specified

Health Policy Analysis Pty Ltd

PHN After Hours Review

November 2016

15

EY

Evaluation of the Headspace Program

October 2015

Not specified

UNSW

Curtin University

University of Western Australia

       

Source: ANAO analysis of evaluation reports involving PHNs.

Appendix 11 Reported data for quantitative Performance Framework measures

1. This appendix provides individual Primary Health Network (PHN) and average reported results for 14 out of the 39 measures in the PHN Performance Framework that are quantitative measures and based on PHN-reported data. The 14 quantitative measures include grant agreement management measures. The following tables summarise performance results for each individual PHN against the 14 quantitative measures based on PHN 12-montly reports to the Department of Health and Aged Care (Health). Averages were calculated by the ANAO. Data that was provided for measure DH2 (‘Rate of health care providers using specific digital health systems’) could not be shown due to inconsistencies across PHNs in the type of data supplied.

2. Health advised the ANAO that comparative analysis between PHNs is not meaningful ‘due to differences in contextual factors, such as demographics, per capita of target population, funding provided, and cost of [service delivery] in different environments (e.g. metro, regional, remote etc)’.73

Table A.8: Core (primary health) measures

 

P3: Rate of general practice accreditation

P13: Number of health professionals available

PHN

2018–19 (%)

2020–21 (%)

2018–19

2020–21

Adelaide

71

85

2064

2088

Australian Capital Territory

96

75

2374

1875

Brisbane North

74

75

1719

1875

Brisbane South

91

85

2097

2090

Central and Eastern Sydney

62

64

2589

2755

Central Queensland, Wide Bay, Sunshine Coast

85

93

1227

Country South Australia

87

98

1062

1121

Country Western Australia

73

91 a

11,342 a

Darling Downs and West Moreton

83

88

1240

1308

Eastern Melbourne

84

88

2866

3044

Gippsland

77

94

322

668

Gold Coast

84

86

1190

1280

Hunter New England and Central Coast

71

86

2802

2271

Murray

91

89

1441

3120

Murrumbidgee

93

93

491

484

Nepean Blue Mountains

66

72

672

629

North Coast

88

89

1332

1371

North Western Melbourne

72

73

2863

3105

Northern Queensland

65

89

1774

2851

Northern Sydney

78

71

1895

1953

Northern Territory

81

80

1236

549

Perth North a

75

91 a

11,342 a

Perth South a

79

91 a

11,342 a

South Eastern Melbourne

75

73

2604

3261

South Eastern New South Wales (NSW)

70

80

1111

1150

South Western Sydney

61

59

1540

1557

Tasmania

84

95

1178

1502

Western NSW

85

94

577

589

Western Queensland

66

65

168

153

Western Sydney

77

79

1815

1928

Western Victoria

88

84

1587

1802

Average

80

82

1578

1700

         

Note a: There are three PHN regions in Western Australia but they are managed by one PHN organisation (Western Health Alliance). Health did not disaggregate data for each PHN region in Western Australia for P3 and P13 in 2020–21. It reported a combined 91 per cent in 2020–21 for P3, and 11,342 in 2020–21 for P13.

Source: ANAO analysis of PHN reporting.

Table A.9: Alcohol and other drugs measures

AOD1: Rate of drug and alcohol commissioned providers actively delivering services

 

 

PHN

2018–19 (%)

2020–21 (%)

Adelaide

100

100

Australian Capital Territory

100

100

Brisbane North

100

100

Brisbane South

88

100

Central and Eastern Sydney

100

100

Central Queensland, Wide Bay, Sunshine Coast

93

100

Country South Australia

100

100

Country Western Australia

100

100

Darling Downs and West Moreton

95

100

Eastern Melbourne

100

100

Gippsland

100

100

Gold Coast

100

100

Hunter New England and Central Coast

100

100

Murray

84

100

Murrumbidgee

100

100

Nepean Blue Mountains

86

100

North Coast

100

100

North Western Melbourne

100

Northern Queensland

100

100

Northern Sydney

96

100

Northern Territory

100

100

Perth North

100

100

Perth South

100

100

South Eastern Melbourne

96

100

South Eastern NSW

100

100

South Western Sydney

100

100

Tasmania

100

100

Western NSW

100

100

Western Queensland

100

100

Western Sydney

100

100

Western Victoria

100

100

Average

98

100

     

Source: ANAO analysis of PHN reporting.

Table A.10: Digital health measures

 

DH1: Rate of health care providers informed about My Health Record

DH2: Rate of health care providers using specific digital health systems a

DH3: Rate of accredited general practices sharing data with PHN

PHN

2018–19 (%)

2020–21 (%)

2018–19 (%)

2020–21 (%)

2018–19 (%)

2020–21 (%)

Adelaide

100

100

85

90

Australian Capital Territory

100

100

80

89

Brisbane North

100

100

92

87

Brisbane South

100

100

81

97

Central and Eastern Sydney

100

100

60

92

Central Queensland, Wide Bay, Sunshine Coast

100

100

78

91

Country South Australia

100

100

55

85

Country Western Australia

100

100

78

96

Darling Downs and West Moreton

100

100

38

84

Eastern Melbourne

100

100

67

97

Gippsland

100

100

94

97

Gold Coast

100

100

85

93

Hunter New England and Central Coast

100

100

78

100

Murray

100

100

81

100

Murrumbidgee

100

100

74

87

Nepean Blue Mountains

100

100

82

91

North Coast

100

100

77

86

North Western Melbourne

100

100

72

91

Northern Queensland

100

100

84

92

Northern Sydney

100

100

66

91

Northern Territory

100

100

93

94

Perth North

100

100

80

91

Perth South

100

100

62

96

South Eastern Melbourne

100

100

61

99

South Eastern NSW

100

100

70

92

South Western Sydney

100

100

75

100

Tasmania

100

100

44

90

Western NSW

100

100

86

96

Western Queensland

100

100

100

93

Western Sydney

100

100

94

92

Western Victoria

100

100

68

86

Average

100

100

75

92

             

Note a: PHNs are required to report against indicator DH2 and provide six different figures (the numbers of general practices, pharmacies and allied health service providers using smart forms, e-referrals and telehealth systems). Reporting was not presented as a rate as required and was too inconsistent between PHNs to allow for presentation in this table.

Source: ANAO analysis of PHN reporting.

Table A.11: Aboriginal and Torres Strait Islander health measures

 

IH1: Numbers of ITC services delivered by PHN

IH2: Types of organisations delivering ITC services a

IH4: Proportion of PHN commissioned mental health services delivered to the regional Aboriginal and Torres Strait Islander population that were culturally appropriate

PHN

2018–19

2020–21

2018–19

2020–21

2018–19 (%)

2020–21 (%) b

Adelaide

42,015

30,252

1

1

84

82

Australian Capital Territory

7,778

9,992

3

2

62

80

Brisbane North

80,413

53,300

1

1

53

85

Brisbane South

70,707

77,607

2

2

73

90

Central and Eastern Sydney

5,308

10,654

2

2

56

90

Central Queensland, Wide Bay, Sunshine Coast

31,152

33,069

3

2

84

81

Country South Australia

38,331

28,010

2

2

74

99

Country Western Australia

52,222

55,763

3

3

95

99.5

Darling Downs and West Moreton

32,691

59,525

3

2

95

93.6

Eastern Melbourne

28,393

15,375

3

3

6

79.8

Gippsland

4,334

5,378

2

2

66

88.9

Gold Coast

22,839

16,659

1

1

85

97.4

Hunter New England and Central Coast

150,055

87,264

3

3

100

92.8

Murray

24,473

17,327

3

3

72

81.5

Murrumbidgee

16,309

13,464

3

2

53

76.9

Nepean Blue Mountains

16,961

27,462

2

2

0.77

100

North Coast

59,203

48,430

3

3

93

100

North Western Melbourne

21,236

14,110

3

3

14

69

Northern Queensland

54,968

99,410

3

3

78

78.1

Northern Sydney

2,607

2,885

1

3

56

95.7

Northern Territory

31,513

32,667

2

3

95

92.2

Perth North

12,475

29,424

3

3

61

93.7

Perth South

40,116

47,632

3

3

73

93.5

South Eastern Melbourne

12,826

15,031

3

3

2

100

South Eastern NSW

33,249

37,780

2

2

88

96.6

South Western Sydney

14,866

23,637

1

1

12

77

Tasmania

10,871

12,596

3

3

48

95.8

Western NSW

22,283

26,164

2

3

90

99.1

Western Queensland

7,588

7,993

1

3

71

77.1

Western Sydney

5,953

47,975

2

1

3.9

62.3

Western Victoria

8,401

2

1

60

78.3

Average

31,037

32,895

2.29

2.29

61.41

87.93

             

Note a: Types organisations include Aboriginal Medical Services, mainstream organisations, and the PHN.

Note b: PHNs reported results in a mix of whole numbers and decimals. Data is presented as reported.

Source: ANAO analysis of PHN reporting.

Table A.12: Mental health measures

 

MH1: Rate of regional population receiving PHN commissioned low intensity psychological interventionsa

MH2: Rate of regional population receiving PHN commissioned psychological therapies delivered by mental health professionalsa

MH3: Rate of regional population receiving PHN commissioned clinical care coordination services for people with severe and complex mental illnessa

MH5: Proportion of people referred to PHN commissioned services due to a recent suicide attempt or because they were at risk of suicide followed up within 7 days of referrala

MH6: Outcomes Readiness — completion rates for clinical outcome measures a

PHN

2018–19

(Clients per 100,000 people)

2020–21

(Clients per 100,000 people)

2018–19

(Clients per 100,000 people)

2020–21

(Clients per 100,000 people)

2018–19

(Clients per 100,000 people)

2020–21

(Clients per 100,000 people)

2018–19

(%)

2020–21

(%)

2018–19

(%)

2020–21

(%)

Adelaide

130

51.79

1,101.88

290.76

795.67

46.24

58.7

62.48

10.4

33.9

Australian Capital Territory

150.36

180.63

101.54

132.1

4.37

3.48

12.3

43

71.1

50.4

Brisbane North

98.86

121.28

194.72

247.74

31.89

26.01

55.5

59

9.8

29.2

Brisbane South

36

337.1

163

316.6

12

95.4

52

58

39

20

Central and Eastern Sydney

57.68

66.1

339.61

214.47

61.24

64.33

46.7

60

21.4

47.5

Central Queensland, Wide Bay, Sunshine Coast

150

145.37

355.21

600.36

141.32

89.27

52.2

46

13.5

58

Country South Australia

256.28

308.52

659.92

878.53

80

243.59

69

86.54

45.5

63.8

Country Western Australia

446.84

341.85

252.67

409.4

67.54

59.64

64.5

84

59.1

85.4

Darling Downs and West Moreton

145.26

149.31

120.26

47.09

144.03

127.71

61.4

49

30.5

41.4

Eastern Melbourne

69.54

48.79

199.62

70.84

60

23.2

49.9

70.9

39

42

Gippsland

101.38

245.93

429.11

957.94

169.58

182.57

46

82

6.5

22

Gold Coast

40

111.52

200

335.88

56.75

76.71

71

83

79.4

83.2

Hunter New England and Central Coast

21

49.39

391.25

414.25

34.17

80.07

68.2

61

11.7

15.2

Murray

95.09

142.04

379.04

800.92

122.58

119.36

28.9

71

53

31.5

Murrumbidgee

226.51

174.98

472.76

315.62

239.26

284.95

56.7

88

32.6

52.8

Nepean Blue Mountains

6.72

186

321.6

420

48.9

51.73

100

79

7.4

77.4

North Coast

121.23

411.24

460

116.43

290

42.9

73

37.3

43.4

North Western Melbourne

180

234

300

380

70

60

63.2

74

24

27.6

Northern Queensland

102.38

85.82

566.48

1295.09

77.18

40.46

58

60

3.1

16

Northern Sydney

8.6

20.5

320

365

33

54.8

52.4

67

48.3

57.6

Northern Territory

311.53

253.3

480.42

708

18.59

112.6

55.8

63.9

0.8

Perth North

94.39

189.44

62.58

73.78

59.66

35.67

68.8

74

48

73.4

Perth South

206.45

267.44

50.21

63.43

45.54

44.93

64.8

90

38.7

82.6

South Eastern Melbourne

160

90

110

180

50

30

43.9

57

34.4

33

South Eastern NSW

14.91

29.5

248.03

303.4

229.87

234.3

40.8

25.5

52.4

South Western Sydney

30.44

76

378.54

580

30.55

60

36

69

27.3

21.1

Tasmania

117

66

448

470

77.18

83

33

30

9

42

Western NSW

108.48

159.86

543.04

386.88

55.54

3.93

15.1

37

2.7

0.3

Western Queensland

520

1192.1

1670

1920.7

580

973.05

57

57

25.5

25.9

Western Sydney

230

688.25

500

1,118.61

340

290

41.3

65

17.4

Western Victoria

10

21.08

650

548.56

60.43

46.22

68.9

74

11.1

34.2

Average

136.9

201.1

400.67

493.74

126.23

126.88

52.74

65.79

28.85

42.69

Note a: PHNs reported results in a mix of whole numbers and decimals. Data is presented as reported. Mental Health Minimum Dataset specifications and methodology available from https://pmhc-mds.com/.

Source: ANAO analysis of PHN reporting.

Footnotes

1 Department of Health and Aged Care, Primary Health Networks [Internet], Health, available from https://www.health.gov.au/our-work/phn/what-PHNs-are [accessed 3 October 2023].

2 Primary health care or primary care is defined by Health as ‘health care people seek first in their community’ including general practice, mental health, drug and alcohol, Indigenous health, sexual and reproductive health and maternal and child health services. Department of Health and Aged Care, About primary care [Internet], Health, available from https://www.health.gov.au/topics/primary-care/about [accessed 14 December 2023].

3 Commissioning in the PHN context is defined by Health as ‘a continual and interactive cycle involving the development and implementation of services based on needs assessment, planning, co-design, procurement, monitoring and evaluation.’ Department of Health and Aged Care, A commissioning overview in the PHN context [Internet], Health, available from https://www.health.gov.au/sites/default/files/documents/2021/06/primary-health-networks-phns-commissioning-information-sheet-overview.pdf [accessed 3 October 2023].

4 Department of Health and Aged Care, Primary Health Networks [Internet], Health, available from https://www.health.gov.au/our-work/phn [accessed 3 October 2023].

5 ibid.

6 1.1 — Health Research, Coordination and Access; 1.2 — Mental Health; 1.3 — Aboriginal and Torres Strait Islander Health; 1.5 — Preventative Health and Chronic Disease Support; 1.6 — Primary Health Care Quality and Coordination; 3.1 — Access and Information; and 3.3 — Aged Care Quality.

7 EY, Evaluation of the Primary Health Network Program, 2018, available from https://www.health.gov.au/resources/publications/evaluation-of-the-primary-health-networks-program [accessed 5 December 2023].

8 Figures for 2023–24 exclude commitments made after August 2023.

9 The 2014 Commonwealth Grants Rules and Guidelines were in effect when PHNs were established in 2015–16. Department of Finance (Finance), Commonwealth Grants Rules and Guidelines, 2014, available from https://www.legislation.gov.au/F2014L00908/asmade/text [accessed 3 November 2023].

10 The National Health Reform Agreement outlines the shared responsibility of the Australian, state and territory governments to work in partnership to improve health outcomes for all Australians and ensure the sustainability of the health system. National Health Funding Body, About the Agreement [Internet], National Health Funding Body, available from https://www.publichospitalfunding.gov.au/public-hospital-funding/about-agreement [accessed 8 November 2023].

11 Minister for Health, ‘Medicare Locals review: Australia’s former Chief Medical Officer Prof John Horvath AO will oversee Australian Government’s review of Medicare Locals’, media release, 16 December 2013.

12 Western Australia has three PHNs operated and managed by a single provider. Twenty-eight PHNs were operating as Medicare Locals prior to the commencement of PHNs, while three new organisations were created to establish the remaining three PHNs. The ANAO did not review the design of the grant opportunities or the assessment process undertaken by Health.

13 Local Hospital Networks (LHNs) manage the delivery of public hospital services and other community-based health services in states and territories as determined by the state or territory government. Some jurisdictions have their own local names for LHNs, such as ‘Local Health Districts’ in New South Wales or ‘Hospital and Health Services’ in Queensland. Australian Institute of Health and Welfare, Local Hospital Network [Internet], AIHW, available from https://meteor.aihw.gov.au/content/491016 [accessed 8 November 2023].

14 Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 11.2.

15 The Executive Committee was called the Executive Board in 2019 and 2020.

16 Telehealth refers to real time clinical consultations conducted via videoconferencing or phone rather than face-to-face.

17 Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 5.3, p. 14. The Commonwealth Grants Rules and Guidelines require the publishing of all grant awards and types of grants such as one-off, ad hoc, competitive, non-competitive, demand-driven and targeted grants.

18 The ANAO selected a targeted sample of PHNs with a range of scores (two high scoring, two low scoring and two middle scoring PHNs) on monitoring and reporting risk indicators in the baseline maturity assessment. They comprised: Brisbane South PHN; Central Queensland, Wide Bay and Sunshine Coast PHN; Country South Australia PHN; Darling Downs and West Moreton PHN; Eastern Melbourne PHN; and North Western Melbourne PHN. The sampling approach did not seek to achieve representation in all states and territories. ANAO did not include in its sample PHNs that had been selected by Health for an audit in 2022–23.

19 Needs assessments outline the health needs in each PHN region.

20 Activity work plans are the types of activities and services a PHN intends to commission.

21 The range in number of required milestone reports in the period 1 July 2019 to 15 November 2023 was 89 reports for Darling Downs and West Moreton PHN to 111 reports for the Central Queensland, Wide Bay and Sunshine Coast PHN.

22 The Performance Rubric listed PHN performance indicators, performance criteria and assessment standards, and sorted the performance indicators into ‘outcome themes’ (Addressing Needs, Quality Care, Improving Access, Coordinated Care and Capable Organisations).

23 Three West Australia PHNs have a single provider who was assessed once. In the analysis below the same risk assessment is counted three times when discussing results for the 31 PHNs.

24 The quantitative risk rating was generated by taking the higher of a PHN’s total annual expenditure and annual budget variation. Expenditure in the baseline maturity assessment excluded funds committed but not yet expended. Budget variation is the difference between the amount provided to each PHN in the reference year via Australian Government grants and the amount spent by the PHN in the corresponding period. Variance is calculated as the percentage of allocated funds not spent. For 29 of 31 PHNs, expenditure was higher and thus was used as the quantitative risk indicator. The baseline maturity assessment recommends revision of the methodology to improve accuracy of the quantitative risk assessment.

25 The qualitative assessment involved a desktop review across six categories: policies and procedures; people; governance and systems; risk and issues management, complaints management; and performance monitoring and reporting. The baseline maturity assessment notes that it considered the design but not the implementation of controls. In the qualitative risk assessment category of performance monitoring and reporting, the baseline maturity assessment report stated that most PHNs were unable to demonstrate that they had a performance monitoring and reporting framework and that very few PHNs had a data assurance program to improve the quality of data underpinning reports to Health. PHNs that were unable to provide evidence of appropriate performance monitoring policies and procedures received a score of 2 out of 5.

26 The date of the most recent update could not be determined for one PHN.

27 Department of Finance, Commonwealth Performance Framework, September 2023, Finance, available from https://www.finance.gov.au/government/managing-commonwealth-resources/planning-and-reporting/commonwealth-performance-framework [accessed 7 December 2023].

28 Department of Finance, Commonwealth Grants Rules and Guidelines 2017, Finance, paragraph 10.2, p. 26.

29 The Department of the Treasury, Commonwealth Evaluation Policy, available from https://evaluation.treasury.gov.au/about/commonwealth-evaluation-policy [accessed 10 November 2023].

30 National Archives of Australia, Principle 7: Business information is saved in systems where it can be appropriately managed, available from https://www.naa.gov.au/information-management/standards/information-management-standard-australian-government/principle-7-business-information-saved-systems-where-it-can-be-appropriately-managed [accessed 12 October 2023].

31 Australian Institute of Family Studies, How to develop a program logic for planning and evaluation [Internet], AIFS, 2016, available from https://aifs.gov.au/resources/practice-guides/how-develop-program-logic-planning-and-evaluation [accessed 26 October 2023].

32 The Commonwealth Evaluation Policy and Toolkit defines outcomes as changes that occur over different periods of time as a result of program activities: short-term outcomes occur as a direct result of program activities and may involve the acquisition of knowledge and skills; medium-term outcomes describe behavioural changes; and long-term outcomes describe system changes.

33 Other outcomes are listed under each of the seven priority area program logics.

34 Intermediate outcomes are as follows. PHNs support general practices and other health care providers to provide quality care to patients. People in the PHN region receive coordinated, culturally appropriate services from local health providers. PHN activities and initiatives address local needs. People in the PHN region are able to access general practices and other services as appropriate. PHNs support general practices and other health care providers to provide appropriate after hours access.

35 Outputs are the actions taken by participants in a program.

36 For example, the program logic for the digital health priority area includes the outcome ‘Health care providers are aware of digital health systems and technologies’, which examines knowledge.

37 For example, the program logic connects the output ‘Local network of health providers established’ with the outcome ‘People in the PHN region receive coordinated, culturally appropriate services from local health care providers’.

38 Health uses ‘theory of change’ to refer to a statement accompanying the program logic that describes how PHN actions will achieve the delivery model objectives.

39 For example, the impacts comprise: ‘reduced potentially preventable hospitalisations’, ‘reduction in lower acuity [emergency department] presentations’, and ‘improved access to quality general practitioner care’.

40 Department of Finance, Commonwealth Grants Rules and Guidelines, 2017, paragraphs 10.7 and 10.8.

41 For example, ‘Your organisation describes the training and education you have provided to local health practitioners in relation to Clinical referral pathways’.

42 For example, ‘The PHN to report on the number of participating [Residential Aged Care Facilities] in their region and the number which have had their virtual access capability services assessed’.

43 For example, ‘Organisations and projects have been commissioned to deliver drug and alcohol treatment services’. There are also 12 indicators related to the alcohol and other drugs priority area which are not listed in the grant agreement. These indicators were developed as part of the Drug and Alcohol Treatment Information Strategy (DATIS), a separate program of funding that PHNs are responsible for delivering, to which the grant agreement refers.

44 For example, ‘Clinical outcomes for people receiving PHN-commissioned low intensity mental health interventions’.

45 For Aboriginal and Torres Strait Islander health activities, there is one indicator in the grant agreement: ‘Activities have been undertaken in accordance with the approved Activity Work Plan as amended and agreed by the Department as appropriate’. Health established an additional 14 indicators related to the Aboriginal and Torres Strait Islander health priority area, which are based on the activities listed in the grant agreement, however, these 14 indicators are documented in 12-month performance report guidance provided to PHNs rather than the grant agreement. There are no separate grant performance measures relating to the population health and workforce priority areas.

46 Department of Finance, Commonwealth Grants Rules and Guidelines, 2017, paragraph 10.9.

47 Department of Health and Aged Care, Primary Health Networks (PHN) performance and quality framework [Internet], Health, 2018, available from https://www.health.gov.au/resources/publications/primary-health-networks-phn-performance-and-quality-framework?language=en [accessed 5 November 2023].

48 Department of Health and Aged Care, Primary Health Network Program Annual Performance Report 2018–19, Health, 2020, p. 1.

49 The ANAO assessed one indicator (‘Numbers of health professionals available’) to be neither a grant agreement management measure, nor a measure of outputs and outcomes.

50 For example, for the performance measure P1, the performance criteria is ‘100% of delivered activities address prioritised needs in PHN Needs Assessment and/or national priorities’. For the performance measure P4 (‘Support provided to general practices and other health care providers’), the performance criteria is ‘PHN delivers a range of support activities to general practices and other health care providers’ and the data source is a ‘Short description of activities undertaken for each focus area above and at each level of support’.

51 Department of Health and Aged Care, PHN Program Performance and Quality Framework, Health, 2018, p. 42.

52 ibid.

53 Department of Health and Aged Care, Primary Health Network Program Annual Performance Report 2018–19, Health, 2020, available from https://www.health.gov.au/resources/publications/primary-health-network-program-annual-performance-report-2018-19?language=en accessed 6 November 2023].

54 Department of Health and Aged Care, Primary Health Network Program Annual Performance Report 2019–20, Health, 2023, available from https://www.health.gov.au/resources/publications/primary-health-network-program-annual-performance-report-2019-20?language=en [accessed 6 November 2023].

55 Department of Health and Aged Care, Primary Health Network Program Annual Performance Report 2020–21, Health, 2023, available from https://www.health.gov.au/resources/publications/primary-health-network-program-annual-performance-report-2020-21?language=en [accessed 6 November 2023].

56 Department of Health and Aged Care, PHN Program Performance and Quality Framework, Health, 2018, p. 42.

57 Indicators not reported in 2018–19 were P11 (‘Rate of discharge summaries uploaded to My Health Record’), P13 (‘Number of health professionals available’) and W1 (‘Rate of drug and alcohol treatment service providers with suitable accreditation’).

58 Indicators not reported in 2019–20 were P8 (‘Measure of patient experience of access to GP’), P11 (‘Rate of discharge summaries uploaded to My Health Record’), P13 (‘Numbers of health professionals available’) and PH2 (‘Cancer screening rates for cervical, bowel and breast cancer’).

59 Indicators not reported in 2020–21 were P7 (‘Rate of GP style emergency department (ED) presentations’), P8 (‘Measure of patient experience of access to GP’), P11 (‘Rate of discharge summaries uploaded to My Health Record’) and P13 (‘Numbers of health professionals available’).

60 Indicators comprise: P9 (‘Rate of GP team care arrangements/case conferences’), P12 (‘Rate of potentially preventable hospitalisations’) and AOD1 (‘Rate of drug and alcohol commissioned providers actively delivering services’).

61 Department of Health, Annual PHN Performance Report 2018–19, p. 14.

62 The Department of the Treasury, Commonwealth Evaluation Policy [Internet], Treasury, available from https://evaluation.treasury.gov.au/about/commonwealth-evaluation-policy [accessed 31 October 2023]. The Department of the Treasury has overseen the Commonwealth Evaluation Policy since the establishment of the Australian Centre for Evaluation on 1 July 2023. The prior Commonwealth Evaluation Policy was overseen by the Department of Finance and took effect on 1 December 2021.

63 What Primary Health Networks are, [Internet], Department of Health and Aged Care, available from https://www.health.gov.au/our-work/phn/what-PHNs-are [accessed 2 November 2023].

64 The Department of the Treasury, Commonwealth Evaluation Policy [Internet], Treasury, available from https://evaluation.treasury.gov.au/about/commonwealth-evaluation-policy [accessed 31 October 2023].

65 Department of Finance, Commonwealth Evaluation Policy [Internet], Finance, 2023, available from https://evaluation.treasury.gov.au/about/commonwealth-evaluation-policy [accessed 8 November 2023].

66 Department of Finance, RMG 130: Evaluation in the Commonwealth, Finance, 2023, available from https://www.finance.gov.au/government/managing-commonwealth-resources/planning-and-reporting/commonwealth-performance-framework/evaluation-commonwealth-rmg-130 [accessed 2 November 2023].

67 Health drafted but did not finalise a 2021–2024 Evaluation Strategy. In July 2021 Health established whole of department guidance to support the evaluation of grant programs. A 2023–2026 Evaluation Strategy was finalised in July 2023. In 2023 Health also developed a range of evaluation fact sheets, including on the development of program logic models, evaluation considerations for new policy proposals, monitoring and evaluating frameworks, designing evaluations and evaluation methods, approaching the market for an evaluator and ethical considerations for evaluations.

68 The delivery model is referred to as a ‘programme’ in the PHN Evaluation Framework.

69 The PHN Evaluation Framework stated that ‘baseline’ data collection would occur in early 2016 that would build on PHN performance reports and publicly available datasets, and that there would be baseline and follow-up interviews and surveys with PHNs.

70 Department of Health and Aged Care, Evaluation of the Primary Health Networks Program [Internet], Health, 2018, available from https://www.health.gov.au/resources/publications/evaluation-of-the-primary-health-networks-program [accessed 31 October 2023].

71 Data for one measure was incomplete and did not support trend analysis. The 13 measures include grant agreement management measures and other types of performance measures.

72 For example, for the mental health measure MH5 (‘Proportion of people referred to PHN commissioned services due to a recent suicide attempt or because they were at risk of suicide followed up within 7 days of referral’), there was an average improvement in this measure over the three years although PHNs are on average 35 per cent below the target (which is 100 per cent).

73 Department of Health and Aged Care, Primary Health Networks PHN performance and quality framework – appendix b – indicator specifications, available from https://www.health.gov.au/sites/default/files/documents/2021/04/primary-health-networks-phn-performance-and-quality-framework-appendix-b-indicator-specifications.pdf.