Introduction

1.1 The Department of Education’s Corporate Plan 2022–23 states that the department contributes to Australia’s economic prosperity and social wellbeing by creating opportunities and driving better outcomes through access to quality education.³

1.2 A range of access and participation programs and initiatives have been implemented to help Australians who want to attend or continue higher education, regardless of circumstances, background or location.⁴ These initiatives are largely administered through the Department of Education.⁵

1.3 The Higher Education Support Act 2003 (HES Act) is the primary piece of legislation governing higher education in Australia. This is supplemented by the Higher Education Support (Other Grants) Guidelines 2022 (made under section 238-10 of the HES Act) which contain requirements on grants payable to higher education providers and other eligible bodies for a variety of purposes.

1.4 These instruments include legislative mechanisms such as grants, programs and schemes to encourage the enrolment of Indigenous persons, people from regional and remote areas, and people from a low socioeconomic background to access and participate in higher education. It should be noted that not all regional higher education programs and initiatives are administered under HES Act.

Regional and remote student statistics

1.5 Figure 1.1 depicts the uptake of higher education for regional and remote students since 2005. Between 2005 and 2021, the number of students from regional and remote areas enrolled in higher education has increased from 134,756 to 215,367 students.

Figure 1.1: Total number of regional and remote students enrolled in higher education, 2005 to 2021

Notes: Regional classification data is based on a student’s postcode of permanent home residence. Regional and remote categories were derived from the Ministerial Council on Education, Employment, Training and Youth Affairs classification (MCEETYA) up until 2010. The MCEETYA codes were derived from the Australian Standard Geographical Classification. From 2011, regional classification was based on the Australian Statistical Geography Standard (ASGS). The 2011 version of the ASGS applied from 2011 to 2015. The 2016 version of the ASGS applies to 2016 onwards to account for adjustments made to the classification standard. Overlap in years 2011 and 2016 account for the transition in ASGS classification standards used. Both superseding and legacy standards were applied for those two years.

Years are calendar years.

Source: ANAO analysis of the Department of Education’s Higher Education Statistics Collection, available online from https://app.powerbi.com/view?r=eyJrIjoiYmNlMzFjZTQtNzIzNC00NTE3LTllYjAtZWI4NmE4NGIyNmExIiwidCI6ImRkMGNmZDE1LTQ1NTgtNGIxMi04YmFkLWVhMjY5ODRmYzQxNyJ9 [accessed 15 February 2023].

1.6 Figure 1.2 highlights the attainment rate⁶ and participation rate⁷ for regional and remote students between 2009 and 2021 at Table A and B institutions.⁸ The rates have remained stable during this time period. The access and participation targets set out in the National Regional, Rural and Remote Tertiary Education Strategy (the Strategy) for achievement by 2030 are outlined in Table 2.6.

Figure 1.2: Award attainment and participation rates for regional and remote students at Table A and B institutions, 2009 to 2021

Notes: Regional classification data is based on a student’s postcode of permanent home residence. Regional and remote categories were derived from the Ministerial Council on Education, Employment, Training and Youth Affairs classification (MCEETYA) up until 2010. The MCEETYA codes were derived from the Australian Standard Geographical Classification. From 2011, regional classification was based on the Australian Statistical Geography Standard (ASGS). The 2011 version of the ASGS applied from 2011 to 2016. The 2016 version of the ASGS applies to 2016 onwards to account for adjustments made to the classification standard. Overlap in years 2011 and 2016 account for the transition in standards used. Both superseding and legacy standards were applied for those two years.

In order to receive government funding, providers must be approved under the HES Act. The Act defines three groups of institutions — Table A, B, and C. Table A institutions are self-accrediting bodies, eligible for all funding under the Act. Table B institutions are self-accrediting bodies, not eligible for general Commonwealth funded places. They are eligible for Commonwealth research funding and can be allocated national priority student places in fields such as nursing and education.

The attainment rates measure award course completions by regional or remote students over all domestic award course completions.

The participation rates measure all regional or remote students over all domestic onshore students.

Years are calendar years.

Higher Education Statistics Collection data utilises a static, point in time head count of students to perform rate calculations.

Source: ANAO analysis of the Department of Education’s Higher Education Statistics Collection, available online from https://app.powerbi.com/view?r=eyJrIjoiYmNlMzFjZTQtNzIzNC00NTE3LTllYjAtZWI4NmE4NGIyNmExIiwidCI6ImRkMGNmZDE1LTQ1NTgtNGIxMi04YmFkLWVhMjY5ODRmYzQxNyJ9 [accessed 15 February 2023].

National Regional, Rural and Remote Education Strategy

1.7 The Government released the National Regional, Rural and Remote Education Strategy (the Strategy), also known as the Napthine Review⁹ on 28 August 2019. The Strategy focused on improving tertiary education participation and outcomes for students from regional, rural and remote (RRR) areas, as part of a broader Regional Education package. Under the Strategy, tertiary education is defined as:

higher education and vocational education and training (VET), focusing on post-secondary education qualifications.¹⁰

1.8 In June 2020, the Australian Government announced the Job-Ready Graduates Package in response to the recommendations made in the Strategy to commence from 2021. The package of programs and initiatives were designed to:

bridge the gap between regional and remote students and metropolitan students, drive productivity for the regions and increase the research capacity of regional universities.¹¹

1.9 A detailed overview of the Strategy and the government’s response is presented at Appendix 3.¹²

Access and participation funding for regional and remote students

1.10 Table 1.1 below outlines funding allocated for the package of in-scope¹³ access and participation programs and initiatives for regional and remote students in response to the Strategy, and relevant programs administered more broadly in the department relating to regional and remote students in Australia.¹⁴ A timeline of programs and initiatives has been presented in Appendix 4.

1.11 This package of programs and initiatives have been funded from 2020–21 through to 2023–24, with the exception of the Rural and Regional Enterprise Scholarships (RRES) Program which commenced in 2017–18 and funding allocated until the end of 2022–23.

Table 1.1: Overview of funding allocation for programs and initiatives

Note: The package of programs and initiatives relate to regional and remote activities undertaken by the Department of Education. Not all activities form part of the Job-Ready Graduates Package or were initiated in response to the Strategy announced by the Australian Government.

Source: ANAO analysis using Department of Education data.

Previous reviews

1.12 Recent Australian Government reviews and strategies, with more detail in Appendix 3, include:

• John Halsey, Independent Review into Regional Rural and Remote Education — Final Report, (Canberra: Department of Education, January 2018)¹⁵; and
• Denis Napthine, National Regional, Rural and Remote Education Strategy (Canberra: Department of Education, August 2019).¹⁶

Rationale for undertaking the audit

1.13 On 19 June 2020, the Australian Government announced a series of programs and initiatives to provide opportunities for regional Australia as part of the Job Ready Graduates Package worth more than $400 million over five years targeted at bridging the gap between regional and remote students and metropolitan students.¹⁷ This package was announced in response to the findings and recommendations from the National Regional, Rural and Remote Tertiary Education Strategy (the Strategy). Refer to Appendix 3 for information on the Strategy.

1.14 This audit provides the Parliament with assurance that the Department of Education’s design and implementation of programs and initiatives to improve the access and participation of regional and remote students to higher education is effective.

Audit approach

Audit objective, criteria and scope

1.15 The objective of this audit was to assess the effectiveness of the Department of Education’s design and implementation of programs and initiatives to improve the access and participation of regional and remote students to higher education.

1.16 To form a conclusion against this objective, the ANAO adopted the following high-level criteria.

• Has an appropriate governance framework been established to manage the programs and initiatives aimed at improving the accessibility and participation of regional and remote students to higher education?
• Has the department effectively implemented the framework to monitor, report and evaluate whether the defined outcomes are being achieved?

Audit methodology

1.17 The audit methodology included:

• examination of the Department of Education’s documentation and meetings with key staff related to access and participation programs and initiatives’ governance, performance monitoring, and processes;
• examination of the Department of Education’s use of data by reviewing the department’s assurance processes. This included frameworks for capturing data and reporting required by higher education providers to inform performance and policy development;
• undertaking a walkthrough of the system and processes to document and review the arrangements in place; and
• assessing the criteria and processes used to determine program and initiative activities by reviewing the completeness and alignment of decision making and risk assessments to program objectives.

1.18 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $373,429.

1.19 The team members for this audit were Synergy Group and David Tellis.

2.1 Effective governance arrangements help to drive accountability for performance by allowing appropriate oversight of program or policy delivery, including how risks are being identified, reported and managed. Key governance actions include identifying timeframes, resources, baseline data and performance information and using a strategic risk-based approach to identify, prioritise and schedule evaluation activities.

2.2 To assess whether the department has established an appropriate governance framework to ensure access and participation outcomes are being met, that aligned with the requirements of the Public Governance, Performance and Accountability Act 2013 (PGPA Act), the ANAO examined whether the department:

• has established programs and initiatives informed by a consideration of evidence and risk;
• established appropriate governance and planning arrangements;
• established criteria and processes to determine program and initiative activities and made decisions that were documented and complete;
• established an appropriate monitoring framework; and
• designed assurance and evaluation activities to support the ongoing monitoring and reporting of programs and initiatives in achieving intended outcomes.

Were program initiatives and supporting measures informed by the consideration of evidence and risk?

2.3 The Australian Government announced numerous programs and initiatives on 19 June 2020 as part of the Job-Ready Graduates Package of reforms to higher education in response to the National Regional, Rural and Remote Education Strategy (the Strategy).¹⁸

2.4 Table 2.1 presents a breakdown of the types of programs and initiatives announced as part of the Job-Ready Graduates Package (the package). The package was made up of new initiatives, adjustments to existing programs, and adjustment to funding allocations.

Table 2.1: Regional and remote programs and initiatives overview

Note a: The Strategy further draws from evidence within in the 2017 Halsey Review.

Source: ANAO analysis based on departmental data.

2.5 As indicated in Table 2.1, four new programs and initiatives were introduced to address recommendations from the Strategy. A more detailed timeline of access and participation programs and initiatives is provided in Appendix 4.

2.6 For the new programs and initiatives announced, the evidence considered was from the Strategy, which drew upon evidence from the Halsey Review.¹⁹ The Halsey Review used an array of information sources including data from Australian Bureau of Statistics, submissions from members of the public, peak bodies, education authorities, private sector organisations and the philanthropic sector, in addition to a review of relevant literature.

2.7 The Strategy’s Expert Advisory Group used the Halsey Review and additional information sources including Australian Bureau of Statistics data, literature reviews and a series of six issues papers based on evidence including from the department’s internal analysis of Australian Census data from 2016 as part of consultation.

2.8 Four initiatives announced as part of the package were existing programs that were adjusted in response to recommendations in the Strategy. These were existing government programs focused on regional and remote access and participation. The package strengthened the focus of each program in an effort to:

bridge the gap between regional and remote students and metropolitan students, drive productivity for the regions and increase the research capacity of regional universities.²⁰

2.9 Two initiatives were adjustments to existing university funding allocations for regional and remote students. The CGS and demand driven Indigenous regional and remote student funding allocations are made to higher education providers based on the agreed number of students. CGS funding pre-dated both the Halsey and the Strategy reviews, but was identified in the Strategy as an initiative that could be expanded. Additionally, the Strategy recommended that funding was uncapped for Commonwealth-supported places for Indigenous students from regional and remote areas (Indigenous regional and remote demand-driven funding for students’ initiative) and that the CGS provide demand-driven funding for Commonwealth supported bachelor-level university places, excluding medical, for domestic students studying at regional and remote campuses.

Evidence base and overview for individual programs and initiatives

Tertiary Access Payment

2.10 The Tertiary Access Payment (TAP) was developed in response to Recommendation 2, Action 7 of the Strategy.²¹ It is a one‐off, non‐indexed, means‐tested payment of $5000 to successful applicants who are school‐leavers from outer regional or remote areas, who need to relocate for full‐time, higher level tertiary education (Certificate IV and above) at an education provider located at least 90 minutes by public transport, from their family home. Following a first stage evaluation of the TAP in 2021, the government announced that the program would be broadened to include successful applicants from inner regional areas²², who would be paid a single means-tested payment of $3000 from 2022.²³ The Department of Education is responsible for policy matters, whilst Services Australia oversees administration of the payment.

Regional Education Commissioner

2.11 The role of the Regional Education Commissioner (REC) was established in response to Recommendation 7, Action 32 of the Strategy.²⁴ The REC is intended to support effective implementation of the Strategy recommendations through partnerships and coordinate efforts across governments, local communities, education providers and employers. The role of the Commissioner is to bring together the efforts of Commonwealth, state and territory governments to provide an ongoing national focus on regional, rural and remote education outcomes. The Terms of Reference for the Commissioner list the following key duties and responsibilities for the role:

• Be a champion for regional, rural and remote education, keeping issues at the forefront for government and the community and articulating their relevance to broader policy agendas.
• Work with regional, rural and remote communities, education providers, employers and across all tiers of government and stakeholders to improve education access, participation and outcomes.
• Provide advice to the Australian Government to ensure it is achieving maximum impact and value for money from investments and initiatives in regional, rural and remote education.
• Provide advice to the Australian Government on effective and efficient strategies to improve access, participation and attainment in regional, rural and remote education in Australia.
• Advocate for the improvement of education policies spanning early childhood education and care, schools, and tertiary education to better support regional, rural and remote students.
• Engage with all tiers of government and stakeholders to improve education outcomes of students from equity backgrounds, including Aboriginal and Torres Strait Islander students and students with disability from regional, rural and remote areas.
• Oversee projects in response to the recommendations of the Napthine Review, using their findings to inform new policy suggestions to governments.
• Coordinate effort and oversee the implementation and monitoring of the Napthine and Halsey Reviews, providing a national focus for regional, rural and remote education, training and research.

Regional Partnerships Project Pool Program

2.12 The Regional Partnerships Project Pool Program (RPPPP) was established in response to findings from the Strategy, which concluded regional students experience additional challenges that may act as a barrier to aspiration and preparedness for higher education. Through the Program:

innovative partnerships will be established between Table A universities and Regional University Centres (Centres) along with other higher education providers including Table B universities and non-university higher education providers, schools, vocational education and training providers, and community organisations.²⁵

2.13 The objectives of the Program are to direct resources to:

• empower students from under-represented backgrounds in regional and remote areas to aspire to higher education; and
• support universities and Centres to develop sustainable partnerships with communities currently underserviced by existing outreach initiatives and where cumulative barriers to higher education exist.

2.14 The Program is administered under the Higher Education Support Act 2003 (HES Act) and the Higher Education Support (Other Grants) Guidelines 2022. The Program is a component of the Indigenous, Regional and Low SES Attainment Fund.

Regional Research Collaboration Program

2.15 The Regional Research Collaboration (RRC) Program was announced to enhance the research capacity of regional universities in response to Recommendation 6, Action 29 of the Strategy.²⁶ The RRC Program funds research collaboration between regional universities, industry and other higher education providers to help regional universities to develop their research strengths. The aim of the program is to encourage regional higher education institutions to develop their research capacity and research systems by teaming up with other institutions in Australia or internationally. To be eligible an institution must be a regionally headquartered Table A²⁷ provider and have a higher than average proportion of Equivalent Full-tome Student Load (EFTSL) (Commonwealth supported places) for students enrolled at their regional campuses. RRC Program projects are selected via a competitive process through annual selection rounds undertaken by the department in consultation with the Minister for Education (the Minister). Since 2021–22, a total of 11 projects have received funding through two rounds of the RRC Program.

Rural and Regional Enterprise Scholarships Program

2.16 The Rural and Regional Enterprise Scholarships (RRES) Program was established as a program in 2016 to improve access to educational opportunities for regional and remote Australians. The intent of the program is to help Australians in regional and remote areas to access higher education. The program supports students studying from Certificate IV through to PhD level.

2.17 Recipients are selected according to program guidelines and priority for receiving scholarships is granted to eligible Aboriginal and Torres Strait Islander students, students affected by natural disasters, people from areas of high unemployment, people from remote and very remote areas, women studying in fields with low female representation and people with a disability. Eligible students may receive up to $18,000 each for four years full-time study (or the part-time equivalent).

2.18 The amount is adjusted depending on the recipient’s study load, the level of their qualifications and the duration of their course. The Department of Education is responsible for policy matters, whilst Queensland Tertiary Admissions Centre (QTAC) conducts administration of the program.

Regional University Centres Program

2.19 The Regional Study Hubs program was initially established in 2018 and was adjusted and renamed the Regional University Centres (RUC) program. The program has been expanded and enhanced over time, including in response to the Strategy. The program includes a broad range of operating models which are tailored to individual community needs through the provision of resources such as study spaces, video conferencing, computer facilities and internet access, as well as administrative, academic and wellbeing support based on the student cohort and their requirements.

2.20 The Strategy recommended providing sufficient program management and governance support to ensure success. A competitive selection process was undertaken for the selection of RUCs across three grant rounds. A panel, consisting of senior departmental representatives and the Regional Education Commissioner, assessed applications and made recommendations to the Minister based on merit lists formed as an outcome of the assessment of applications. For the three grant rounds the Minister approved the panel’s recommendations. In announcing the RUC program expansion in 2020, the government also announced a scoping study to inform the selection of future RUC locations.

National Priorities Pool Program

2.21 The National Priorities Pool Program (NPPP) became a standalone program in 2021 after it was separated from HEPPP as part of reforms to equity funding.²⁸ The program provides funding to higher education providers to conduct research projects and trial initiatives to inform equity policy and practices. Research projects and trial initiatives are (amongst other things) designed to help increase the number of persons from regional and remote areas to access, participate in, remain in and succeed in higher education. While the NPPP includes the possibility of research on regional students and issues from 2021, this has not been a focus of the program in that time.

Higher Education Participation and Partnerships Program

2.22 The Higher Education Participation and Partnerships Program (HEPPP) was established in 2010 to fund universities to undertake activities and implement strategies that improve access to undergraduate courses for people from low socioeconomic status (SES) backgrounds and improve their retention and completion rates. From 1 January 2021, the HEPPP was expanded from a solely low SES focus to also include First Nations students and regional and remote students. The HEPPP was refocused to promote equality of opportunity in higher education by improving:

• outreach to widen aspiration and promote higher education to persons from a low SES background, persons from regional areas and remote areas, and Indigenous persons; and
• the extent to which persons from a low SES background, persons from regional areas and remote areas, and Indigenous persons access, participate, remain and succeed in higher education, and obtain higher education awards.

2.23 From 2021, grants were made to eligible higher education providers each year based on the provider’s respective share of domestic undergraduate students from a low SES background, students from regional areas and remote areas and Indigenous students. Prior to 2021, grants were made to eligible higher education providers each calendar year based on the provider’s respective share of domestic undergraduate students from a low SES background.

Increased Commonwealth Grant Scheme (CGS) funding for regional campuses

2.24 Under the HES Act, the CGS provides funding to public universities, and some private higher education providers in national priority areas, for the supply of Commonwealth supported places (CSP)²⁹ each year, for which students pay a ‘student contribution’ towards the cost of their course. Public universities decide how many domestic students they enrol in bachelor level courses (excluding medicine) up to a Maximum Basic Grant Amount (MGBA) as determined by the Minister for Education. On 19 October 2020, the Australian Parliament passed legislation for the Job-ready Graduates Package (the package) of reforms to higher education. The package affected the Commonwealth and student contributions and provided additional funding support for students in regional and remote Australia. CGS funding agreements with higher education providers are in place for the period 2021–2023.

Indigenous regional and remote demand-driven funding for students

2.25 From 2021, all First Nations students who live in regional and remote Australia are now guaranteed a Commonwealth supported place at a university of their choice, when they meet the eligibility requirements and are accepted into their chosen course of study. An eligible university place is a non-designated, bachelor level course at an Australian public university.

2.26 The Strategy highlighted the increased challenges and very low higher education participation rates for First Nations students in regional and remote areas. Introducing demand driven funding for these students responds to Recommendation 5 of the Strategy.

Consideration of risk at establishment

2.27 Risk is a key determinant in designing activities to effectively deliver and achieve intended outcomes. The accountable authority of a Commonwealth entity must establish and maintain an appropriate system of risk oversight and management for the entity, and an appropriate system of internal control for the entity to ensure public resources they are responsible for are properly managed.³⁰

2.28 Risks considered by the department when briefing government included feedback from the higher education sector, other government programs and funding mechanisms, state and territory governments, peak bodies, students, and public responses.

2.29 The risks associated with each option proposed to government were rated low by the department. For all programs and initiatives, the risks considered by the department prior to the announcement of the program and initiative was documented and briefed to government. Table 2.2 shows the risks identified by the department in briefings to government.

2.30 The risks considered were largely focused on reputational risk and the potential for negative response from stakeholders. There was limited consideration of the impact of the program or initiative in terms of outcomes. Additionally, risks were not considered at the package level or with a strategic lens. In particular, areas across the package such as gaps, overlaps and dependencies were not documented or briefed to government.

Table 2.2: The Department’s consideration of risk during program and initiative development

Source: ANAO analysis based on departmental data.

Alignment with Department’s Risk Management Policy and Framework

2.31 The Department of Education’s Enterprise Risk Management Policy and Framework provides structure and guidance for staff to proactively engage with identifying, managing and sharing risks in line with the department’s risk appetite, tolerance, governance, and accountability arrangements. The department’s Executive Board has responsibility for overseeing the department’s enterprise risk management arrangements.

2.32 The department’s Enterprise Risk Management Policy and Framework sets out how different risk assessment plan types are to be reviewed and reported on.

2.33 All programs and initiatives (listed at Table 2.1) are required to be reported on at a minimum every six months or more regularly depending on requirements or changes in internal or external context. Evidence that ongoing and new program risks are managed through the department’s risk management system RiskNet2 (RN2) was documented. Although risks are reported individually through departmental reporting in accordance with the guidelines of the Risk Management Policy and Framework, there is no evidence that the risks of all the programs are reported in a cohesive or interlinked way.

2.34 Table 2.3 sets out whether the risk management plans existed and whether risk reporting was done in accordance with the department’s Enterprise Risk Management Policy and Framework. All 10 programs and initiatives performed risk management activities in alignment with the department’s risk framework.

2.35 The Risk Framework states that RN2 is the enterprise risk management system, developed to record, manage and report on operational and shared risks. RN2 generates a risk rating and considers the risk matrix and treatment effectiveness as part of its assessment process through referencing a source table embedded in the system. This allows the approver of the plan to consider whether the risks are mitigated effectively within the department’s appetite and tolerance in the approver stage of the assessment.

2.36 Operational risks are identified and reviewed on an ongoing basis in RN2. All identified risks must have a risk owner and treatment owner, that is, a person with the appropriate authority and accountability to make a decision to accept a risk or not and to escalate the risk to management if it is outside the department’s risk tolerance. Operational risks are required to be monitored through respective business area risk plans and reported through risk metric reporting to departmental governance committees.

2.37 The risk management plans reviewed across the programs and initiatives were up-to-date and documented administration and implementation related risks.

Table 2.3: Risk management by program and initiative

Source: ANAO analysis based on departmental information.

2.38 Table 2.4 highlights the programs and initiatives where adjustments were made when new risks were identified in alignment with guidance in the department’s risk framework.

Table 2.4: Program and initiative changes informed by risk

Source: ANAO analysis based on departmental information.

2.39 Case study 1 shows the changes to the TAP governance and administration to mitigate risk.

Note a: Membership consists of representatives from the department and Services Australia representing the following areas: National Manager – Participation Multi-Disciplinary Team Branch; National Manager – Payment, Debt & Child Support Systems; Director – Participation Multi-Disciplinary Team; Assistant Secretary – Access and Equity Branch; Higher Education Division; and Director – Higher Education Regional Implementation Team.

Note b: Membership consists of Deputy Secretaries from the department and Services Australia.

Were appropriate planning and governance arrangements established?

Planning

2.40 The Strategy established the expected benefits of reform in access and participation programs and initiatives for regional and remote students. Consultation across the higher education sector and within government informed the development of policy and funding options for government consideration.

2.41 Following budget announcements for higher education reform in 2019, the department released a discussion paper³¹ outlining the programs and initiatives targeting support for students from regional and remote Australia. The paper states that post the COVID-19 pandemic, higher education reform commencing 2021 is essential to delivering the skills and job outcomes that will drive productivity and economic recovery and will deliver better regional outcomes, namely:

• more opportunity for regional students; and
• supporting growth corridors and economic opportunity, delivering higher productivity and a $25 billion dividend in 2050.

2.42 Following budget decisions, the department undertook planning to determine the scope of each activity and identify component projects. High-level consolidated branch planning for the 2021–22 financial year included coverage of programs and initiatives for this audit (see Table 1.1) and which components would be prioritised in which quarter.

2.43 Planning was undertaken through briefings to ministers, including grant calculations under the Higher Education Support (Other Grants) Guidelines 2022 (made under section 238-10 of the HES Act) and criteria for grants programs. Ministerial approval was obtained as part of planning before implementation of each program. There were linkages between what was originally planned, and the final activity announced as part of the package. Government announcements about each of the programs and initiatives can be traced to the final program and initiative that was implemented.

2.44 Case study 2 provides an example of implementation planning undertaken by the department at the individual program and initiative level for the Regional Education Commissioner role.

2.45 Implementation level planning in the form of discrete implementation plans and activity timelines was also in place for the RRES, RRC, RUC, RPPPP, NPPP and TAP programs and initiatives. There was no evidence of implementation planning for HEPPP. Specific implementation planning was not required for the Increased Commonwealth Grant Scheme funding for regional campuses and Indigenous regional and remote demand-driven funding for students’ initiatives due to their nature and the increase in funding allocation only.

Governance arrangements

2.46 The department has governance arrangements in place for the individual programs and initiatives. Programs and initiatives are currently managed across different areas of the department.

2.47 Figure 2.1 presents an overview of regional and remote programs and initiatives undertaken by the department and which areas hold responsibility and accountability for administering them. For TAP and RRES, the department receives administrative support from Services Australia and Queensland Tertiary Admissions Centre (QTAC) to deliver the program.

2.48 The Higher Education Access and Equity Branch includes teams such as Equity Policy and Regional Policy. The Regional Policy Team was established in mid-2019, with timing to coincide with the release of the Strategy and development of the government response and implementation. The department advised on 4 April 2023 that both the Equity Policy and Regional Policy team analyse relevant data and research for policy and program purposes.

Figure 2.1: Ownership of regional and remote programs and initiatives across department

Source: ANAO using departmental information.

2.49 The legislative basis for eight of the ten programs and initiatives is the HES Act³² and four are also subject to the Higher Education Support (Other Grants) Guidelines 2022³³, these are outlined in Table 2.5.

2.50 The Minister’s delegation instrument delegates powers to the Secretary and SES Bands 1–3. Delegates for the programs and initiatives include the Assistant Secretary, Higher Education Access and Equity Branch and the Assistant Secretary, Research Policy and Programs Branch, both in the Higher Education Division.

Table 2.5: Program and initiative reporting and administration arrangements

Note a: Terms of Reference between the REC and the department are available from https://www.education.gov.au/regional-education-commissioner/resources/regional-education-commissioner-terms-reference [accessed 6 April 2023].

Source: ANAO analysis based on departmental information.

2.51 TAP has a project Governance Committee that provides a forum for Services Australia and Education Executive to discuss matters affecting the project, including strategies and outcomes. The Committee has a Terms of Reference.

2.52 According to the governance principles for the department³⁴, the REC is formally required to report annually. The Commissioner meets with the Minister and/or the Minister’s office quarterly, or as required. To date the Commissioner has been meeting mainly monthly with the Deputy Secretary. The Governance principles were agreed between the department and the REC.

2.53 The department does not have overarching governance arrangements in place providing coordinated and consolidated governance, risk assessment and reporting across all programs and initiatives. There is benefit in establishing governance arrangements that provide oversight of how the programs and initiatives are performing against the overall access and participation outcomes for regional and remote students. A departmental internal audit completed in November 2022 recommended that the regional and remote programs and initiatives be governed as a portfolio in order to readily demonstrate the overarching achievement of Strategy initiatives and policy intent. The department agreed to the recommendation and noted in its management response:

The department agrees that there would be value in considering a portfolio approach to reporting on the Napthine measures, noting that no concerns were found with the implementation approach for individual measures and that the measures only involve implementation of the measures by one Group (HERI) in the Education portfolio, with the activities of Services Australia being covered by a Memorandum of Understanding with Education that include service level measures.

2.54 The department is in the process of developing tools to assist in effectively tracking the progress and achievement of the Strategy response at a package level over time.

Were criteria and processes used to determine program activities and key decisions complete and documented?

2.55 Criteria and processes used to determine program and initiative activities were reviewed by the ANAO. Activities were largely developed from the programs and initiatives announced by the government as part of the package. Where changes were made to a program or initiative, these were made with Ministerial approval.

2.56 For the following programs (HEPPP, NPPP, RPPPP, RRC, RUC and RRES), Ministerial approval was sought and received for program design, program criteria, guidelines, grant amounts and selection of successful competitive grants applications.

2.57 For the adjustments to the two funding allocations for CGS and Indigenous remote demand-driven funding for students, the department sought approval to make the funding adjustments under the HES Act.

2.58 For the REC, the department sought Ministerial approval for the Commissioner’s appointment, work program and terms of reference. As a significant (but non-statutory) appointment, the Prime Minister and Cabinet were consulted in the appointment of the REC. For the TAP, the department sought Ministerial approval for program design, program criteria, guidelines and program changes.

Documentation of decision and record keeping

2.59 The department’s Information Management Policy specifies requirements for the creation, ongoing management, and disposal of the department’s information and records. According to the policy, the department’s official record keeping systems include Content Manager, RiskNet2 and SharePoint. Ministerial decisions and final documents reviewed as part of this ANAO audit were saved on Content Manager and other official record keeping systems.

2.60 The working documents were maintained in the shared network drives for five of the programs and initiatives, which is inconsistent with guidance from the department’s Information Management Policy on record keeping and document storage. These programs and initiatives included: Increased Commonwealth Grant Scheme funding for regional campuses; RRES; RUC; RRC and Indigenous regional and remote demand-driven funding for students.

2.61 The use of shared network drives was also observed in the November 2022 internal audit that reviewed the REC, RUC and RPPPP programs. The audit found that:

There is an opportunity for program areas to comply with record keeping requirements by transitioning information and records relating to their program in shared network drives to official record keeping systems in alignment with the department’s Information Management Policy. Discussions with business areas responsible for the two Napthine measures sampled identified that both areas use shared network drives for working documents, which is not an official record keeping system. However, both business areas advised that final and approved documents are maintained in the department’s official record keeping systems, including Content Manager, SharePoint and RiskNet 2 (RN2).

2.62 With relation to the use of shared network drives, the Information Management Policy states:

The department aims to transition all Shared drives (Team, Branch, Group, Special Drives, G:\ H:\ I:|\ O:\) to be read only. Any information of ongoing value on a shared drive should be identified and maintained in a repository identified for official business information. Any information that requires updating or amending should be saved in an approved system. The department aims to transition all share drives to read only by 30 December 2019, however, all new or modified information should be stored in the systems identified in this reckoner from the policies date of effect. Information stored in email folders, shared drives, personal drives or external storage media is considered as unofficial and is unmanaged.

2.63 In response to the improvement opportunity identified in the internal audit, the department implemented a new SharePoint site to store key information related to the Strategy response programs and initiatives which is consistent with the guidance within the department’s Information Management Policy.

2.64 The Regional Policy Team maintains a record keeping register which highlights relevant content manager (TRIM) files, risk plans and SharePoint sites. The transition to SharePoint and the checking of records was in progress at the time of the internal audit finalised in November 2022.

2.65 Additionally, for the RRES program, a policy tracker was established in 2021 and used to record key decisions in relation to the program, and meeting notes from monthly contract management meetings between the department and QTAC. This was in response to the outcomes of an internal audit conducted in 2021. Further information on the internal audit is provided in Table 3.2.

Was an appropriate performance measurement framework established?

2.66 Establishing a framework for performance measurement over the package provides transparency and enables high-level monitoring of its performance, providing reliable, relevant and complete performance measures are developed in alignment with intended outcomes.³⁵

2.67 Performance information, and how it is used to demonstrate whether public funds are making a difference and delivering on government objectives, is a key element of the PGPA Act.³⁶

2.68 The three intended outcomes and targets outlined in the National Regional, Rural and Remote Tertiary Education Strategy (the Strategy)³⁷ are shown in Table 2.6.

Table 2.6: Strategy outcomes and targets

Note a: The attainment rates measure award course completions by regional or remote students over all domestic award course completions.

Note b: The participation rates measure all regional or remote students over all domestic onshore students.

Source: Department of Education, National Regional, Rural and Remote Tertiary Education Strategy – Final Report, 2019, p. 8.

2.69 Figure 2.2 provides an overview of regional and remote student baseline data collected in 2016 contained within the Strategy. This highlights the 2016 differential in attainment and participation that informed the creation of the targets.

Figure 2.2: 2016 baseline data for targets within the Strategy

Notes: Target One: By 2030, halve the disparity between metropolitan and each regional and remote location category for Cert IV and above attainment rates for 25–34 year-olds.

 Target Two: By 2030, halve the disparity between metropolitan and each regional and remote location category for attainment rates of higher education for 25–34 year-olds.

 Target Three: By 2030, halve the disparity between metropolitan and each regional and remote location category in the rate of participation for Cert IV and above by work age Australians (15–64 years).

 The Australian Statistical Geography Standard (ASGS) defines remoteness areas derived from the Accessibility/Remoteness Index of Australia Plus (ARIA+) produced by the Hugo Centre for Population and Migration Studies at the University of Adelaide. Remoteness areas divide Australia into five classes of remoteness based on relative access to services: major cities, inner regional, outer regional, remote and very remote. The terms ‘major cities’ and ‘metropolitan’ have both been used, depending on context. Both refer to the same location as captured in the major cities category in ASGS.

Source: ANAO analysis based on Department of Education, National Regional, Rural and Remote Tertiary Education Strategy – Final Report, 2019 data, p. 58.

2.70 Figure 2.3 represents the 2016 baseline and target percentage point differentials between regional and remote students and metropolitan students for achievement by 2030.

Figure 2.3: 2016 baseline and target percentage point differentials for Strategy outcomes

Notes: Target One: By 2030, halve the disparity between metropolitan and each regional and remote location category for Cert IV and above attainment rates for 25–34 year-olds.

 Target Two: By 2030, halve the disparity between metropolitan and each regional and remote location category for attainment rates of higher education for 25–34 year-olds.

 Target Three: By 2030, halve the disparity between metropolitan and each regional and remote location category in the rate of participation for Cert IV and above by work age Australians (15–64 years).

 The Australian Statistical Geography Standard (ASGS) defines remoteness areas derived from the Accessibility/Remoteness Index of Australia Plus (ARIA+) produced by the Hugo Centre for Population and Migration Studies at the University of Adelaide. Remoteness areas divide Australia into five classes of remoteness based on relative access to services: major cities, inner regional, outer regional, remote and very remote. The terms “major cities” and “metropolitan” have both been used, depending on context. Both refer to the same location as captured in the major cities category in ASGS.

Source: ANAO analysis based on Department of Education, National Regional, Rural and Remote Tertiary Education Strategy – Final Report, 2019 data, page 58.

Program performance measurement

2.71 At a package level, the department has not established a performance measurement framework that includes intermediate specific targets to measure success or demonstrate progress towards 2030 targets in the Strategy for access and participation programs and initiatives for regional and remote students in achieving Strategy related outcomes.

2.72 An internal audit report completed in November 2022 (focusing on the REC, RUC and RPPPP) noted the following with regard to Program level performance measurement:

There would be benefit in a more structured approach to oversee and report on the implementation of the portfolio of initiatives (page 3).

The department has not yet implemented a structured approach for overseeing and reporting on the implementation of initiatives from the Napthine Review at a portfolio-level. This includes lack of defined key stakeholders, methodologies for measuring performance and achievement of policy objectives, data and information requirements and quality assurance mechanisms to ensure reporting is complete and reliable (page 21).

2.73 The report included a recommendation that the department develop, document and implement processes to support the development of the REC Annual Report, including the following:

• stakeholder engagement plan, including defining key stakeholders and frequency of engagement;
• defining a performance measurement methodology, key performance measures and targets that can be used to demonstrate achievement of policy intent for the REC, RUC and RPPPP initiatives;
• defining data and information requirements to support the agreed performance measurement methodology; and
• quality assurance mechanisms to review and validate information to ensure the Annual Report is accurate, complete and reliable.

2.74 The department agreed to the recommendation and is currently developing tools to assist in effectively tracking the progress and achievement of the Strategy response at a program level over time. The Stakeholder Engagement Strategy was due by 30 March 2023 and the Performance and Data framework was due by 30 April 2023. Both artefacts were developed and approved in April 2023 but are yet to be implemented by the department.

2.75 The Performance and Data Framework states:

Performance against the individual measures and the overall Napthine targets will be tracked and recorded as new data, including as Census data becomes available, in conjunction with the work undertaken by the Regional Education Commissioner Policy Secretariat in delivering the Commissioner’s Annual Report to Government.

The success of each individual Napthine measure is based on its ability to contribute towards the aim, objectives and targets outlined above. However, in measuring performance, consideration must be given to the specifics of each measure, including the individual context in which it operates, its key stakeholders, and the availability and longevity of data and information.

Regional Education Commissioner

2.76 The Regional Education Commissioner (REC) role was established following the Strategy in part to coordinate efforts and oversee the implementation and monitoring of recommendations from the Napthine and Halsey Reviews, providing a national focus for regional, rural and remote education, training and research.

2.77 As the role was established in 2021, the first annual report to the Minister for the role was delivered in November 2022.

2.78 The REC Annual Report provides an update on implementation and outcomes relating to individual programs and initiatives. The November 2022 annual report included implementation and short-term outputs for each Strategy measure and did not include information on performance against targets or Strategy outcomes. The Terms of Reference state:

The Commissioner will report annually to the Minister for Education, and the Assistant Minister for Education, as required, on the state of education in the regions. The annual report will include:

• an update on progress of measures taken in response to the Napthine and Halsey Reviews;
• examples of successful existing initiatives, highlighting cases of best practice;
• an analysis of the current, emerging and future education needs of regional, rural and remote areas;
• an examination of trends in regional, rural and remote student outcomes;
• advice on improving tertiary student achievement, access and attainment using relevant indicators across all education sectors;
• a summary of outcomes of projects undertaken in response to the Napthine Review, including any policy recommendations;
• a summary of stakeholder engagement and consultations undertaken; and
• responses to other issues, as required.

Performance measurement for individual access and participation programs and initiatives

2.79 For the individual programs and initiatives, the requirements of funding recipients and providers were defined. However, performance requirements are based on the achievement of milestones and delivering outputs, not linked back to intended outcomes.

2.80 For programs with a Conditions of Grant agreement in place, the department has documented the process undertaken for the awarding of funding and its ongoing management, as well as how the program and initiative is addressing requirements contained within the guidelines. Table 2.7 provides an overview of program and initiative level performance measurement activities. The table includes key performance measures or targets for some of the programs or initiatives as documented in the Performance and Data Framework developed by the department in April 2023.

3.1 In accordance with guidance from the Department of Finance’s Commonwealth Performance Framework (the framework) and Evaluation in the Commonwealth – Resource Management Guide (RMG) 130, the effective implementation of a monitoring, reporting and evaluation framework will enable the department to manage and monitor its programs and initiatives with higher education providers.³⁸

3.2 Key areas to consider during implementation planning include project scheduling and management, governance, stakeholder engagement, risk management, monitoring and evaluation (reflected in an evaluation plan), and resource management.

3.3 To determine whether access and participation programs and initiatives are monitored, reported and evaluated in accordance with the guidance outlined in the framework, the ANAO examined whether:

• the department established an appropriate implementation schedule with program milestones and review points;
• milestones were monitored;
• data collection and reporting processes were implemented to support ongoing reporting against program outcomes;
• assurance and evaluation activities were designed, conducted and reported on; and
• reporting and escalation of program related issues were in accordance with the department’s risk management protocols.

Did the department establish an implementation schedule with program milestones and review points and were milestones monitored?

Program and initiative implementation schedules

3.4 The National Regional, Rural and Remote Tertiary Education Strategy (the Strategy) states that:

No single recommendation or action can solve the education challenge facing RRR [regional, rural and remote] Australia on its own. The additional barriers facing individuals in RRR areas are multifaceted and require cultural and social shifts that cut across governments, communities and families. It will be a combination of actions taken together that will shift the tide. Recently, there has been significant investment in initiatives to provide RRR students with greater choice and access to opportunities. However, given the scale and complexity of the problem, more needs to be done.³⁹

3.5 A consolidated implementation schedule for the programs and initiatives reviewed as part of this audit was not established by the department. Of the ten programs and initiatives reviewed, four were established in some form prior to the Strategy (program timeline at Appendix 4 of background refers), and the other six were new programs and initiatives established in response to the Strategy. The department established the Job-ready Graduates Implementation plan and Strategy timeline which provide high-level coverage over the implementation of five out of 10 programs and initiatives.⁴⁰

3.6 The current delivery model of access and participation programs and initiatives being implemented and delivered individually without an overarching framework creates a risk that there is no common awareness over shared stakeholders, risks and mitigations, key milestones, and dependencies.

Program and initiative level implementation schedules

3.7 The department has developed implementation schedules on an individual program and initiative basis for the RUC Program and REC measure.

3.8 For five of the ten programs and initiatives, the outcomes, objectives, and milestone reporting requirements for each funding recipient were documented within the Conditions of Grant. Refer to Table 2.7.

3.9 For three out of the 10 programs and initiatives — Increased CGS funding for regional campuses, National Priorities Pool Program (NPPP), and Indigenous regional and remote demand-driven funding for students — there was no provider level implementation schedule in place due to the nature of the initiative.

3.10 For the Regional Education Commissioner, there was a Minister endorsed forward work plan (including priorities and timeframes), guiding principles and terms of reference which contained the elements included in an implementation schedule.

Review Points

3.11 The department collects information and data related to nine of the programs and initiatives as required in each of the Conditions of Grant, service schedule for TAP and the provider agreement with Queensland Tertiary Admissions Centre (QTAC) for the RRES Program. The exception is that no data is collected from the Regional Education Commissioner. The department’s data collection for programs and initiatives is shown in Table 3.1

3.12 The Department of Finance has developed the Resource Management Guide Evaluation in the Commonwealth (RMG 130) as a guide for conducting program evaluations. The Resource Management Guide states that:

undertaking performance measurements during program implementation allows the Department to determine whether the program or activity is achieving the expected results.⁴¹

3.13 At the overarching program and initiative level, there are no review points that have been documented or undertaken to date that target access and participation outcomes for regional and remote students or the package of programs and initiatives announced in response to the Strategy.

3.14 The REC advised the ANAO on 16 March 2023 that:

The department is committed to a high-level Review of Higher Education Equity Policy and Programs. The high-level review of equity programs should give a broad sense on how a number of the Napthine measures are tracking and will be timed to ensure it is able to be considered by the Accord Panel as part of their work.

Provider level review points and milestones

3.15 The Higher Education Support Act 2003 (HES Act) requires each higher education provider to provide the Minister with a financial statement for each annual financial reporting period in which the student of a provider receives assistance under the HES Act.

3.16 For the seven grants programs and the RRES program, financial information is provided annually by the relevant providers (and by Services Australia and QTAC for the relevant programs) to the department for acquittal purposes.

3.17 Milestone reporting is an element of a Conditions of Grant between the department and a higher education provider. The milestone reporting forms the basis for the department to provide future funding. Program specific assessment templates are developed by the program teams as relevant, in administering the relevant Conditions of Grant, as requirements are often tailored to the specific project, program or recipient.

3.18 Funding Agreements or Conditions of Grant have been developed for five programs and initiatives where required. The specific programs and initiatives that have a Funding Agreement or Condition of Grant in place are outlined in Table 2.7. These key control documents describe the outcome and output requirements of the funding to the higher education providers. These requirements are monitored by the department through the receipt of financial reporting, progress reports and deliverables as outlined in the Condition of Grant.

3.19 For the RPPPP, due to the nature of Phase 1 objectives (3-month co-design ahead of a competitive bid for further funding), the only requirement for Phase 1 progress reports was a submission to the department. Phase 1 progress reports were reviewed to ensure all sections of the template were completed, and this was tracked by the department using a tracking spreadsheet. The requirements also included participation in a Phase 1 forum and the submission of a Final Report in the form of a Phase 2 bid. These elements were also documented and tracked using the spreadsheet.

3.20 The NPPP process for assessing provider compliance with Conditions of Grant requirements includes using a tailored departmental assessment template to assess progress and financial reports aligned to the requirements contained within the program guidelines.⁴²

3.21 With the RRC program, the department issues progress and financial report templates to all universities receiving RRCP Commonwealth grants. Conditions of Grant requirements are incorporated in the template structure, so that universities report against their agreed deliverables. As each RRCP project has different deliverables, the standard template is tailored to individual projects and information is pre-populated where necessary.

3.22 For the RUC program, the assessment process is similar to the RRC program with the utilisation of a tailored assessment template.

3.23 Once the assessment process has been undertaken by program areas for programs with Conditions of Grants in place, they are submitted for delegate approval to finalise the process and initiate payment.

3.24 The remaining programs and initiatives are not administered under a Condition of Grant and therefore do not follow similar assessment and approval processes. For CGS and Indigenous remote demand-driven funding for students there were no milestones and payment is wholly based on formulas outlined within the HES Act. With relation to the TAP and RRES programs, the department receives and monitors reporting provided by QTAC and Services Australia in accordance with the relevant services schedule and contract that also outlines service levels and conditions for payment.

Were data collection and reporting processes implemented to support ongoing reporting against program outcomes?

Higher education data

3.25 The department is responsible for the collection and dissemination of statistics relating to the provision of higher education at all Australian institutions.⁴³

3.26 Data in the Higher Education Statistics Collection is disseminated through statistics publications, datasets, tabulations, analyses prepared for clients, and reports. The data is used by both internal and external stakeholders to government.⁴⁴

Tertiary Collection of Student Information

3.27 The Tertiary Collection of Student Information (TCSI) is a framework for the collection of data from higher education and VET Student Loan providers, which is used to inform policy and programs, produce publications, and assist Services Australia with the administration of student payments.⁴⁵

3.28 Student data is collected under the HES Act, the VET Student Loans Act 2016 and the Tertiary Education Quality and Standards Agency Act 2011.

3.29 All higher education providers that are approved under the HES Act are required to report data for the Higher Education Data Collection.

Activity and program level data collection

3.30 The department has additional mechanisms in place to obtain specific activity level data from higher education providers through the reporting requirements set out in each higher education provider and RUC’s Conditions of Grant. Conditions of Grant arrangements were in place for four of the 10 programs and initiatives, as outlined in Table 2.7.

3.31 The six programs and initiatives that did not have a Conditions of Grant were the CGS funding for regional university campuses initiative, the Indigenous regional and remote demand-driven funding for students’ initiative, and the TAP, HEPPP, REC and RRES programs. There is a signed provider agreement in place with Services Australia for the TAP and with QTAC for the RRES for the provision of services and a Terms of Reference with the Regional Education Commissioner.

3.32 The data for the programs and initiatives that have a Condition of Grant in place is obtained through milestone reporting and is used for administration and acquittal purposes. Program administrators receive data that assists in program verification and acquittal reconciliation for all required programs and initiatives with a Condition of Grant agreement in place.

3.33 There are two programs that are administered by external agencies. Administration processes for the TAP are undertaken by Services Australia and the agency provides program level reporting to the department on the application status and payments, while the RRES program is administered by QTAC on behalf of the department. Under the contract, QTAC is required to provide the department with an annual report, application outcomes report, scholarship outcomes report and periodical progress reports for each scholarship cycle.

3.34 All other programs and initiatives within the package administered by the department do not have review points or milestones and are reliant on formulas within the HES Act to allocate funding amounts.

3.35 Data is captured at the individual program and initiative level for nine of the 10 programs and initiatives reviewed, with data from the higher education providers and RUCs relating to student demographics (student equity and participation) and financial expenditure data being collected.

3.36 Table 3.1 describes the data being collected for each program and initiative and what, if any, analysis is undertaken.

Table 3.1: Program and initiative level data and analysis

Note: Program acquittal refers to reporting of progress against funded activities undertaken by the funding recipient. Financial acquittal refers to reporting of financial expenditure against funding requirements (and may include audited financial statements) provided by the funding recipient.

Source: ANAO analysis using departmental data.

3.37 The department does not undertake additional analysis of the data (to what is outlined in Table 3.1) to determine whether the programs and initiatives are achieving outcomes outlined in the Strategy.

3.38 The department has not generated performance information against the targets in the Strategy to measure its progress or trajectory for success. The department has not reported on its progress against the baseline or target data since the release of the Strategy in 2019 (which used 2016 data). For the newer programs and initiatives established out of the Strategy, there are limitations to the data currently available to measure progress.

3.39 The department is currently developing the reporting requirements for Phase 2 of the RPPPP. These are anticipated to include performance measures relating to the delivery of outreach initiatives and will include requirements for self-monitoring and evaluation in line with the Student Equity in Higher Education Evaluation Framework (SEHEEF). Data reporting as part of these requirements will be in line with the SEHEEF and will be used to monitor progress against project and program objectives.

3.40 The evaluation undertaken over the TAP program in 2021 highlighted that capturing data and performance reporting analysis was an issue:

While KPIs currently exist to monitor TAP administration (e.g., promotion, payment, and reporting), there is an opportunity to identify KPIs to monitor the progress of TAP in achieving its intended outcomes, and in relation to other Australian Government initiatives with similar widening tertiary participation outcomes.

3.41 The HEPPP program evaluation further identified that universities should be provided with a clear incentive to target HEPPP projects at low socio-economic status (low SES) students. This is to ensure that the HEPPP projects are additional to what universities would otherwise deliver in university student support.⁴⁶ The evaluation report stated that:

The current HEPPP Participation component does provide some incentive for universities to enrol and retain low SES students (as, all things being equal, funding increases for each low SES student a university has enrolled), but current practice suggests that this could be strengthened through improved targeting. Second, incentives around success and completion could also be strengthened — success and completion rates for low SES students continue to track below those of medium and high SES students. Third, more specific data on whether Participation activities are impacting low SES students’ retention, success and completion rates would help universities better evaluate and determine which types of equity projects are the most effective to implement.

Were assurance and evaluation activities designed, conducted and reported on?

3.45 Assurance activities and program evaluations perform a vital role in determining the performance of a program or initiative and providing insights into future program design changes.

Assurance activities

3.46 The department has established a range of internal assessment processes across programs including a self-assessment of quality assurance. A Higher Education, Research and International Assurance Framework and Strategy was developed in October 2022, but has not yet been implemented. The purpose of this Assurance Framework and Strategy is to support the implementation of a consistent approach to assurance across the Higher Education, Research and International (HERI) Group (the Group) within the Department of Education. The Framework is intended help to identify what assurance is, how assurance activities can be implemented, and how assurance can be measured in activities and programs across the Group. The Framework and Strategy are supported by an Assurance Framework Guidance document and an Assurance Maturity Assessment Tool.

3.47 Some programs and initiatives have been subject to review as part of internal audits conducted at the department in the past two years. These are outlined in Table 3.2.

Table 3.2: Overview of internal audits for programs and initiatives

Note: The department advised on 4 April 2023 that the Australian Universities Accord is undertaking a review of the Job-ready Graduates package which includes CGS funding arrangements and Indigenous regional and remote demand-driven funding for students.

Source: ANAO analysis based on departmental information.

3.48 The TAP and RRES programs had an internal audit undertaken in October 2021. Three recommendations were raised relating to TAP administration and completeness of RRES reporting. All recommendations have now been closed and endorsement of audit closure from the department’s Chief Audit Executive. The ANAO did not review the implementation of recommendations as part of this performance audit.

3.49 The RPPPP, RUC and REC programs were reviewed through an internal audit in November 2022. The internal audit made one recommendation to embed a portfolio approach to oversee and report on the implementation of Strategy initiatives. This included consideration of a stakeholder engagement plan, defining a performance measurement methodology, key performance measures and targets, defining data and information requirements and quality assurance mechanisms. Further detail is provided in paragraphs 2.71–2.74.

Evaluation activities

3.50 The Commonwealth Performance Framework states that evaluation activities help to understand, drive and demonstrate the performance of government programs and initiatives. It can also help to confirm that expected results are being achieved and risks are being managed. In particular, it is good practice to plan how government programs and initiatives will be monitored and evaluated from the start, and to collect robust performance information that supports continuous improvement, the management of risk, accountability and decision-making at all stages of a policy cycle. ⁴⁷

Using evaluations to support decision-making and improvements in program and initiative administration

3.51 Since the programs and initiatives were established and implemented, the department has used evaluations on an ad-hoc basis to inform changes to programs and initiatives.

3.52 While there was no strategic approach to evaluation in place at a portfolio level, the department provided examples of where evaluations have been conducted, and findings used to inform key activity changes. For example:

• Recommendations from the 2021 TAP evaluation contributed to considerations and subsequent policy changes to the TAP made by the government during 2021. Refer to Case study 2 in Chapter 2. These changes included:
  • broadened eligibility criteria;
  • administration of the program moving to Services Australia;
  • payments being made on student-led demand; and
  • the introduction of differential payments of $5000 for students from outer regional and remote areas, and $3000 for students from inner regional areas.
• In response to the HEPPP review undertaken by ACIL Allen in 2017, the department engaged the Institute for Social Science Research at the University of Queensland to design the Student Equity in Higher Education Evaluation Framework (SEHEEF). The SEHEEF (not part of the ANAO’s audit) was designed to measure and monitor the impact of HEPPP programs and is in the process of being implemented.⁴⁸
• A first stage evaluation of the RUC program has been completed and the Minister was briefed on the Final Report in November 2021. The report included improvement opportunities and action items for the program relating to program and model effectiveness, program implementation, program appropriateness and financial analysis. The most recent update on these was in September 2022 in which 11 actions were marked as complete, and five still ongoing. The outstanding actions include the development of a data strategy, analysis of enrolment trends, the finalisation of outcomes relating to the completed research project on RUC-university partnerships, and consideration of longer term funding mechanisms for enabling programs. Stage two of the RUC program evaluation is planned for 2023–24.

Evaluating Program and Initiative Outcomes

3.53 The department has not established an evaluation framework to assess the outcomes of access and participation programs and initiatives.

3.54 The Terms of Reference for the Australian Universities Accord Review (to be undertaken in 2023) states that one of the key areas for review is:

Access and Participation

Improve access to higher education, across teaching, learning and research. This will include recommendations for new targets and reforms to support greater access and participation for students from underrepresented backgrounds (including First Nations Australians, those from low socio-economic backgrounds, people with disability, and regional and rural Australians).

3.55 The Accord may consider programs and initiatives supporting students to access and participate in higher education, including a number of programs and initiatives examined in this audit.⁴⁹

3.56 The Department of Education’s website states:

The Australian Universities Accord review process will build a long-term plan for Australia’s higher education system and is the first broad review of the higher education system since the 2008 Review of Higher Education (the Bradley Review). This review will examine equity outcomes including access and participation for RR [regional and remote] students.⁵⁰

3.57 The government has committed $2.7 million over two years from 2022–23 to deliver the Australian Universities Accord through a 12 month review of Australia’s higher education system, led by a panel of ‘eminent Australians’ (the Panel).⁵¹

3.58 The objective of the Accord is to devise recommendations and performance targets that will improve the quality, accessibility, affordability and sustainability of higher education, in order to achieve long term security and prosperity for the sector and the nation.

3.59 The ANAO was advised by the department on 17 February 2023 that the department is awaiting the outcomes of the Accord review before changes to the access and participation programs and initiatives will be enacted. The REC advised the ANAO on 16 March 2023 that:

the programs and initiatives are still young and data for many measures is limited which reduces the effectiveness of undertaking an outcomes-based evaluation at this point (or earlier).

3.60 As noted in Case study 2 in Chapter 2, the role of the REC is to coordinate effort and oversee the implementation and monitoring of the Strategy recommendations. A key deliverable of the role is to oversee a formal evaluation of all programs and initiatives taken in response to the Strategy in 2022–23. The REC advised the ANAO in March 2023 that evaluation of the Strategy programs and initiatives has been deferred to 2024.

Program and initiative level evaluations

3.61 Evaluation activities funded by the department have been undertaken on individual programs and initiatives to varying degrees. Evaluations conducted of programs and initiatives are outlined in Table 3.3.

Table 3.3: Evaluations undertaken at the program and initiative level

Source: ANAO analysis based on departmental information.

3.62 The TAP (2021), RUC (2021), HEPPP (2017) programs have each been subject to review through an evaluation commissioned by the department and undertaken by an external service provider.

3.63 Increased CGS funding for regional campuses, Indigenous regional and remote demand-driven funding for students, REC, RRES, RRC, RPPPP and NPPP, have not been subject to an evaluation.

3.64 A second stage evaluation and assessment of the TAP program is proposed to consider the longer-term impacts of the program, including its impact on higher education access and participation. This next evaluation of the TAP program is scheduled to take place in the 2023–24 financial year.

3.65 The department plans to undertake an evaluation of the RRC Program in 2024. The evaluation is expected to examine the effectiveness of the Program in delivering the outcomes specified in Section 2 of the program guidelines. The purpose of the evaluation is to provide evidence of the benefits delivered through the RRC Program objectives.

3.66 NPPP has not been evaluated in its current form (a standalone program since 2021). The previous iteration (as a subset of HEPPP) was evaluated in 2017 as part of the overall ACIL Evaluation.

3.67 The assurance and evaluation activities identified to date have been undertaken on an ad-hoc basis and have not been considered as part of a broader strategic approach to package evaluation and assurance.

Were the reporting and escalation of program related issues done in accordance with the department’s risk management protocols?

3.68 As discussed at paragraphs 2.31 to 2.39, all programs and initiatives reviewed are managed through the department’s Enterprise Risk Management Policy and Framework and risks are recorded within risk management plans that are lodged in the department’s RiskNet2 system. Risk management plans had been established for all 10 programs and initiatives.

3.69 Although risks are largely reported individually in accordance with the guidelines of the Risk Management Policy and Framework, there was no evidence that the risks of all activities are reported in a cohesive and interlinked way (see paragraph 2.33). Consequently, risks against the achievement of activity objectives and issues have not been considered by the department.

3.70 The risk management of programs and initiatives is undertaken by program directors and administrators within the department.

3.71 For both the TAP and RRES programs, the department has accountability for risk management arrangements, with third parties responsible for administering the program. Table 3.4 outlines the roles in risk management arrangements between the department and its service providers.

Table 3.4: Programs and initiatives with shared risk management arrangements

Source: ANAO analysis using departmental data.

3.72 The TAP Governance Committee (comprising the department and Services Australia) is responsible for establishing and monitoring the progress of the TAP Services Schedule, as well as key risks and mitigation strategies. The TAP Services Schedule was executed in September 2022 and includes key performance indicators (KPIs) to track performance of Services Australia’s administration of the payment.

3.73 RRES has a provider agreement (contract) in place with Queensland Tertiary Admissions Centre (QTAC) in relation to the administration of RRES Program. The contract assigns obligations to QTAC as a scholarship support organisation.

3.74 The department is responsible for program management, while QTAC is responsible for supporting running of the scholarship program, including:

• provision of documentation for the department’s approval (including implementation plan, risk plan, stakeholder engagement, and communications plan);
• management of scholarship funding as ‘other CRF’ money as required under the Public Governance, Performance and Accountability Act 2013;
• provision of reporting to the department including implementation of the program and achievement of program outcomes; and
• assessing scholarship applications against the eligibility criteria as set out in the Program Guidelines.

3.75 QTAC developed a risk management policy and then QTAC together with the department developed a risk management plan that provides a comprehensive description of all potential risks associated with administering RRES Program.

3.76 The department manages program risks in the RiskNet2 system and identifies and escalates risks as required with the First Assistant Secretary and the Minister through their office in line with the department’s Enterprise Risk Management Policy and Framework.

3.77 For all programs and initiatives, program areas raise risks and issues with the Assistant Secretary, First Assistant Secretary and, where appropriate, to the Minister through their office. Further detail on risk management and reporting lines is provided in Table 2.3 and 2.5. The department advised the ANAO in April 2023 that risks are escalated to the Executive through updating of the Higher Education Division’s issues register and meetings with the Branch and Division Executive. The department provided examples from July and December 2022 of how outcomes of these meetings and the actions items from the issues register are documented.

3.78 Briefing to the Minister includes a section on identification of the risks relating to the subject matter.

Appendix 1 Entity response

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021–22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;
• introducing or revising policies, strategies, guidelines or administrative processes; and
• initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

• Stage two RUC program evaluation planned for 2023–24 (see paragraph 3.52).
• a review looking into the CGS and Indigenous regional and remote demand-driven funding for students has been planned for 2023 as part of the Accord review process and initiatives from the Job-ready Graduates package (see paragraph 3.58 and Table 3.2).
• as a key deliverable of the role, the REC will oversee a formal evaluation of all programs and initiatives taken in response to the Strategy (postponed to 2024) (see paragraph 3.59 and Table 2.7).
• In response to internal audit recommendations, the department is currently in the process of developing tools to assist in effectively tracking the progress and achievement of the Strategy response at a program level over time. A collective stakeholder engagement plan and the Performance and Data Framework have been developed and approved, but yet to be implemented (see paragraph 2.74 and Table 2.7).

Appendix 3 Overview of the National Regional, Rural and Remote Education Strategy and Government’s Response

Context

1. In November 2018, the Minister for Education announced that the government would develop a National Regional, Rural and Remote Education Strategy (the Strategy⁵²), also known as the Napthine Review, released by the Government on 28 August 2019, focused on improving tertiary education participation and outcomes for students from regional, rural and remote (RRR) areas, as part of a broader Regional Education package.

2. The development of the Strategy built on the Government’s response to the Independent Review into Regional, Rural and Remote Education (the 2017 Halsey Review⁵³).

3. Recommendation 11 of the Halsey Review proposed that the government:

• Establish a national focus for regional, rural and remote education, training and research to enhance access, outcomes and opportunities in regional Australia.

4. In addition to the above, the Strategy was also to consider how to build on initiatives that support regional education, including:

• supporting the performance of regional universities and vocational education providers;
• enhancing the role of regional education providers in the economic development of regional centres; and
• attracting people to the regions.

Review Approach

5. A Regional Education Expert Advisory Group was established in 2018 to lead the Strategy work and members were appointed by the Minster of Education. The members of the Regional Education Expert Advisory Group included:

• The Hon Dr Denis Napthine, former Victorian Premier (Chair);
• Emeritus Professor Peter Lee, former Vice Chancellor Southern Cross University;
• Ms Caroline Graham, Chief Executive Officer, Regional Skills Training; and
• Ms Meredith Wills, former Director, Geraldton Universities Centre.

6. To support the work of the Expert Advisory Group, the department established a small taskforce, and under their direction the department prepared the Strategy.

7. The Expert Advisory Group:

• engaged with selected stakeholders including national peak professional organisations, education authorities, universities and vocational education and training providers; and
• met as required, through teleconference and face to face meetings to provide advice and steer the work of the departmental taskforce.

8. The Terms of Reference for the Expert Advisory Group⁵⁴, outlined the purpose as:

• to build on the work of the Halsey Review and investigate the need for a commissioner to oversee the implementation of the Strategy;
• identify strategies and policies for the Australian Government to put in place to bring the attainment rate of RRR students to parity with students from metropolitan areas;
• examine the literature and research on aspiration, access and success for regional students and the barriers to access to tertiary education facing people in regional Australia;
• leverage research undertaken on the provision of tertiary education in regional Australia and the support services provided to RRR students at university;
• consider the current policy options supported by the Government, including student income support, as well as the opportunities presented by the Regional University Centres Program;
• recommend how current and potential new approaches should be best combined as a coherent suite of policy responses to address issues particular to a region, to deliver improved higher education outcomes to students in RRR Australia; and
• recommend a target for regional and remote education outcomes.

9. The Strategy stated that:

The Strategy is the culmination of a detailed literature review, public submissions and responses to a series of issues papers, face-to-face consultations around Australia, and analysis of available data and evidence.

Review Outcomes

10. At the core of the Strategy is a vision of a tertiary education system which:

supports equal opportunity and access for individuals from RRR areas. This vision is underpinned by several core objectives and targets, specifically focused on halving the current disparity between RRR and metropolitan students in relation to tertiary education attainment and participation by 2030.⁵⁵

11. The Advisory Group provided the National Regional, Rural and Remote Tertiary Education Strategy final report (the Strategy) in June 2019, and it was released by the Government in August 2019.

12. The Strategy included seven recommendations and proposed 33 related actions.

Australian Government’s Response to the National Regional, Rural and Remote Tertiary Education Strategy (the Strategy)

13. In June 2020, the Australian Government announced⁵⁶ a range of programs and initiatives, as part of the 2020–21 Budget, the Job-Ready Package in response to the recommendations made in the Strategy to commence from 2021. The initiatives were designed to bridge the gap between regional and remote students and metropolitan students, drive productivity for the regions and increase the research capacity of regional universities.⁵⁷

14. Table A.1 below outlines programs and initiatives within the package in response to specific Strategy recommendations:

Table A.1: Overview of programs and initiatives in response to the Strategy

Note a: CGS funding originally commenced in 2013

Note b: Regional Study Hubs program was originally established in 2017.

Note c: HEPPP was first established in 2010.

Source: ANAO analysis based on Strategy and departmental data.

Appendix 4 Timeline of Access and Participation Programs and Initiatives