The objective of this audit was to assess the effectiveness of the Department of Education and Training’s design and implementation of the Vocational Education and Training (VET) Student Loans program.

Summary and recommendations

Background

1. The vocational education and training (VET) sector aims to deliver workplace specific skills and knowledge across a wide range of careers, industries and levels of personal development. VET service providers include technical and further education institutes, adult and community education providers, agricultural colleges, universities and schools, industry skill centres, commercial and enterprise training providers, as well as other private providers.

2. The VET Student Loans (VSL) program replaced the VET FEE-HELP (VFH) scheme. It was formally endorsed by the Australian Government in October 2016 for commencement from 1 January 2017 and is administered by the Department of Education and Training (the department). VET Student Loans are income-contingent loans available to eligible students. The core objective of the VSL program is to remove upfront financial barriers to VET training in order to make it more accessible to students who may not otherwise have access.

3. In the Portfolio Additional Estimates Statements 2016–17, the Government committed $45 million over four years to implement the VSL program and to deliver additional compliance measures. An additional $36.2 million over four years was allocated in the 2018–19 Budget to fund the implementation of a new IT system to support VSL compliance and regulation. As at 30 June 2018:

  • 42,220 students had accessed a VSL loan in 2017 and 43,294 from 1 January to 30 June 2018;
  • $343.9 million in VET student loans were approved from 1 January 2017 to 30 June 2018, which is eight per cent of the $4.14 billion cap available for the two years commencing 1 January 2017 (that is, a $2.07 billion each calendar year);
  • 387 approved courses are on the eligible course list; and
  • 180 registered training organisations had been approved to provide training under the VSL program.

Rationale for undertaking the audit

4. The VSL program was selected for audit because it is intended to address a number of significant deficiencies identified with the administration of the VFH scheme, as noted in:

  • Auditor-General Report No. 31 of 2016–17 Administration of the VET FEE-HELP Scheme;
  • Redesigning VET FEE-HELP: Discussion Paper (April 2016) and related public submissions (April to June 2016); and
  • The Senate Education and Employment References Committee report, Getting our money’s worth: the operation, regulation and funding of private vocational education and training (VET) providers in Australia (October 2015).

5. An assessment of the extent to which the department has incorporated lessons learnt from the former scheme into the new program allows design deficiencies to be addressed in a timely manner. The audit also provides assurance to Parliament about the robustness of the design and implementation of the VSL program.

Audit objective and criteria

6. The objective of the audit was to assess the effectiveness of the department’s design and implementation of the VET Student Loans program. To form a conclusion against the audit objective, the ANAO adopted the following high level criteria:

  • Was an appropriate design process established to support the achievement of the Government’s policy objectives?
  • Were sound arrangements established to support the implementation and management of the program?

7. The audit focused on:

  • activities pursued by the department in designing the VSL program;
  • the extent to which lessons learnt from the VFH scheme were incorporated in its design (including the learnings noted in Auditor-General Report No.31 of 2016–17 Administration of the VET FEE-HELP Scheme);
  • stakeholder consultation on the design and implementation of the new loans program;
  • the extent to which risks were considered during the design process and its implementation; and
  • the adequacy of the department’s performance management approach.

Conclusion

8. The Department of Education and Training’s design and implementation of the VET Student Loans program was largely effective.

9. The department executed an appropriate design process that considered the impacts of the new program on key stakeholders and was informed by lessons learnt from the former VFH scheme and consultation with stakeholders.

10. The department’s arrangements to support the implementation and management of the program are largely sound as it has established appropriate oversight, performance management and risk management. The department’s approach to compliance is risk-based but could be improved by detailing, in the compliance strategy, how the activities are prioritised and resourced on a risk basis.

Supporting findings

Design of the VET Student Loans program

11. The department executed an appropriate process for designing the VSL program.

12. The objectives outlined in key program documents are clearly expressed and broadly align with the Government’s policy intent.

13. There is a strong alignment between the reform options considered as part of the reviews and lessons learnt of the former VFH scheme and the final design features of the VSL program.

14. During the design phase, the department considered the impact of the VSL program would have on key stakeholders, including students and providers. Within the context of tight timeframes, the department did not develop a clear risk-based approach to planning for implementation during the period leading to commencement of the program.

15. The department appropriately consulted with stakeholders during the design and implementation of the VSL program.

16. The advice the department provided to the Government on various aspects of program design and implementation was generally sound. However, the department has not provided detailed advice about the current outcomes of the program and whether it is achieving the Government’s policy intent.

Implementation of the VET Student Loans program

17. The department established an effective performance management system, including: an evaluation plan that includes measurable key performance indicators; performance monitoring against the key performance indicators; and a range of internal and external reports.

18. The department has established appropriate arrangements to manage the program, including organisational structures and committees, which provide adequate oversight of program performance and risk management.

19. By July 2017, the department had provided useful guidance material and an appropriate range of training activities to staff involved with the VSL program.

20. The department developed largely effective communications and stakeholder engagement strategies for the VSL program, and monitored the impact of its early communication activities.

21. The department has established and maintains an appropriate risk management framework for the VSL program, which has matured over time. Relative to the VFH scheme, the department has adopted more robust compliance arrangements for the VSL program. The compliance strategy refers to risk management but does not detail how compliance activities are prioritised on a risk basis.

22. The department has put in place appropriate mechanisms aimed at supporting improvements to the coordination and effectiveness of VSL program regulatory activity.

23. The department has developed an evaluation strategy and has completed a number of reviews of aspects of the program. Early indications are that the program is progressing well against its objectives. The reviews note areas for further analysis and potential adjustments to the program’s parameters.

Recommendation

Recommendation no. 1

Paragraph 2.19

The Department of Education and Training analyse maturing VET Student Loan program data to enable it to monitor progress of the program, analyse its impact on stakeholders and identify potential enhancements to the program and its administration.

Department of Education and Training’s response: Agreed.

Recommendation no. 2

Paragraph 3.7

The Department of Education and Training develop key performance indicator(s) to measure the contribution of the VSL program to promoting confidence in the regulated VET market.

Department of Education and Training’s response: Agreed.

Summary of entity response

24. The proposed report was provided to the Department of Education and Training, which provided a summary response set out below. The full response from the department is provided at Appendix 1.

Department of Education and Training

The department acknowledges the work conducted by the ANAO and thanks the review team for the collaborative way in which the audit was conducted.

On 5 October 2016, the Government announced the commencement of the VET Student Loans program, from 1 January 2017, to replace the VET FEE-HELP scheme. The VET Student Loans program offers greater protection for students and focuses on courses that address industry needs, creating better opportunities for employment. VET Student Loans offers income­contingent loan support to eligible students studying certain diploma level and above vocational education and training qualifications.

The department’s design and implementation of the VET Student Loans program was effective in addressing significant issues with the operation of the VET FEE-HELP scheme. The VET Student Loans program safeguards public money by ensuring payments to providers are made in arrears, and that students demonstrate engagement to continue to access the loan. The program has also strengthened the compliance powers available to the department in administering the program.

I welcome the report and recommendations and note the positive findings, including that the department’s design and implementation of the VET Student Loans program was effective, and that sound arrangements to support the implementation and management of the program were in place.

Further information and updates about the VET Student Loans program, including fact sheets for students and providers participating in the program, are available on the department’s website at www.education.gov.au/vet-student-loans.

Key learnings for all Australian Government entities

Below is a summary of key learnings, including instances of good practice, which have been identified in this audit that may be relevant for the operations of other Commonwealth entities.

Policy / program design

Governance and risk management

Performance and impact measurement

1. Background

Australia’s vocational education and training sector

1.1 The vocational education and training (VET) sector aims to deliver workplace specific skills and knowledge across a wide range of careers, industries and levels of personal development. VET service providers include technical and further education institutes, adult and community education providers, agricultural colleges, universities and schools, industry skill centres, commercial and enterprise training providers, as well as other private providers.

1.2 VET is an area of shared responsibility between the Australian Government, state and territory governments and industry. The Australian Government provides financial assistance to the states and territories to support the national training system, including the VET Student Loans (VSL) program. In 2017, of the estimated 4.2 million students participating in VET activities, 1.2 million were enrolled in the government-funded system.

The VET FEE-HELP scheme

1.3 Auditor-General Report No.31 of 2016–17 Administration of the VET FEE-HELP Scheme found that the VET FEE-HELP (VFH) scheme was not effectively designed or administered. Poor design and a lack of monitoring and control by the Department of Education and Training (the department) led to costs significantly increasing. The VFH scheme was weighted heavily towards supporting growth in the VET sector (increasing student access and supporting a sufficient number of providers), but appropriate quality and accountability frameworks addressing identified risks were not put in place. In effect, the department’s focus on increasing participation overrode integrity and accountability considerations that would have been expected given the inherent risks.

1.4 In addition, the administration of the VFH scheme did not safeguard its operation and did not support the achievement of scheme objectives relating to integrity, quality, value and sustainability. Similar to the scheme’s design and implementation failures, there were weaknesses identified in the department’s administrative processes for: approving VFH providers; developing and undertaking risk, fraud and compliance activities; controlling payments to providers; making information readily available to students about their rights and obligations under the VFH scheme; and managing and resolving student complaints.

1.5 In 2015 and 2016, the Australian Government introduced a suite of legislative changes to the VFH scheme to address many of the abovementioned deficiencies.1 On 3 December 2015, in announcing measures to reform the VFH scheme designed to control growth and lift the performance of training providers ahead of a more fundamental re-design of the scheme, the Minister for Vocational Education and Skills stated that the reforms were intended to address ‘…unscrupulous marketing and enrolment practices by some training providers and restore the integrity of the VET FEE-HELP scheme’. At the same time, the Government announced its intention to introduce a new model to commence in 2017.2

The VET Student Loans program

1.6 On 5 October 2016 the Government announced that the VSL program would replace the VFH scheme. The program commenced on 1 January 2017 and is administered by the department. VET student loans are income-contingent loans available to eligible students. The core objective of the VSL program is to remove upfront financial barriers to VET training in order to make it more accessible to students who may not otherwise have access. The VSL program also aims to:

  • ensure the delivery of quality training to students, in courses that have qualifications that are more likely to lead to employment outcomes; and
  • ensure this delivery at costs that are affordable to both the student and the Government.3

1.7 In the Portfolio Additional Estimates Statements 2016–17, the Government committed $45 million over four years to implement the VSL program and to deliver additional compliance measures.4 An additional $36.2 million over four years was allocated in the 2018–19 Budget to fund the implementation of a new IT system to support VSL compliance and regulation.5 As at 30 June 2018:

  • 42,220 students had accessed a VSL loan in 2017 and 43,294 from 1 January to 30 June 2018;
  • $343.9 million in VET student loans were approved from 1 January 2017 to 30 June 2018, which is eight per cent of the $4.14 billion cap available for the two years commencing 1 January 2017 (that is, a $2.07 billion each calendar year);6
  • 387 approved courses are on the eligible course list; and
  • 180 registered training organisations had been approved to provide training under the VSL program.7

1.8 Figure 1.1 provides a comparison of student numbers and loan amounts under VFH and VSL from 2010 to 30 June 2018.

Figure 1.1: VFH and VSL student numbers and loan amounts, 2010 to 30 June 2018

 

Note: EFTSL is equivalent full time student load.

Source: ANAO analysis of Department of Education and Training data.

1.9 Figure 1.2 provides a comparison of student commencements and provider types under VFH and VSL from 2016 to 30 June 2018.

Figure 1.2: VFH / VSL commencements by provider type, 2016 to 30 June 2018

 

Source: ANAO analysis of Department of Education and Training data.

1.10 The key features of the VSL program include:

  • the list of approved courses is based on an analysis of employer, state and territory government and Australian Government data. This list applies to all approved VSL providers, with additional courses approved for some specifically listed and not-for-profit providers;
  • three loan caps set for courses depending on their estimated delivery cost (with some exceptions, such as aviation courses)8;
  • a requirement for all VSL students to confirm engagement in their studies via an extension to the electronic Commonwealth assistance form system. This new requirement, effective from 1 July 2017, is intended to ensure students are active and represent legitimate enrolments;
  • a new application process for providers wishing to participate in the VSL program. This involves the setting of a higher bar to entry, including consideration of the link the provider has with industry and other bodies relevant to the course, a range of financial criteria, student completion rates, employment outcomes, and past performance as an educational institution;
  • strengthened legislative, compliance and payment conditions, including paying approved providers in arrears; the ability to cap provider loan amounts and student numbers; and enhanced powers to suspend poor performing providers from the program, cancel their payments and revoke their approval; and
  • prohibition of approved providers from using ‘brokers’ or acting inappropriately in signing up students and limiting the subcontracting of training.

1.11 The VSL program and the loan process, which involves a number of stakeholders, is outlined in Figure 1.3.

Figure 1.3: Overview of the VET Student Loans process

This figure provides a high level overview of the processes to be followed by students (steps 1, 3, 5, 7, 9 and 12 as noted in the graphic), VET providers (steps 2, 4, 6 and 8) and the Department of Education and Training (steps 10 and 11) in applying for and processing a VET student loan. The figure is an adaptation of a more detailed flowchart graphic included in the VET Student Loans Manual for Providers, December 2017, page 9.

Note a: From 2018-19, recipients are required to commence repaying their loans when they earn above $51,956.

Source: Adapted from Australian Government, VET Student Loans: Manual for Providers, Version 2.0, December 2017, p. 9.

1.12 A timeline of key events in the development and implementation of the program is included at Figure 1.4.

Figure 1.4: Timeline on design and implementation of the VET Student Loans program

Timeline on design and implementation of the VET Student Loans program

Source: ANAO analysis of Department of Education and Training documentation.

Rationale for undertaking the audit

1.13 The VSL program was selected for audit because it is intended to address a number of significant deficiencies identified with the VFH scheme, as noted in:

  • Auditor-General Report No. 31 of 2016–17 Administration of the VET FEE-HELP Scheme;
  • Redesigning VET FEE-HELP: Discussion Paper (April 2016) and related submissions (April to June 2016); and
  • The Senate Education and Employment References Committee report, Getting our money’s worth: the operation, regulation and funding of private vocational education and training (VET) providers in Australia (October 2015).

1.14 An assessment of the extent to which the department has incorporated lessons learnt from the former scheme into the new program allows design deficiencies to be addressed in a timely manner. The audit also provides assurance to Parliament about the robustness of the design and implementation of the VSL program.

Audit approach

Audit objective, criteria and scope

1.15 The objective of the audit was to assess the effectiveness of the department’s design and implementation of the VET Student Loans program.

1.16 To form a conclusion against the audit objective, the ANAO adopted the following high level criteria:

  • Was an appropriate design process established to support the achievement of the Government’s policy objectives?
  • Were sound arrangements established to support the implementation and management of the program?

1.17 The audit focused on:

  • activities pursued by the department in designing the VSL program;
  • the extent to which lessons learnt from the VFH scheme were incorporated in its design (including the learnings noted in Auditor-General Report No.31 of 2016–17 Administration of the VET FEE-HELP Scheme);
  • stakeholder consultation on the design and implementation of the new loans program;
  • the extent to which risks were considered during the design process and its implementation; and
  • the adequacy of the department’s performance management approach.

Methodology

1.18 In undertaking the audit, the ANAO:

  • reviewed and analysed policy documents, guidelines, procedures and relevant departmental files;
  • interviewed: department managers involved in the design and implementation of the VSL program; selected VSL service providers; the peak VET bodies (TAFE Directors Australia, Australian Council for Private Education and Training and Community Colleges Australia); several state government agencies responsible for the VET sector; and representatives from the Australian Skills Quality Authority, Office of the Commonwealth Ombudsman, Department of Finance and the Australian Taxation Office; and
  • received 32 submissions from a range of key stakeholders: 27 VSL-approved service providers (14 private and 13 public providers); three state entities responsible for TAFE sectors; and two general public contributions. The first two groups represented more than 900,000 VET students.

1.19 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $416,000.

1.20 The team members for this audit were Brendan Gaudry, Brendon Gittins, Barbara Das, Alana Tolman and Deborah Jackson.

2. Design of the VET Student Loans program

Areas examined

This chapter examines the design of the VET Student Loans (VSL) program, including: the policy development process undertaken by the Department of Education and Training (the department); the alignment between the Government’s policy objectives and program objectives; lessons learnt from the former VET FEE-HELP (VFH) scheme; impacts on key VSL stakeholders; stakeholder consultation; and departmental advice provided to Government.

Conclusion

The department executed an appropriate design process that considered the impacts of the new program on key stakeholders and was informed by lessons learnt from the former VFH scheme and consultation with stakeholders.

Area for improvement

The ANAO has made one recommendation aimed at effective use of available data.

Did the department have an appropriate process for designing the VSL program?

The department executed an appropriate process for designing the VSL program.

2.1 The department adopted a systematic approach to designing the VSL program which drew upon:

  • Auditor-General Report No. 31 of 2016–17 and other VFH-related reviews;
  • a range of task force, working group and committee deliberations of the VFH scheme and VSL program;
  • learnings arising from legislative changes made to the VFH scheme in 2015 and 2016;
  • the experiential learnings of personnel directly involved in implementing and administering the VFH scheme;
  • stakeholder consultations; and
  • the consideration of a range of options, particularly in early 2016 when developing the Redesigning VET FEE-HELP: Discussion Paper.

2.2 The comprehensive approach comprised the key elements of a policy design process, which included: identifying learnings and key issues within the VFH scheme; formulating a range of policy options, many of which emerged from the activities outlined above; consulting with stakeholders during the design; and establishing an evaluation strategy.9 Details of these key policy design inputs are noted in Table 2.1 and are discussed in more detail throughout Chapters 2 and 3 (paragraph numbers are indicated in the first column of the table).

Table 2.1: Key elements of the VSL design process

Design element

Key elements of this process

Learnings and issues identification

(see paragraphs 2.7 to 2.11)

  • VET FEE-HELP Reform Working Group deliberations (April - September 2015)
  • Legislative changes to former VFH scheme (April and July 2015; January 2016) supported by department analysis of issues with the scheme
  • Consideration of recommendations from the Senate Inquiry (Education and Employment References Committee) into VET providers — Getting our money’s worth: the operation, regulation and funding of private vocational education and training (VET) providers in Australia (October 2015))
  • Learnings of key department personnel involved with managing the former VFH scheme

Policy options analysis

(see paragraphs 2.12 to 2.13)

  • Development of key issues papers by the department prior to formal VFH redesign consultations led by the Assistant Minister for Vocational Education and Skills (February - March 2016)
  • Redesigning VET FEE-HELP: Discussion Paper (April 2016)
  • Government consideration of Redesigning VET FEE-HELP discussion paper submissions (July 2016 )
  • VFH Redesign Regulation Impact Statement (September 2016)

Stakeholder consultation

(see paragraphs 2.24 to 2.30)

  • Redesigning VFH national consultations, held in Perth, Adelaide, Melbourne, Sydney, Brisbane and Cairns; hosted by the Assistant Minister for Vocational Education and Skills (early April 2016)
  • Redesigning VET FEE-HELP discussion paper submissions (consultations open 29 April to 30 June 2016 - 121 submissions received)
  • Teleconference with state and territory government officials (July - August 2016)
  • Further jurisdictional discussions on redesign through the Skills Senior Officials Network (August 2016)
  • Initial VSL Course Eligibility consultation (10-23 October 2016)
  • Consideration of issues arising from the Senate Inquiry (Education and Employment References Committee) into draft VSL legislation — VET Student Loans Bill 2016, VET Student Loans (Charges) Bill 2016 and VET Student Loans (Consequential Amendments and Transitional Provisions) Bill 2016 (November 2016)
  • Review of VSL course list and loan caps methodology (discussion paper released 12 April 2017)
  • Post-Implementation Review of VSL program (report received by the department, December 2017)

Evaluation

(see paragraphs 3.52 to 3.61)

  • Three-year strategy approved by the Minister for Education and Training (27 March 2017)
   

Source: ANAO analysis of Department of Education and Training documentation.

Were clear objectives developed for the program that align with the Government’s policy objectives?

The objectives outlined in key program documents are clearly expressed and broadly align with the Government’s policy intent.

2.3 When approving the VSL program, the Government’s policy intent was described in broad terms as introducing a scheme that is financially sustainable, meets student and industry needs, is outcomes focused and governed by effective legislation and compliance.

2.4 The objectives of the VSL program are described in key program documents, such as the Program Administration Plan, as:

  • the program is fiscally sustainable and contributes to national economic growth;
  • the program removes financial barriers to training and improves equity of access to higher level VET;
  • the program promotes the delivery of quality and affordable training for students;
  • the program balances industry needs, employment outcomes and student choice;
  • the program is student centred through adequate protection for students (particularly from disadvantaged backgrounds) and access to information that enables informed decision making; and
  • the program has program integrity, manages risk and promotes confidence in the regulated VET market.

2.5 These objectives were described as principles in the Program Implementation Plan and as objectives / principles in the Evaluation Strategy. The Program Implementation Plan expressed the core objective of the VSL program as: to make VET training more accessible to students who may not otherwise have access by removing upfront financial barriers. The plan added that the VSL program aims to:

  • ensure the delivery of quality training to students, in courses that have qualifications that are more likely to lead to employment outcomes; and
  • ensure this delivery at costs that are affordable to both the student and the Government.

2.6 As such, there is a lack of consistency about the program’s objectives across key program documentation.10 Nevertheless, the objectives broadly align with the government’s policy intent.

Were lessons learnt from the former VET FEE-HELP scheme reflected in the design of the program?

There is a strong alignment between the reform options considered as part of the reviews and lessons learnt of the former VFH scheme and the final design features of the VSL program.

2.7 A range of reform options to the VFH scheme were considered at various times from late 2014, resulting in a series of changes to the scheme, effective from 1 April 2015, 1 July 2015 and 1 January 2016. The reform measures were designed to address the issues of rising fees and unscrupulous provider behaviour. The changes were informed by, among other things, the findings and recommendations of the Senate’s Education and Employment References Committee inquiry into the operation, regulation and funding of private VET providers in Australia11, a 2014 taskforce and the VFH Reform Working Group. This, in turn, informed the department’s later thinking on what aspects of the VFH scheme needed to be reformed.

2.8 Subsequent to the Government’s announcement in December 2015 that a new model would be introduced in 2017, the department established the VET FEE-HELP Redesign Steering Committee in February 2016.

2.9 The Redesign Steering Committee was responsible for making decisions and providing advice on: project planning and risk management; policy development and costings; transitional strategies and implementation; stakeholder management and consultations; and any complementary and consequential measures that may be required.

2.10 The committee considered a number of issues, including:

  • whether VET FEE-HELP is a possible mechanism for enhancing quality in VET;
  • course eligibility and loan caps;
  • student engagement requirements;
  • building an informed market; and
  • impacts on the VET market.

2.11 Papers addressing these areas succinctly canvassed the main issues and identified options, many of which were reflected in the final program design. This included, for example, considering the option of using state and territory Subsidised Training Lists, the Skills Shortage List or the Skilled Occupation List for the VSL eligible course listing methodology. Deliberations of the committee were used to brief the Minister for Vocational Education and Skills for a series of national consultations conducted during the first two weeks of April 2016.

Redesigning VET FEE-HELP: Discussion Paper

2.12 The Redesigning VET FEE-HELP: Discussion Paper, issued by the Australian Government in late April 2016, stated that the paper had been informed by feedback received from the Minister’s national consultations earlier that month. The discussion paper and the issues papers referred to above were also informed by VFH reforms.

2.13 The redesign discussion paper provided a detailed analysis of the take-up trends and history of the VFH scheme and the problems that beset it. It also posed a series of questions around options for change, focusing on the themes of: protecting students, regulating providers and managing the overall system. The department received 121 submissions by the due date of 30 June 2016.. These submissions covered: course list and caps; the use of third party providers and brokers; online learning; the role of tuition assurance; the proposal to establish a student support mechanism, either through an Ombudsman, the Australian Skills Quality Authority (ASQA) or a department complaints handling function; the use of completion rates to measure course quality; and VSL-provider application processes. The department considered the feedback in the design of the VSL program.

Lessons learnt from Auditor-General Report No. 31 of 2016–17

2.14 Auditor-General Report No. 31 of 2016–17 was tabled in December 2016 and the department adopted the emerging lessons learnt from this audit when designing and implementing the VSL program. Table 2.2 outlines the alignment between lessons learnt and key elements of the program’s final design. Further details are also noted in the remainder of this chapter and in Chapter 3.

Table 2.2: Integration of lessons learnt from the Auditor-General’s audit in the design and implementation of VSL

Key learnings from the Auditor-General’s report include the importance of:

How the ANAO lessons learnt are reflected in the VSL program

Thoughtfully considering the critical differences between a new program and any existing program on which it has been modelled, including how different incentive structures for key participants (including financial incentives) will create risks to the achievement of program objectives. Similarly, in revising an ongoing program, recognising how substantially altered incentive structures will change behaviours and risks.

The VSL program includes more stringent approval requirements for providers, including: strengthened consistency in reporting, accountability and transparency requirements; an increased emphasis on assessing compliance; and strengthened financial viability requirements.

Enhanced regulatory powers of the department and key VET regulators.

Learning from comparable experiences in other agencies or jurisdictions, and carefully considering the supporting program documents, such as regulation impact statements, when designing and implementing programs.

The department’s experience with the former VFH scheme informed the design and implementation of the VSL program. This is apparent, for example, in the Redesigning VET FEE-HELP: Discussion Paper.

Integrating risk management principles and processes into the design, implementation and administration of a program, to effectively manage risks to the achievement of the objectives and outcomes of programs.

The department has developed a risk strategy, a range of risk and compliance plans, and a detailed Program Implementation Plan.

A range of departmental governance and reporting processes have also been established to oversee program design and implementation.

The department has also adopted a risk-based methodology and process for assessing and approving VSL providers.

Placing emphasis on achieving all program objectives and outcomes, rather than excessively focusing on the prime objective (such as participation in a program). Integrity, quality and sustainability are often intrinsically linked to the primary objective and need to be achieved

The Program Implementation Plan notes the need to ensure that adequate weighting was given to achieving all program objectives.

Developing key performance indicators to measure the success of the program against all key objectives and outcomes. This will focus attention on achieving all objectives and prevent entities from overlooking key risks. Evaluating programs with a focus on understanding their impact will indicate whether the underlying policy approach is an effective intervention.

Key performance indicators and an evaluation strategy were developed by the department for the VSL program. The latter included the undertaking of a Post-Implementation Review in the second half of 2017, with further reviews scheduled for late 2018 and 2019.

Establishing a strong data analytics capability and management reporting process to identify emerging threats and promote understanding and visibility of the outcomes of the scheme. In demand driven programs, modelling and sensitivity analysis should be undertaken to forecast demand and monitoring both uptake and cost can provide early warnings of potential threats to the effective and efficient implementation of programs.

The department has increased its internal capacity to collect and analyse data. This included creating a detailed data roadmap which identified: current issues, risks and their management, protocols for data requests and responses, and also the resources required to develop a data analytics capability.

 

Clarifying roles and responsibilities and introducing effective mechanisms for information sharing and engagement with all entities with a role in design or implementation. Where other regulators have a role, the key implementation agency should consult with those regulators to analyse the strength of the regulatory environment and address any notable shortcomings, including by drawing these to the attention of the Government as early as possible.

 

Governance processes were established for information sharing (internally and externally) and that risks / issues are identified as a priority to be addressed by program managers, department executives and the Ministers in a timely manner.

Mechanisms (such as memorandums of understanding) are also in place to facilitate open-ended consultation and clarity of the roles and responsibilities with relevant Australian Government entities, state and territory governments, and VET regulators.

The VET Student Loans Ombudsman was established from 1 July 2017 to manage student complaints and advise the department of any identified systemic issues.a

Ensuring fraud, risk and compliance arrangements are operational from the commencement of a program, and accurately reflect program risks and requirements.

 

Strengthened VSL program arrangements (legislative and administrative) have applied since 1 January 2017. These include restrictions on course eligibility, capped loan amounts for courses, payment caps for providers, and payments being made in arrears based on department-attested data. Students are also required to demonstrate engagement (or progression) with their course.

Under the VSL legislation, the department’s monitoring and regulatory powers expanded, including the ability to suspend a provider or freeze its payments on non-compliance, or suspicion of non-compliance.

   

Note a: As at 31 March 2018, the VET Student Loans Ombudsman had received 48 complaints about the VSL program, compared to 5,193 complaints about the VFH scheme received since 1 July 2017. The most common issues identified through these VSL complaints were the loan amount, including loan caps, complainants’ applications for VET loan assistance, and course quality and progression.

Source: ANAO analysis.

Were the impacts on key stakeholders considered in the design of the program?

During the design phase, the department considered the impact of the VSL program would have on key stakeholders, including students and providers. Within the context of tight timeframes, the department did not develop a clear risk-based approach to planning for implementation during the period leading to commencement of the program.

Impact on stakeholders

2.15 Building on reviews of the VFH scheme, the department assessed the impact of the new model on key stakeholders. For example, the Redesigning VET FEE-HELP: Discussion Paper included a detailed analysis of the impact on providers, students and in managing the overall VET system from an Australian Government perspective. The VET FEE-HELP Redesign Regulation Impact Statement, developed by the department in September 2016 as input to the Government’s consideration of the new VSL program, also drew on past analysis and considered the impact of the proposed model on students and service providers. In the Regulation Impact Statement, the department estimated that there would be no regulatory impact from the new program.

2.16 Some of the areas considered were:

  • Student engagement: the department determined that this process would have a small regulatory cost, estimated at five minutes per fee period.12 The department considered that this cost would be balanced by: increased awareness by students that their studies are not free and that the loans must be repaid; an increase in the enrolment of genuine students; and the encouragement of increased completion rates among engaged students. The Post-Implementation Review suggested that unit completion rates are at least 10 per cent higher under VSL than they were in the last four years of the VFH scheme. However, trends in enrolments and course completion rates, relative to the former VFH scheme, may not be known for some time.
  • Loan caps: the department noted that the introduction of loan caps would result in some students incurring additional costs as a result of fee gaps as providers were expected to charge fees above the loan caps. However, the department considered that the introduction of loan caps and the competition for students should decrease fees, thereby minimising the impact on students of out-of-pocket fee costs. Figure 2.1 shows that average tuition fees have decreased, and that fees charged for the top 10 VSL courses, by commencements, are lower in 2017 than they were in 2016 under the VFH scheme.

Figure 2.1: Average tuition fees for top ten VSL courses, as at July 2018

 

Note: Data is for equivalent full time student load (EFTSL).

The composition and ranking of the top ten courses has changed over time. The top 10 courses as at July 2018 have been used to allow for a comparison of average tuition fees across the three years.

Source: ANAO analysis of Department of Education and Training data.

  • Course mix and choices: the department suggested that students would have increased choice of training options under VSL. However, due to the lower number of VSL-eligible courses relative to VFH, student choice is more limited.
  • Regulatory costs on providers: the department noted that the impact on providers would include new compliance and reporting arrangements and a requirement for all private providers who wish to participate in the new scheme to apply. These impacts were expected to be balanced by greater transparency and increased provider quality. In addition, it was anticipated that streamlining the provider application process would result in significant efficiencies. A June 2018 internal audit by the department found that there were opportunities to rationalise the information required in the application process.
  • Closure of providers: the department noted that a number of existing VFH providers were likely to close as a result of the introduction of VSL (as had been occurring under VFH), which would increase tuition assurance events. The department advised that the likely number of closures and related tuition assurance activations under VSL was not modelled at the time due to time constraints and that it focused on developing stronger VSL tuition assurance requirements for providers and operators. The department undertook subsequent modelling when considering tuition assurance arrangements in 2018 and 2019. Under VSL, two operators were given provisional approval as tuition assurance scheme operators until 31 December 2017. On 10 November 2017, the department announced that it would administer the tuition assurance arrangements for 2018 while it considers future tuition protection.
  • Consultation with regulators: the department met with regulators, including ASQA, to discuss the potential impacts of the new model.
  • Student take-up of loans and fiscal sustainability: the VSL program was developed on the basis of the assumption that the take-up for VSL would be similar to 2016 VFH levels (115,000 EFTSL in 2016) — 135,000 EFTSL in 2017, increasing to 159,000 EFTSL in 2020 — and the value of loans under the program was capped at $2.07 billion each calendar year. The department advised the ANAO that 42,220 students accessed a VSL loan in 2017 and 43,294 students from 1 January to 30 June 2018. From 1 January 2017 to 30 June 2018, $343.9 million in VET student loans were approved. In August 2018 the department advised the ANAO that it does not have clear data on why take-up is lower than expected. As such, it has not analysed the reasons why program outcomes differ from forecast take-up levels and, therefore, whether it is achieving the Government’s policy intent of introducing a scheme that is, among other things, fiscally sustainable and meets student needs.

2.17 The department stated that its analysis of potential impacts of the program on key stakeholders was based on information that was available during the design process, but that it did not have sufficient data on provider and student behaviour to effectively model key elements of the program. However, data on the VFH scheme was available to the department that could have informed analysis of student and provider behaviour and Deloitte Access Economics, which was commissioned to review course eligibility and loans cap methodologies in early 2017, was able to access a range of relevant data.13 The department informed the ANAO that it has undertaken some modelling and analysis, for example, the total value of loans and change in the number of approved providers. It is also planning to examine: the interaction of the array of state and territory subsidy schemes and VSL; fee gaps for approved courses; provider coverage impacts on geographical access; and general VSL take-up trends, by equity groups.

2.18 As the program has now been operating for over a year and a half, more robust data will be available to the department to: monitor the progress of the program; assess the program’s impact on stakeholders; understand the reasons underlying the low take-up of VSL; and identify potential improvements to the program and its administration.

Recommendation no.1

2.19 The Department of Education and Training analyse maturing VET Student Loan program data to enable it to monitor progress of the program, analyse its impact on stakeholders and identify potential enhancements to the program and its administration.

Department of Education of Training response: Agreed.

2.20 The department has increased its data analytics resources. Action is underway to enhance the current analytical tasks with a focus on stakeholders to anticipate potential issues, which will assist program development. Additional reporting dimension tables have been added to the existing data model to harness the use of the department’s visual analytics and reporting tools, such as Microsoft Power Bl, on the Program User’s desktop. The combination of these resources will provide the department with enhanced analytical abilities to monitor and maintain the program integrity.

Impact on the department

2.21 The Government announced the VSL program on 5 October 2016, the legislation was assented to on 7 December 2016, and the program commenced on 1 January 2017.14 As a result, the time available to the department to plan for implementation and to develop necessary program documents and processes was limited, which was an implementation risk. The department advised the ANAO that it took a risk-based approach to implementing the new program and discussed the implementation risk with the Ministers. This approach involved the development of a series of Gantt charts, which in the design phase focused on the VFH grandfathering / opt-in process, transitional provider approvals, an IT second pass business case and milestones concerning the passage of VSL legislation. The department did not develop a risk-based implementation plan to prioritise tasks and activities in the period leading to commencement of the program.

2.22 Table 2.3 provides examples of the date at which some components of the program were decided or issued and the impact of the timeframe.

Table 2.3: The department’s readiness to implement the VSL program

Advice to providers and students

VET Student Loans Rules 2016 (VSL Rules) assented to by Parliament (7 December 2016)

  • used by providers and states in the absence of formal department guidance until the provider manual was issued in May 2017

VET Student Loans (Courses and Loan Caps) Determination 2016 (21 December 2016)

  • providers reported that they were not in a position to respond during the 2016 Christmas break, in terms of having IT systems, student support, course availability and enrolment information in place
  • some students had to arrange to enrol in an alternative course supported by the VSL program

VET Student Loans information booklet (23 Dec 2016)

  • feedback suggests that not all students were aware of this booklet; it was only accessible via the VSL application form and electronic Commonwealth assistance form (eCAF)

VET Student Loans Manual for Providers (23 May 2017)

  • in the absence of the provider manual, providers had to rely on the their interpretation of the VSL Rules and the department’s online enquiry process

VET Student Loans Newsletter to Providers (April 2017)

  • first standalone VSL newsletter circulated by the department (although the department used other communication channels such as website, factsheets, newsletters, letters and email; see paragraphs 3.28 to 3.33 for further details

VET Student Loans Student Engagement and Progression (13 June 2017) and VET Student Loans - Student engagement and progression FAQs (26 July 2017)

  • timely advice provided to providers

VSL-provider application processes

  • Given the short time to process applications, providers were granted provisional approval for six months. As a result, many providers were uncertain about their future status as a VSL provider and reported that this impacted on their ability to make operational decisions.

Department IT systems

  • Existing IT systems had to be reconfigured to support the new program
  • The provision of program data to the Australian Taxation Office was often delayed due to issues identified with the department’s interim IT solution
  • In April and May 2017, a systems issue allowed 750 students to submit an invalid VSL application. All student records were amended in May 2017.

Source: ANAO analysis of Department of Education and Training documentation.

2.23 While the department used various channels to communicate with providers and students, comprehensive manuals and relevant newsletters and factsheets were not available to stakeholders until after the program commenced. See Appendix 2 for stakeholder feedback, including further comments about this issue.

Were stakeholders appropriately consulted in the design and implementation phases of the program?

The department appropriately consulted with stakeholders during the design and implementation of the VSL program.

2.24 Figure 2.2 provides an overview of the stakeholder consultation processes used by the department in the design and implementation of the VSL program.

Figure 2.2: VSL stakeholder consultation

VSL stakeholder consultation

Source: ANAO analysis of Department of Education and Training documents.

Stakeholder consultation — design phase

2.25 As noted earlier, the department established the VET FEE-HELP Redesign Steering Committee in February 2016, with its remit including the provision of advice on stakeholder management and consultations. The Committee’s deliberations, including feedback received from the ministerial consultations held in early April 2016 and other departmental research inputs, culminated in the Redesigning VET FEE-HELP: Discussion Paper.

2.26 The discussion paper outlined how the Government intended to manage the consultation process. It provided a period of two months, to June 2016, for stakeholders to provide submissions and included a commitment to undertake a further detailed and wide consultation process regarding the options outlined in the paper. The department discussed the new program with state/territory representatives during June to August 2016. However, providers, industry and peak bodies, regulators and organisations that represent student groups were not formally consulted following the deadline for submissions to the discussion paper until after the announcement of the VSL program on 5 October 2016.15

2.27 The department outlined to the government two options to implement the new program. One option was a slower roll out of the new program, which would allow for more consultation; the other was a ‘hard and fast’ implementation, limiting further consultation. The government opted for the latter. The department advised the ANAO that the Minister for Education and Training determined in early August 2016 that sufficient consultation had been undertaken to progress with the design of a new program and sought a 1 January 2017 commencement date, which precluded further consultations. This meeting was not minuted and the decision was not documented at the time of the meeting. The department also advised that there was limited time to undertake further discussion. In its advice to the Minister for Education and Training in September 2016, it was noted that the same themes were continuing to emerge and that the design took into account the views of a broad range of stakeholders.

2.28 The department advised the Minister for Education and Training that ‘Undertaking a consultative approach provides the opportunity to engage stakeholders and gain their confidence in the new scheme. It provides adequate time to educate both stakeholders and students about the changes.’16 Stakeholders advised the ANAO that they felt marginalised by the department not consulting after the deadline for submissions to the discussion paper. Stakeholder engagement was important in achieving the objective of promoting confidence in the regulated VET market.

Stakeholder consultation — implementation phase

2.29 The department uses a number of fora to consult with stakeholders, including:

  • the VET Student Loans Implementation Advisory Group (IAG) (see Box 1);
  • Council of Australian Government’s Industry Skills Council and its associated Skills Senior Officials Network;
  • an inter-departmental committee (IDC), comprising representatives from ASQA, Australian Competition and Consumer Commission, Australian Taxation Office and the Departments of Finance, Human Services, the Prime Minister and Cabinet and Treasury;
  • an Interagency Working Group, which is chaired by the VSL Ombudsman and has broadly the same membership as the IDC; and
  • annual provider information forums.

These fora provide an opportunity for the department to engage with stakeholders, communicate more broadly with the VET sector and share information about the program.

Box 1: VET Student Loans Implementation Advisory Group

The IAG is a key mechanism for the department to consult with a broad range of stakeholders Established in November 2016, it met monthly until 2018 and now meets every six months. The group includes representatives from industry, peak bodies, public and private providers, consumer bodies and ASQA. The IAG provides advice on a range of issues relevant to the implementation of the program, such as the VSL Rules, proposed annual and application fees and charges for providers and the communications strategy.

Most of the members of the IAG interviewed by the ANAO viewed their involvement as a means by which the department could hear a range of stakeholder views on various aspects of the program, but questioned their influence on the implementation of, and improvements to, the program.

The department advised that the IAG is expected to cease in September 2018 but that it will continue to consult with stakeholders as part of normal business practice, including seeking feedback on changes to the VSL program as appropriate.

Source: ANAO analysis of Department of Education and Training documentation.

2.30 The main stakeholder consultation undertaken since January 2017 has been in relation to the post-implementation review and a review of eligible courses and caps methodologies. These are discussed from paragraph 3.55 to 3.60. Appendix 2 summarises stakeholder feedback received by the ANAO.

Was sound advice provided to the Government on program design and implementation?

The advice the department provided to the Government on various aspects of program design and implementation was generally sound. However, the department has not provided detailed advice about the current outcomes of the program and whether it is achieving the Government’s policy intent.

2.31 The department provided a range of advice to the Ministers, covering for example:

  • VFH Redesign submission feedback;
  • pre-Government approval advice on the VSL proposal;
  • submissions in response to initial VSL eligible courses list;
  • provider application processes;
  • analysis of changes in course fees from 2016 to 2017 by VSL providers;
  • eligible courses and loan caps methodologies review; and
  • results of the Post-Implementation Review.

2.32 The ANAO reviewed key advice provided by the department to the Government and found that it was generally clear and comprehensive. The advice included informative background information, supporting documentation and analysis as required. The ANAO noted several examples of detailed analysis being included in the advice. For example: advice to the Government about the Redesigning VET FEE-HELP: Discussion Paper included detailed analysis on the key issues raised in the 121 submissions received in response to the paper17; and options for the VSL eligible course list methodology was provided as input to commissioning the VET Student Loans – Eligible Course list and Loan Cap methodologies review.

2.33 As noted earlier, the department has stated that it did not have sufficient data on provider and student behaviour to effectively model key elements of the program, which impacted on the advice it provided to the Government about the new program. The Minister for Education and the Minister for Small and Family Business, Skills and Vocational Education also receive monthly reports that include statistics on the VSL program (discussed further in paragraphs 3.14 to 3.15). However, as at September 2018, the department has not provided detailed advice to the Government about the current outcomes of the program, including the reasons for the lower than anticipated take-up of VSL loans and whether it is achieving the Government’s policy intent.

3. VSL program implementation and management arrangements

Areas examined

This chapter examines the Department of Education and Training’s (the department’s) arrangements to support the implementation and management of the VET Student Loans (VSL) program, including: performance management; oversight arrangements; guidance and training; communications; risk management and compliance; regulatory roles and responsibilities; and evaluation.

Conclusion

The department’s arrangements to support the implementation and management of the program are largely sound as it has established appropriate oversight, performance management and risk management. The department’s approach to compliance is risk-based but could be improved by detailing, in the compliance strategy, how the activities are prioritised and resourced on a risk basis.

Area for improvement

The ANAO has made one recommendation to develop a key performance indicator to measure the extent to which the VSL program promotes confidence in the regulated VET market.

Has an effective performance management system been established to measure and report outcomes from the program?

The department established an effective performance management system, including: an evaluation plan that includes measurable key performance indicators; performance monitoring against the key performance indicators; and a range of internal and external reports.

3.1 To support the implementation and administration of the VSL program, the department put in place the key components of an effective performance management system, including:

  • an evaluation plan, including key performance indicators (KPIs);
  • performance monitoring; and
  • a range of internal and external reporting.

3.2 The department developed a detailed VET Student Loans Evaluation Strategy for the program, which was approved by the Minister for Education and Training in March 2017. The plan included evaluation questions that covered the program’s objectives, assessing the following areas:

  • sustainability of the program and its affordability to students;
  • quality of the training delivered;
  • degree to which it achieved a balance between the needs of industry and the student;
  • program integrity; and
  • whether there are any unintended regulatory burdens.

3.3 The evaluation will be conducted in three stages from 2017 to 2019. The initial Post-Implementation Review (PIR), was finalised in December 2017. This is to be followed by a progress report scheduled for completion by 30 November 2018 and a final evaluation report scheduled for completion by 30 November 2019. The 2018 and 2019 reports are to focus on how VSL has addressed the policy intent of the program.

3.4 KPIs for the VSL program were outlined in the evaluation strategy. Six of the 19 KPIs focus on measuring the quality of VET training, with a further five linked to VSL’s integrity; the remainder are divided across the other objectives.18 The KPIs map to the program objectives, and they are relevant, reliable and complete, and baselined against VFH data where equivalent information is available (that is, for 16 of the 19 KPIs).

3.5 Data to measure performance against the KPIs is sourced from a variety of internal and external sources, such as internal IT systems, the Australian Government Actuary and the VET Student Loans Ombudsman. The VSL quarterly performance dashboard (discussed in paragraph 3.13) includes quantitative reporting against seven of the 19 KPIs. The department does not report against the qualitative KPIs in one document. Appendix 3 lists the KPIs, against the program objectives, and the ANAO’s analysis of their relevance, reliability and completeness. Appendix 4 shows the department’s performance against the KPIs as reported in the VSL quarterly performance dashboard.

3.6 The department has not developed a KPI to measure the impact of the VSL program in promoting confidence in the regulated VET market. The department advised the ANAO that the program’s design addresses the issues observed in the former VFH scheme, which will in turn help address the overarching objective of promoting confidence in the sector. The department also noted that the reputation of the VET sector is affected by additional influences beyond the VSL program; for example, its attractiveness to international and domestic students considering the choice between a VET and higher education qualification. Nevertheless, the department should develop a measure to aid ongoing monitoring of the program’s contribution to this objective.

Recommendation no.2

3.7 The Department of Education and Training develop key performance indicator(s) to measure the contribution of the VSL program to promoting confidence in the regulated VET market.

Department of Education and Training response: Agreed.

3.8 Under the Post-Implementation Review (PIR), a Key Evaluation Question relating to quality and integrity was ‘Does the program design contribute towards improving confidence in the VET market’. Consultation with stakeholders has already shown that the exit of ‘non-genuine’ providers has resulted in improved confidence in the VET sector.

3.9 Every four months, students submit a form to indicate progression through their courses. The submission includes a survey for students to rate whether they are satisfied with the quality of their training provider and whether they would recommend the provider to anyone. The department conducts regular analysis on these results. Almost 90 per cent of survey respondents indicated their provider was of high quality and would recommend the provider to others for study purposes.

3.10 Phase 2 of the PIR will further assess whether the VET Student Loans program promotes confidence in the VET market. Phase 2 will commence in early 2019, as the program will have been embedded for two years with sufficient data and information available to inform the review.

3.11 During the initial implementation of the VSL program (January to June 2017), weekly executive reports were provided to members of the VET Student Loans Implementation Taskforce, as well as the Secretary and Deputy Secretary of the department. These reports were an effective mechanism for providing high-level status updates on the grandfathering process, provider approvals and closures, and the IT two-pass business case.

3.12 The ongoing performance of the VSL program is reported using the business-as-usual reports shown in Table 3.1.

Table 3.1: Internal and external reporting of VSL performance

Internal reporting

External reporting

• VSL quarterly performance dashboard

• Monthly ministerial reporting

• Payments review

• Branch manager briefings to the Ministers

• Six monthly and annual statistical reports

• Student Progression Survey

• VET Student Loans Ombudsman quarterly updates

• Annual Report

• Department financial statements and budget papers (where appropriate)

   

Source: Department of Education and Training, VET Student Loans Program Administration Plan, January 2018.

3.13 The VSL quarterly performance dashboard is the key internal reporting mechanism, covering a range of the program’s KPIs. The report provides readers with an accessible overview of the program’s performance against the KPIs, with ’traffic light’ performance ratings based on comparisons with VFH performance. The dashboard report is considered at each quarterly meeting of the VET Student Loans Program Performance and Risk Committee (discussed in paragraph 3.22).

3.14 The Minister for Education and the Minister for Small and Family Business, Skills and Vocational Education receive monthly reports that include the following data:

  • comparison of VFH/VSL providers by provider type;
  • student commencements by month and cumulative by month;
  • equivalent full time student load by month and cumulative by month;
  • VET loan amount by month and cumulative by month;
  • VET loan amount by reference month (month paid less two) and cumulative by month; and
  • overview of all providers.

3.15 The monthly reports provided to the Ministers contain sufficient information to gain an understanding of student participation and loans approved. However, benchmarking against the former VFH scheme is limited to figures for VFH students grandfathered in 2017 and 2018.19 The inclusion of comparable pre-2017 VFH data, where available, to support comparison between VSL and VFH would allow greater oversight of the program’s overall performance.

3.16 The VSL six monthly and annual statistical reports are the primary form of public reporting on the performance of the VSL program. These reports meet the legislated reporting requirements set out in the VET Student Loans Act 2016 (VSL Act), and include information regarding:

  • number of providers;
  • value of loans;
  • number of VSL-assisted students;
  • course completions; and
  • tuition fees charged.

3.17 In the department’s 2016–17 annual report, VSL performance information was aggregated as part of its reporting on the Higher Education Loans Program (HELP) program and the VET sector more broadly. Including key high-level VSL-specific information, such as the total number of students in receipt of VSL, in the annual report would align VSL reporting with the other HELP loans (including HECS-HELP and FEE-HELP) for which disaggregated student numbers are provided.

3.18 Overall, the VSL reports are designed to provide the Parliament, Government, the department’s Executive and the public with an overview of the program’s performance.

Are appropriate oversight arrangements in place to effectively manage the program?

The department has established appropriate arrangements to manage the program, including organisational structures and committees, which provide adequate oversight of program performance and risk management.

Key players in VSL management

3.19 The current oversight arrangements for the program are shown in Figure 3.1.

Figure 3.1: VSL program governance

VSL program governance

Note: The Minister for Education administers the Department of Education and Training, while the Minister for Small and Family Business, Skills and Vocational Education has portfolio responsibility for the VET sector and VSL program.

Source: ANAO adaption of Department of Education and Training documentation.

3.20 Management of the VSL program is largely the responsibility of the department’s Skills Programs Group, which includes the VET Student Loans and Skills Programs Compliance branches. These branches work closely together, meeting regularly to discuss the current status of issues and activities of each team. The Skills Market Group, which includes the Skills Outcomes and Financing branch, also has a role in administering the program.

3.21 A VET Student Loans Governance Committee was established by the department in January 2017 to discuss a range of issues related to the implementation of the program and to ensure strong links between the VET Student Loans Implementation Taskforce and the VET Student Loans branch.20 The ANAO reviewed the minutes maintained by this Committee and notes that an issues register was created and reviewed weekly, and a record was kept of action items opened and closed. The Committee held its last meeting in September 2017, being superseded by the Performance and Risk Management Steering Committee.

3.22 The Performance and Risk Management Steering Committee, established in September 2017, was responsible for reviewing and improving program administration, including program direction, and delivering against the objectives of the VSL program and VFH scheme. In January 2018 it was renamed the VET Student Loans Program Performance and Risk Committee. This committee meets bi-monthly and comprises membership across the Skills Program and Skills Market Groups. As well as focusing on reviewing program performance (and the reporting thereof) and risk management, its remit includes a quarterly review of the internal Program Administration Plan developed in January 2018. Further details on the risk management deliberations of this committee are discussed in paragraphs 3.38.

Has appropriate guidance and training been provided to VSL program staff?

By July 2017, the department had provided useful guidance material and an appropriate range of training activities to staff involved with the VSL program.

3.23 Key guidance materials available to departmental staff involved with the VSL program include the:

  • VSL Program Implementation Plan and VSL Program Administration Plan 2018;
  • VSL Manual for Providers;
  • VET FEE-HELP Branch Business Plan 2016–17;
  • Standard Operating Procedures developed for the VSL and Skills Programs Compliance branches; and
  • an internal, detailed VSL overview document that outlines the key elements of the program, how it works, what features are supported by legislation or the VET Student Loans Rules 2016 (VSL Rules), key dates and a summary of 2015 and 2016 changes made to the VFH scheme that were retained under VSL.

Collectively, the documents provide useful guidance for departmental staff to administer the program.

3.24 The department has developed a comprehensive and well targeted VSL training program, which identified the officers to be trained and the timing of training. The program included: risk management and regulation, communications, provider application process, probity, project management, compliance and IT systems. The training program was not based on a formal training needs analysis; it was based on management’s prior experience with the VFH scheme and their understanding of the new VSL program.

3.25 In some cases the training was delivered when needed. For example, communications training was provided to call centre staff at the time of the Government’s announcement on 5 October 2016 to establish the new program. However, most training and related guidance was provided to relevant staff after the program had commenced on 1 January 2017, often to coincide with the release of operational plans and processes. The department has advised that it did not evaluate this training.

Data analytics capability

3.26 A key learning noted in Auditor-General Report No. 31 of 2016–17 was a need for the department to establish a strong data analytics capability and related management reporting processes to identify emerging threats and promote understanding and visibility of the outcomes of the scheme.

3.27 The department has recently expanded its VSL data analytics capacity. In August 2018 it established VSL Data Analytics and VSL Data and Reporting teams. Responsibilities of the teams include: data analytics for policy and compliance, IT system liaison and VFH/VSL data reporting. The department advised the ANAO that current systems do not support easy access to data and that its data analytics capability requires strengthening. These issues are expected to be addressed through the development of a new IT platform (scheduled to be fully operational from mid-2020) and related capability improvements.

Has an effective communications strategy been established that provides appropriate information to key stakeholders?

The department developed largely effective communications and stakeholder engagement strategies for the VSL program, and monitored the impact of its early communication activities.

3.28 The department’s communications with key stakeholders are guided by a Communications Plan21 and a Stakeholder Engagement Plan. The department developed the Communications Plan in February 2017, with additional plans developed for specific campaigns; for example, related to compliance, provider fees and various aspects of the provider application process. The Stakeholder Engagement Plan, developed in April 2017, maps stakeholders and identifies key issues and mechanisms for targeting communications. The Stakeholder Engagement Plan provides a higher level summary of the engagement approach adopted by the department, with the Communications Plan providing greater detail of target audiences, key messages, communication channels and evaluation parameters.

3.29 To support the move from the VFH scheme to the VSL program and inform affected stakeholders about the changes and sources of support, the department undertook a range of communications activities as outlined in its plans. There were three phases to the campaign:

  • Phase 1. October to December 2016 — VSL program announcement and closure of VFH;
  • Phase 2. January to June 2017 — implementation of the VSL program; and
  • Phase 3. July to 31 December 2017 — business-as-usual communications.

3.30 VSL program communications is multi-faceted, providing information to stakeholders, primarily students, and platforms through which stakeholders can contact the department. Communications channels include:

  • VSL program and general student advisory websites and social media22;
  • a range of general program fact sheets targeted at students and providers;
  • a series of targeted frequently asked questions pages on the department’s website on key topics, including the student opt-in process and changes in the VSL provider application process;
  • VSL program inbox (VETStudentloans@education.gov.au) — mainly targeting providers;
  • ad hoc provider newsletters and provider information forums;
  • electronic mail-outs to students and providers;
  • VSL student and provider information manuals;
  • meetings with communications channelled through the various VET peak bodies;
  • call centres, supported by the Department of Human Services and Australian Taxation Office; and
  • providing information through the VSL Implementation Advisory Group.

3.31 The information was largely well targeted, clearly presented, accurate and fit-for-purpose, and was generally timely. The department’s approach to communications could have been improved in some areas. For example, stakeholders advised the ANAO that during the early stages of the program providers were not always informed of messages to students. Also, some of the communications to students was technical in nature. Combined with the lack of a comprehensive manual for providers, which was released in May 2017, this meant that it was difficult for providers to respond to student enquiries in a constructive and timely manner.

3.32 The department reviewed the impact of the communications and engagement activities in early 2017, concluding that the communications and stakeholder engagement measures appeared to work well. The review covered the period from the Minister for Education and Training’s announcement of the new program on 5 October 2016 through to March 2017.

3.33 Of the 32 submissions received by the ANAO from a range of stakeholders, 19 included some form of feedback on the communications activities undertaken by the department in implementing the VSL program. Appendix 2 summarises the key issues identified through the ANAO’s stakeholder consultation process.

Are appropriate risk management and compliance arrangements in place?

The department has established and maintains an appropriate risk management framework for the VSL program, which has matured over time. Relative to the VFH scheme, the department has adopted more robust compliance arrangements for the VSL program. The compliance strategy refers to risk management but does not detail how compliance activities are prioritised on a risk basis.

Risk

3.34 The department has established and maintains a comprehensive risk management framework for the VSL program, using the department’s existing risk management platform, RiskActive. The framework includes the VET FEE-HELP Scheme and VET Student Loans Program Risk Strategy, created in February 2017, which outlines processes such as governance, monitoring and risk escalations, and provides a high level overview of the VSL project risk plans (discussed below).

3.35 The VET Student Loans Program Risk Management Plan and the VET FEE HELP Scheme Risk Management Plan outline the risks relevant to the program. As at August 2018, these plans included eight VSL risks and six VFH risks, as outlined in Table 3.2.

Table 3.2: VSL and VFH risk ratings

Risk category

VFH risks

VSL risks

Business continuity management

1 high

1 medium

Finance

1 high, 1 extreme

-

Fraud

1 high

-

Implementation / service delivery

-

1 high

Legal and regulatory

1 medium

1 high

Planning / policy

-

1 high

Planning / processes

-

1 high

Reputational

1 medium

-

Resource management

-

1 high

Stakeholder / relationship

-

1 medium

Technology

-

1 extreme

     

Source: ANAO analysis of the VET Student Loans Program Risk Management Plan (updated 21 August 2018) and the VET FEE-HELP Scheme Risk Management Plan (updated 31 August 2018).

3.36 To complement the program risk management plan, project risk plans were also developed during the early stages of the VSL program (over the period November 2016 to February 2017). As at March 2017, eight project risk plans for the former VFH scheme and VSL program had been developed for: student grandfathering; digital media campaign for the VSL program; communication strategy; VFH and VSL payments transition; IT two-pass process; provider full approvals; provider closures; and the IT interim solution. As the program has matured, the department advised that five of the initial eight project risk plans have been retired and replaced with risk plans using Risk Active related to ongoing operational, compliance and monitoring business-as-usual activities.

3.37 The ANAO examined the VSL Risk Strategy and program and project risk management plans. The VSL Risk Strategy and its related plans have appropriately identified the key risks, personnel responsible for the management of program and project risks, roles and responsibilities, and risk escalation protocols that outline how issues are raised and resolved. In contrast, a robust risk management process was not in place under the former VFH scheme.23

3.38 The VET Student Loans Program Governance Committee provided oversight of risk management quality assurance.24 During the early implementation phase of the VSL program, this committee met weekly and considered risks linked to key implementation activities. Program risks were reviewed monthly and project risks analysed quarterly. Program reporting in the initial stages of implementation focused on the progress of key activities, rather than the management of key program risks.

3.39 Over time, the program’s risk management arrangements have matured, with the Performance and Risk Management Committee becoming more focused on the consideration of key program risks.

Compliance

3.40 The VSL compliance requirements are constituted through legislation and regulation.25 Underpinning this legislation has been the development of a compliance strategy, related compliance management plans and a draft general compliance business plan.

3.41 The compliance strategy notes that risk is an integral component of compliance and that a risk-based approach allows the department to prioritise resources and pursue appropriate compliance actions. For example, the department informed the ANAO that its activities are based on elevated risk factors that identify suspected non-compliance or fraudulent practices. The strategy links to the risk management plans. The strategy does not detail how compliance activities are prioritised or the process by which the level of resources are allocated to these activities.

3.42 The key components of the department’s compliance activities are prevention, detection and response.

3.43 In terms of prevention, approved VSL providers are required to satisfy a range of eligibility and assessment criteria to qualify for, and maintain, approved course provider status under the VSL Act. As noted earlier, the VSL program also introduced restricted course eligibility, the capping of loan amounts payable for particular courses, and the setting of payment caps on approved course providers, which contribute to its prevention strategy.

3.44 The department has sound processes in place to monitor and enforce the ongoing compliance of approved providers against the program’s overall requirements and the specific conditions imposed on each provider as part of their approval. The department’s prevention activity is supported by the Program Assurance Plan, a systematic and data-rich plan that aims to address potential non-compliance issues before they develop into compliance issues. Education and communications also form part of this strategy.

3.45 In terms of detection, the department states that it operates through four main mechanisms:

  • payments – analysis of self-verified data from providers for any potential anomalies or instances of non-compliance before making payment;
  • a range of audit activities:
    • desktop audits — assesses provider and departmental data, provider websites, media reports and other information sources for compliance against information publication. These audits, undertaken on a rolling basis, do not generally involve contact with providers during the audit process. As at 14 August 2018, of the 60 providers audited in 2018, 35 providers were compliant with publication requirements, 24 were in contravention of a small number of minor publication requirements and a compliance notice had been issued to one provider;
    • provider audits (health checks) — assesses provider practices, such as student marketing, to address potential patterns of non-compliance and identify providers with data anomalies and issue. As at 14 August 2018, 40 provider health checks had been undertaken in 2018, with five providers identified as having data pattern concerns; and
    • site audits — assesses whether providers are complying with their requirements under the relevant law and whether students enrolled by providers are bona fide students. Since late 2017, the department has commissioned the Australian Skills Quality Authority (ASQA) to undertake 12 compliance audits — see Box 2 for further details.
  • surveys and complaints – to ascertain students’ experiences and feedback on course marketing and enrolment practices. Since 1 July 2017, VSL complaints have been managed by the VET Student Loans Ombudsman (VSL Ombudsman).
  • the gathering and analysis of relevant information and intelligence from a number of regulators, Australian and state government agencies to assist in detecting potential non-compliance, and by reviews of media and other open-source information – see next section for further details on collaboration between the department, key regulators and other government entities.

Box 2: Compliance audit activity undertaken by ASQA on behalf of the department

In November 2017, the department commissioned ASQA to undertake 12 provider compliance audits in 2017–18, with a further 12 flagged for 2018–19. ASQA is authorised to conduct these audits under the VSL Act, with protocols agreed through a memorandum of understanding (MoU) with the department. ASQA and the department use a range of risk indicators to select which VSL providers to audit.26

The department advised that, as at 20 August 2018, ASQA had completed three audits, with the reports provided to the department for action. One audit was in the process of being finalised while provider feedback had been sought on the other two reports.

In June 2018, the department developed an assessment management plan to manage the assessment and provision of recommendations regarding compliance decisions following the receipt of ASQA-conducted compliance audit reports. The plan outlines a process for assessing audit report findings and the appropriate action to take.

There is evidence of some providers not fully complying with all their obligations under the relevant Acts and VSL Rules.

Source: ANAO analysis of Department of Education and Training documentation.

3.46 A VET loans provider compliance table has been developed and is maintained by the department, which includes comprehensive details of those providers the department, the Australian Competition and Consumer Commission (ACCC)27 and ASQA have investigated and/or are taking action against. This provides a clear picture of which entity is undertaking which form of activity, which assists the three entities to manage their remit.

3.47 The third component of the department’s compliance strategy is response. This involves investigations to address issues of serious non-compliance, undertaking of a range of compliance actions, and imposing civil penalties, as prescribed under the VSL Act. As at 14 August 2018, there have been no investigations of VSL providers.

3.48 One of the key challenges in VSL program compliance has been the process by which operational matters related to lower levels of non-compliance become significant enough to be escalated as compliance issues. The department has drafted a General Compliance Business Plan 2018–19 with the aim to better coordinate and delineate the escalation process. While the plan is in draft, the department has advised that the activities outlined are now business as usual activities and that the plan will be finalised as part of the department’s business planning process.

Have appropriate mechanisms been established to operationalise regulatory roles and responsibilities?

The department has put in place appropriate mechanisms aimed at supporting improvements to the coordination and effectiveness of VSL program regulatory activity.

3.49 The department collaborates with key regulators involved in the VET sector through: formal MoUs, which outline the roles and responsibilities of each party; regular meetings to discuss emerging issues and the recent activities of each entity; and informal exchanges and sharing of information and intelligence, as agreed within the respective MoUs. Table 3.3 outlines the key features of the arrangements.

Table 3.3: Arrangements between the department and key entities

Entity

Key features

Australian Skills Quality Authority

(MoU signed May 2017)

  • agreed protocols for sharing data, information, student surveys, audit and investigation progress, outcomes and reports as a result of ASQA’s and the department’s respective regulatory activities
  • coordinate and collaborate on audits, investigations and ASQA’s regulatory activities to identify non-compliance with regulatory requirements
  • collaborate and notify each other of enforcement actions (including notice of intention to issue infringement notices prior to issue)
  • members of a VSL working group that meets monthly to share intelligence on areas of risk and emerging risk (from an analysis of department data and ASQA intelligence) and to discuss ASQA-conducted compliance audits of approved VSL providers

Tertiary Education Quality and Standards Agency

(MoU signed 28 October 2016)

  • sharing of higher education data and information to assist in the administration of TEQSA’s and the department’s respective and joint functions (including risk, compliance and financial information)
  • notifying each other of regulatory enforcement actions, including changes to application and approval statuses for providers

Australian Competition and Consumer Commission

(MoU signed March 2016)

  • sets out agreed arrangements for the future funding of litigation conducted by the ACCC and the department in relation to proceedings against VFH providers alleged to have contravened the Australian Consumer Law
  • meet regularly as part of the IDC arrangements

VSL Ombudsman

(MoU signed May 2017)

  • sets out the funding for the functions of the VSL Ombudsman; information exchange arrangements and reporting requirements; and outlines the arrangements for the transfer of complaints function from the department to the VSL Ombudsman effective from 1 July 2017
  • staff meet at least quarterly to discuss relevant issues
  • the VSL Ombudsman provides a quarterly report on its complaints management operations. Subject to the requirements of the Ombudsman Act 1975, the VSL Ombudsman will report other matters, including systemic bad behaviour of providers or uncooperative providers
   

Source: ANAO analysis.

3.50 The department also gathers information and intelligence to assist it to detect potential non-compliance and exploitative behaviours. This is sourced from a review of media and other open-source information and information sharing arrangements with:

  • entities such as the Australian Taxation Office, Department of Human Services and the Department of the Prime Minister and Cabinet;
  • regulators such as ASQA, TEQSA, ACCC and state and territory-based Offices of Fair Trading;
  • tuition assurance scheme operators; and
  • the VSL Ombudsman (in its role as administrator of the VSL external dispute resolution scheme).

3.51 In ANAO interviews with ASQA, the VSL Ombudsman, the Department of Finance and the Australian Taxation Office, officers noted that their respective roles and responsibilities are clearer and more focused as a result of legislative changes associated with the VSL program. They also stated that a more collaborative and coordinated approach has been adopted by the department for managing the VSL program.

Is an evaluation strategy in place to assess whether program objectives are being achieved?

The department has developed an evaluation strategy and has completed a number of reviews of aspects of the program. Early indications are that the program is progressing well against its objectives. The reviews note areas for further analysis and potential adjustments to the program’s parameters.

3.52 The department has reviewed the progress of VSL, or aspects of the program, at several stages. The key reviews undertaken to date are discussed below.

Internal health check review

3.53 The department’s internal audit section undertook a health check review in early 2017. The objective was to provide assurance to the department’s Executive on the early implementation of the program. The review concluded that the department had made considerable progress in developing a governance framework for implementing the Program. It made four operational recommendations and three management improvement suggestions. The recommendations were aimed at improving risk assessments and reporting, performance management and program compliance.

3.54 The department monitors the implementation of internal audit recommendations through reports to the department’s Audit and Assurance Committee and in progress reports periodically requested by department senior executives. The committee papers recorded that the recommendations were closed by 1 March 2018.

Evaluation strategy

3.55 As discussed earlier, the VET Student Loans Evaluation Strategy provides a framework for the PIR, and includes key evaluation questions, review timetable and key performance indicators.

3.56 The first phase of the department’s three year evaluation was based on data analysis, a range of stakeholder interviews and student surveys. The resultant PIR report (December 2017) concluded that the VSL program is meeting its objective of fiscal sustainability, is addressing provider misconduct and early indicators suggest that training quality has improved. The review also concluded that some program features designed to increase integrity have had an adverse impact on other objectives, for example, the focus on fiscally sustainability and unscrupulous providers needs to be balanced with student access to the VSL program.

3.57 The report made 22 recommendations related to the objectives of the program. The department agreed to 10 recommendations, eight were agreed in principle, two were noted and two were not agreed. The department monitors PIR recommendation implementation using a spreadsheet that notes the proposed response, timeframe for implementation, and manager assigned the responsibility to action the recommendation. Following on from the PIR, the department has developed several papers, including papers on earned autonomy for providers and improving student access to the program, and is pursuing a number of issues to inform potential options.

3.58 One of the recommendations related to a key stakeholder concern – the two-state eligible course list process.28 The review recommended that the department streamline the process. The department agreed in principle to this recommendation, but noted that the 2017 eligible courses and caps review had concluded that there was not enough evidence to warrant significant change. The spreadsheet notes that the department will consider how to operationalise this recommendation in the context of the review of courses and loan caps (see below) and the final PIR evaluation, due to be completed by the end of 2019.

3.59 The next phase of the evaluation is scheduled for completion by 30 November 2018, with a final evaluation report to be completed by 30 November 2019. The 2018 and 2019 reports are to focus on how VSL has addressed the policy intent of the program.

Review of courses and loan caps

3.60 The VET Student Loans – Course list and loan cap methodologies, Final Report concluded that the early phases of the VSL program appear to be broadly in line with its policy intent, and that there was little evidence on which to base immediate actions to change the eligible course list and loan cap settings.29 The report notes that the conclusion should be understood in the context of the limited amount of data available and the short time period in which the program has been operating. The report also notes that marginal changes could be made in the near term, for example: considering approaches for rebasing loan caps, in collaboration with states and territories; and developing a clear monitoring and evaluation approach to overcome limitations in the current dataset.

Implementing changes

3.61 It is reasonable for the department to adopt an evidence-based approach to adjustments, including how the new program and its significant changes (relative to the VFH scheme) is progressing. Effectively monitoring the impact of the program, including changes in provider and student behaviour, and taking action as appropriate should mitigate the risk that the type of problems that beset the former VFH scheme will eventuate.

Appendices

Appendix 1 Entity response

Entity response

Appendix 2 Feedback from stakeholder consultation

1. The Australian National Audit Office (ANAO) engaged with a range of VSL-approved service providers (and in the case of the publicly-funded TAFE sector, the entities to which they report) seeking their views on the design and implementation of the VSL program in relation to:

  • the consistency, fairness and transparency of the departments’ activities;
  • the effectiveness of the departments’ engagement with your entity in relation to the design and early implementation of the new loans program; and
  • any other matters related to the two high level audit criteria.

2. Thirty-two submissions were received, representing over 900,000 of VET students enrolled in the government-funded system, as follows:

  • three State entities responsible for their TAFE sectors;
  • 13 public VET providers;
  • 14 private providers, including one not-for-profit; and
  • two other public submissions.

3. The ANAO’s stakeholder consultation process also included face-to-face interviews and teleconferences with stakeholders across Australia and the establishment of an online portal through which submissions could be made to the audit. The information gained through interviews and submissions provided additional information for the ANAO to discuss with VSL program personnel within the department and were considered in developing audit findings.

4. Table A.1 represents the key issues identified through the ANAO’s stakeholder consultation processes as received by the ANAO and the action or position that the department has taken.

Table A.1: Stakeholder feedback on VSL program design and implementation

Issue

Feedback received

Department’s position / action

Strategic intent

  • Recognition that VFH scheme should cease and that the continued removal of unscrupulous providers has been a positive outcome
  • Government intent to introduce a new model, with appropriate protection features for students
  • Raising the bar (tougher criteria) for providers intended to remove unscrupulous providers

Stakeholder consultation

  • Design process more effective if the department had consulted with stakeholders on the actual proposed VSL model, not just the early concepts outlined in the redesign discussion paper
  • Urgent changes to VFH scheme were necessary. It was unfortunate though that the department did not genuinely engage with states and territories who are co-owners of the national training system
  • It was a Government decision not to consult further (given the same reform themes were apparent) and introduce the new program from 1 January 2017

Communications with providers and students

  • Unclear language and instructions used in student and provider communications
  • Inadequate communications provided to students around separate enrolment and VSL processes
  • Email advice from the department to providers were not timely (four weeks or more to respond), too generic and often did not respond to the specific concerns raised
  • Department staff communicating with providers appeared to lack the support and knowledge to assist providers with questions on new program
  • Provider webinars did not provide any beneficial information or advice. Where enquiries were noted by facilitators for further review, no response ever received
  • Large number of automated emails to students created confusion and uncertainty. Students responded by contacting providers for support
  • Stakeholder engagement would be improved if the department provided the option to discuss VSL program queries directly with staff via the phone
  • A communications strategy was developed for the program and is monitored by the department as to its effectiveness
  • Communications language and content is continually under review
  • Training of key staff (for example, telephone operators) was provided
  • Timelines dictated by the Government’s announcement of 5 October 2016 to commence the new program from 1 January 2017

Student progression, and opt-in and census processes

  • Providers required by department to follow up student non-compliance to ensure they got paid
  • Opt-in process for grandfathering VFH students was difficult for students to understand, with no prior consultation with providers. Many students made contact with their providers given the difficulties they experienced
  • Progression form does not consider information captured by state government surveys
  • Opt-in process response rate (at around 12 per cent) better than anticipated
  • The progression processes are a key design feature of the program
  • The department agreed to a ‘soft roll-out’ of the progressions process. Payments will not be made (and students will not incur the VSL debt) if students fail to complete two consecutive progression forms, or in cases where the students indicated they never commenced study, or withdrew.

Eligible course list review and loan caps methodology and processes

  • All VET-accredited courses should be eligible. Concerns about quality addressed by ASQA provider and course accreditation processes
  • Eligible course criteria too narrow — do not recognise community service and creative arts courses and omit other courses that develop creative, innovative and entrepreneurial skills
  • Course list two-state rules ignores industry priorities unique to a single state – rather than imposing this rule, loans should be available for all courses subsidised in any state
  • Loan caps fail to cover the full cost of delivering a course but rather reflect fees charged for students in a subsidised course. The result is fee gaps, which can act as barriers to entry and disrupt valid fee-for-service (mostly private provider) markets
  • Caps set to Independent Pricing and Regulatory Tribunal formula for NSW metropolitan area but differences in delivery costs exist nationally
  • The department commissioned an eligible courses and loan caps methodology review in 2017. No recommendations for change were proposed
  • The eligible course list and loan caps are to be considered further as part of the program’s three year evaluation

Tight implementation timelines

  • Late passing of legislation disadvantaged students who had enrolled in courses prior to December 2016 – left an administrative burden with providers to follow up with students
  • Pre-Christmas announcement of VSL Rules left providers with no time to update systems, communicate with students and train staff
  • Timelines dictated by the Government’s announcement of 5 October 2016 to commence the new program from 1 January 2017

Students making more informed choices but taking courses that are VSL-eligible only

  • Imposition of loan caps for first time led to the emergence of fee gaps, distorting student choices
  • Courses being tailored to reduce course costs to fall within loan caps. Results in reduced employability / quality from studying such courses
  • Fee gaps and the type of courses commenced are monitored by the department

Administrative and reporting burdens

  • Duplication of reporting - to the department and state governments
  • Progression process costly in terms of provider follow-up of students, with income lost or delayed
  • Unclear how department uses the data collected
  • While accepting the need for change, additional controls under VSL place an unnecessary administrative burden on low-risk providers
  • Cost burdens monitored as part of department’s evaluation strategy
  • The department is continually examining ways to reduce red tape

Provider application processes

  • VSL provider application process has successfully removed a substantial number of high risk providers from the program
  • Paperwork required to support applications is onerous and expensive to compile in terms of the time taken and resources to complete
  • The bar has been set too high and impacts more on non-TAFE sectors
  • Recent internal audit review of these processes recommended a simplification of information requirements
  • Raising the bar (tougher criteria) for providers intended to remove unscrupulous providers
  • Composition of providers has changed significantly, in large part due to removal of unscrupulous providers
     

Source: 32 submissions received by the ANAO and interviews with stakeholders, including members of the VSL Implementation Advisory Group and state departments responsible for TAFEs.

Appendix 3 VSL key performance indicators

Table A.2: ANAO assessment of VSL key performance indicators

VSL objective

Key performance indicator

Characteristics of good performance information

 

 

Relevant

Reliable

Complete

The program is fiscally sustainable and contributes to national economic growth

VET Student Loans (VSL) payments are less than VFH and does not exceed the $2.07 billion legislated program cap

Mostly

Yes

Yes

Proportion of new debt not expected to be repaid for VSL is lower than VFH

Yes

Yes

Yes

The program removes financial barriers to training and improves equity of access to higher level VET

Downward pressure on VSL tuition fees initially and the ongoing rise in tuition fees are consistent with CPI

Yes

Yes

Yes

VSL fees are not inflated for specific groups such as:

  • Indigenous;
  • regional and remote;
  • low socioeconomic

Analysing average student gap costs

Yes

Yes

Yes

The program promotes the delivery of quality and affordable training for students;

Higher unit completion rates

Yes

Yes

Yes

Higher course completion rates

Yes

Yes

Yes

Lower number of VSL complaints on quality per 1000 students

Yes

Yes

Yes

Greater student satisfaction through satisfaction survey

Yes

Yes

Yes

Lower number of VSL complaints per 1000 students

Yes

Yes

Yes

Number of providers unsuccessful for VSL due to insufficient evidence re industry links

Mostly

Yes

Yes

The program balances industry needs, employment outcomes and student choice

Higher number of VSL students studying to increase employment outcomes

Mostly

Yes

Yes

Employment outcomes and views

Mostly

Yes

Yes

The program is student centred through adequate protection for students (particularly from disadvantaged backgrounds) and access to information that enables informed decision making

VSL fees are not inflated for specific groups such as:

  • Indigenous;
  • regional and remote;
  • low socioeconomic

Yes

Yes

Yes

Percentage of students enrolled in VSL from:

  • Indigenous;
  • regional and remote;
  • low socioeconomic

Yes

Yes

Yes

The program has integrity, manages risk and promotes confidence in the regulated VET market

Compliance survey – questions regarding knowledge of student loan

Yes

Yes

Yes

Number of VSL providers

Yes

Yes

Yes

Lower number of multiple course enrolments under VSL

Mostly

Yes

Yes

Payments to providers accurately reflects student engagement

Mostly

Yes

Yes

Lower number of VSL students who did not complete a single unit of study

Mostly

Yes

Yes

         

Note: Assessed against the Department of Finance’s Quick Reference Guide - RMG 131 Developing good performance information, April 2015, which describes the characteristics of good performance information as:

  • Relevant — performance information should clearly state who benefits and how they benefit from the entity’s activities.
  • Reliable — performance information should use information sources and methodologies that are fit-for-purpose and verifiable.
  • Complete — performance information should help stakeholders judge whether the purposes of an entity are being achieved.

In the instances when a KPI has been rated as mostly relevant, the link between the KPI and the project’s outcomes and/or beneficiaries could have been more clearly stated.

Source: ANAO analysis of VSL key performance indicators.

Appendix 4 Performance against key performance indicators

Table A.3: VSL performance against quantitative indicators

Key performance indicator

VSL Performance

 

Q3 2017

Q4 2017

Q1 2018

VET Student Loans (VSL) payments are less than VFH and does not exceed the $2.07 billion legislated program cap

Loans issued ($m)

$70

$42

$58

Downward pressure on VSL tuition fees initially and the ongoing rise in tuition fees are consistent with CPI

Average tuition fees / EFTSL

$7,529

$8,152

$8,023

Analysing average student gap costs

Average gap cost as proportion of tuition fee / EFTSL

5.4%

6.5%

8.1%

Higher unit completion rates

Unit of study completion rate (% EFTSL)

61.8%

87.5%

80.9%

Greater student satisfaction through satisfaction survey

Student satisfaction rate

87.1%

84.3%

84.1%

Higher number of VSL students studying to increase employment outcomes

Students studying to increase employment outcomes

61.2%

66.3%

67.6%

Percentage of students enrolled in VSL from:

  • indigenous
  • regional and remote
  • low socioeconomic areas

Number of Indigenous Students

2.0%

2.1%

2.1%

Number of regional and remote students

5.3%

6.1%

5.4%

Number of low SES students

18.8%

19.1%

19.3%

Payments to providers accurately reflects student engagement

Number of payment records rejected

3.0%

6.8%

2.3%

       

Note: KPIs as reported in the VET Student Loans Quarterly Performance Report for Quarter 1 2018.

Source: ANAO analysis of VSL key performance indicators.

Footnotes

1 On 1 July 2015 measures introduced included: tightening marketing and recruitment; improving available information; streamlining debt waiver and revocation processes; and strengthening the assessment and scrutiny of providers. On 3 December 2015, the Government announced additional measures, effective 1 January 2016, which included: freezing the total loan limit for existing providers at 2015 levels; introducing new entry requirements for training providers wishing to offer VFH loans; moving to payment in arrears for certain providers; and pausing payments to providers for new enrolments where concerns existed around performance.

2 Minister for Vocational Education and Skills, Reforms to proposed VET students pass the Parliament, media release, 3 December 2015.

3 Department of Education and Training, VET Student Loans Program: Program Implementation Plan – Version 6, September 2017, p. 6. See paragraphs 2.3.to 2.5 for a discussion about the program’s objectives.

4 Australian Government, Portfolio Additional Estimates Statements 2016–17 — Education and Training Portfolio, Commonwealth of Australia, Canberra, February 2017, p. 16.

5 Australian Government, Budget Measures: Budget Paper No. 2: 2018–19, Commonwealth of Australia, Canberra, 2018, p. 89. Of the $36.2 million budgeted, $27 million was allocated to the Department of Human Services, as the program uses that department’s existing IT platform.

6 Australian Government, VET Student Loans Rules 2016, Division 2, Section 155 — Cap on amount of VET student loans. The cap on the total amount of VET student loans that can be approved for each of the calendar years 2017, 2018 and 2019 is $2.07 billion. The cap was derived from the department’s established Higher Education Loan Program (HELP) costing model based on agreed estimates of the expected expenditure on the former VFH scheme.

7 Department of Education and Training, Six-monthly report: 1 January 2018 to 30 June 2018, Canberra, August 2018, p. 5. The 180 approved course providers comprised: 23 TAFEs; 13 other public organisations; and 144 private providers.

8 The loan caps are indexed annually, as provided for by the VET Student Loans Act 2016 and according to the provisions set out in the VET Student Loans (Courses and Loan Caps) Determination 2016. As at 1 January 2018, the three main loan caps were: Band 1 — $5,075, Band 2 — $10,150 and Band 3 — $15,225. Schedule 2 in the Determination refers to approved aviation courses, with the cap set at $76,125. Department of Education and Training, VET Student Loans Courses and Loan Caps fact sheet, Canberra, May 2018.

9 Government of South Australia, Department of the Premier and Cabinet, Cabinet Guide No. 2 Developing policy: Turning ideas into action, Adelaide, January 2015.

10 In this report, the objectives are taken to be those described in paragraph 2.4.

11 Education and Employment References Committee, The Senate, Getting our money’s worth: the operation, regulation and funding of private vocational education and training (VET) providers in Australia, Canberra, October 2015. The report made 16 recommendations, focused mainly on the need to review the former VFH scheme to improve its operation and regulation.

12 Refers to the process whereby students are required to engage with the VSL online portal in February, June and October each year. This measure, introduced from 1 July 2017, is part of the strengthened monitoring and compliance measures, and serves to increase the department’s visibility and assurance of student engagement, ensuring taxpayer funds are directed to students that are progressing with studies and ensuring payments are not made for non-genuine students.

13 Deloitte Access Economics, VET Student Loans – Course list and loan cap methodologies, Final Report, December 2017.

14 The student engagement and progression process and a more rigorous application and approval process for registered training organisations commenced on 1 July 2017.

15 The Federal election was announced on 8 May 2016 and the new government was sworn in on 18 July 2016.

16 Department of Education and Training, VET FEE-HELP scheme redesign, Ministerial brief, 4 August 2016.

17 This analysis was also used in later advice to the Government when seeking agreement to the VSL program.

18 The objectives are those described in paragraph 2.4, and do not include the core objective and aims described in the Program Implementation Plan.

19 Eligible students who commenced their studies prior to 1 January 2017 were given the option of continuing their studies under the VFH program until 31 December 2017 and, for some students, in 2018.

20 Standing items on the Committee’s agenda included: two-pass IT Solution, data and systems, payments, student grandfathering, communications strategy, temporary provider approvals, full provider approvals, provider closures, the review of courses and general risks and issues.

21 Department of Education and Training, Communications Plan - A plan to support the implementation of the Australian Government’s VET Student Loans Program, February 2017.

22 There are two key websites relevant to the VSL program: www.studyassist.gov.au, which provides information about government assistance for financing tertiary study; and www.myskills.gov.au, the national directory of VET organisations and courses.

23 Auditor-General Report No.31 of 2016–17 Administration of the VET FEE-HELP Scheme, Canberra, December 2016, p. 8.

24 Now the Performance and Risk Management Committee.

25 The legislative requirements that underpin the VET Student Loans program and VET FEE-HELP scheme include: the VET Student Loans Act 2016; the VET Student Loans Rules 2016; the Higher Education Support Act 2003; the Higher Education Support (VET) Guideline 2015; and Regulatory Powers (Standard Provisions) Act 2014.

26 Additionally, providers needed to be VFH providers, transitionally approved and VSL approved providers with more than 100 VSL student enrolments.

27 The ACCC is responsible for administering the Australian Consumer Law, and promoting competition, fair trade and consumer protection in Australian markets. The department is engaged with the ACCC to investigate a number of VFH providers for alleged misrepresentations and unconscionable conduct in contravention of the Australian Consumer Law.

28 To be VSL-eligible courses, they must be current, be on at least two state or territory skills lists, be science, technology, engineering or mathematics related, or be tied to a licenced occupation requirement. The aim of the ‘two-states’ rule is to ensure that VSL loans are only provided for courses that are recognised by states and territories as being worthwhile. Stakeholders felt that the process was too burdensome and suggested the department consider alternative approaches to approving courses.

29 Deloitte Access Economics, VET Student Loans – Course list and loan cap methodologies, Final Report, December 2017.