The audit objective was to assess whether the Department of Employment effectively designed and monitors the progress of the jobactive program.

Summary and recommendations

Background

1. The current federal employment services program, jobactive, became operational on 1 July 2015, and replaced the previous employment services model, Job Services Australia. The objectives of jobactive are: helping job-seekers find and keep a job; helping job-seekers move from welfare to work; helping job-seekers meet their mutual obligations;and jobactive organisations delivering quality services.

2. The Department of Employment (Employment, or the department) administers jobactive and there are approximately 750 000 job-seekers at any given time in the program. Employment is forecast to spend approximately $7.3 billion over the contracted five year period of jobactive. The delivery of employment services is contracted to 65 providers who deliver one or more of the five services of the program: jobactive employment services; New Enterprise Incentive Scheme; Work for the Dole Coordinators; Harvest Labour Services; and the National Harvest Labour Information Service.1

3. Employment oversees the delivery of the contracted services, and has established compliance and performance monitoring arrangements to provide assurance that the services are delivered as expected.

Audit objective and criteria

4. The audit objective was to assess whether Employment effectively designed and monitors the progress of the jobactive program. To form a conclusion against the audit objective, the Australian National Audit Office (ANAO) adopted the following high-level criteria:

  • the jobactive program was designed to support the achievement of the Government’s policy objectives; and
  • the department effectively monitors the progress of the jobactive program against the Government’s policy objectives.

Conclusion

5. Employment effectively managed the design of jobactive and its monitoring approach has resulted in a reasonable level of assurance being obtained that the program is being delivered as required.

6. There was a sound reason for redesigning the employment services model, the governance arrangements established by the department were comprehensive, stakeholders were adequately consulted and the Minister was briefed on a range of design and implementation topics. Since the implementation of jobactive on 1 July 2015, the department has reviewed and amended elements of the program’s design.

7. Employment has established a suitable committee structure to oversee the jobactive program and the department has identified and managed risks at the program and provider level. The prioritisation of activities to deliver the program could be improved to ensure that required activities are completed. The principles-based guidelines do not always clearly articulate Employment’s expectations of providers.

8. Employment has obtained a reasonable level of assurance that the jobactive program is being administered as designed and expected. The Assurance Strategy for the jobactive 2015–2020 contract includes new program assurance elements that have strengthened the department’s monitoring of employment service provider’s compliance with contractual obligations. While the department is currently reviewing the operation of the Assurance Strategy, the review does not address some of the key elements of the program, including whether the strategy reflects the department’s preferred level of compliance.

9. The performance frameworks for the five jobactive services measure the performance of providers. The Key Performance Indicators developed by the department, which align with program objectives, have been developed for three of the five services, but performance targets have only been established for one of the services. Employment has an evaluation strategy for jobactive, but it does not address some aspects of the program, including contract management and the Star Ratings.

Supporting findings

Program design

10. Employment identified significant weaknesses in the former Job Services Australia model, which was the key reason for developing the new employment services program, jobactive. These weaknesses were appropriately considered when designing the jobactive program. The governance arrangements established by the department for the design and implementation of the jobactive program were comprehensive, with strong senior executive engagement and support from the 16 working groups that had responsibility for developing the policy of specific subject areas.

11. During the design of jobactive, stakeholders were consulted and Employment considered their views when designing the program. Employment provided sound advice to the Minister for Employment and the Assistant Minister for Employment on a range of design considerations and associated risks.

12. In consultation with providers and peak bodies, Employment has made several changes to the design of the jobactive program. The department established working groups or external reviews to guide improvements to the program.

Governance arrangements for ongoing management

13. Employment has established an appropriate governance structure to provide oversight and make decisions about the ongoing management of jobactive. Improved prioritisation of the activities necessary for effective delivery of the jobactive program is needed to ensure that objectives are being achieved.

14. Employment has established an appropriate framework to manage risks to the jobactive program. While the management of high-level program risks has been generally sound, there is scope for the department to improve the timeliness of monitoring and mitigation actions for identified provider risks. Further, the department should make greater use of compliance and performance data to identify and manage provider risks.

15. Employment has established a principles-based approach for the jobactive guidelines to allow providers to deliver flexible solutions tailored to an individual job-seeker’s circumstances. On occasion, this has led to some inconsistency in the interpretation and application of the guidelines.

Program assurance of providers

16. Employment has established a comprehensive framework to manage provider compliance that incorporates approaches to prevent, deter, detect and correct non-compliance. Employment has a framework to monitor employment service providers’ compliance with contractual obligations. The framework enables the department to obtain a reasonable level of assurance about employment service providers’ compliance. There is scope to improve the analysis of complaints data, the provision of Rolling Random Sample results to providers in a timely manner, and the effective prioritisation of targeted assurance activities.

17. Employment manages non-compliance through the Remedial Action Framework. As at November 2016, six incidents of non-compliance had been investigated under the framework. The department has not analysed the reasons for the low number of recorded non-compliance cases.

18. Employment has not assessed the effectiveness of the Assurance Strategy, but has commenced a review to identify best practice and focus resources on areas of highest priority.

Performance measurement, reporting and evaluation

19. Employment has performance frameworks in place for the five services which measure the performance of providers. Key Performance Indicators for employment service providers, New Enterprise Incentive Scheme providers and Work for the Dole Coordinators align to the objectives of the jobactive program, but performance targets have only been set for Coordinators.

20. The Star Ratings were designed to be used by job-seekers and employers to inform their choice of employment service provider, and by Employment for business reallocation. The department does not have a strategy in place to assess whether the Star Ratings are used by job-seekers or employers to influence their choice of employment service provider. The department has used the Star Ratings for the first round of business reallocations.

21. Employment monitors progress against the targets in the Portfolio Budget Statement and Corporate Plan through performance reporting to the relevant governance committee. The department publicly reported on progress against the targets in its 2015–16 Annual Report.

22. Employment has developed an evaluation strategy for two of the five services of the jobactive program, including jobactive employment services, which adopts a staged approach to evaluation. The strategy does not cover three of the services or other key elements of the jobactive program, such as contract management and the Star Ratings.

Recommendations

Recommendation no.1

Paragraph 3.10

The Department of Employment should implement a risk-based approach to prioritising the activities required to effectively manage and monitor the delivery of the jobactive program.

Department of Employment’s response: Agreed.

Recommendation no.2

Paragraph 4.44

The Department of Employment should assess whether the current compliance regime is structured to effectively and efficiently detect and manage non-compliance, and adjust as appropriate.

Department of Employment’s response: Agreed.

Summary of entity response

23. The summary response from Employment is provided below, with the full response included in Appendix 1.

The department welcomes the audit’s conclusions and the overall positive findings. The department is particularly encouraged by the ANAO’s recognition that the jobactive program design was effectively managed, with appropriate stakeholder consultation. I appreciate the audit’s acknowledgment of the department’s sound reasoning for the redesign of employment services, and the establishment of comprehensive governance arrangements. In addition, I welcome the ANAO’s conclusion the department has obtained a reasonable level of assurance that the jobactive program is being administered as designed and expected.

In terms of areas of potential improvement, particularly with regard to our approach to the prioritisation of program management activities and the current compliance regime, the department is taking steps to consider and address these issues. This includes our current Assurance Review and through ongoing monitoring of the Delivery and Engagement Group calendar and protocol.

1. Background

Introduction

1.1 The Australian Government has outsourced national employment services to employment service providers since 1998.2 The current federal employment services program, jobactive, became operational on 1 July 2015. The objectives of the jobactive program are: helping job-seekers find and keep a job; helping job-seekers move from welfare to work; helping job-seekers meet their mutual obligations3;and jobactive organisations delivering quality services.

1.2 Participation in jobactive is compulsory for those job-seekers who receive income support payments, such as the Newstart Allowance or Disability Support Pension, and have been identified by the Department of Human Services as being able to actively look for work.4 Individuals not receiving an income support payment, or those in receipt of income support without compulsory requirements may volunteer for one period of jobactive services for up to six months.5 The jobactive program is administered by the Department of Employment (Employment or the department).

Changing the employment services model

1.3 The jobactive model replaced Job Services Australia, which operated from 1 July 2009 to 30 June 2015. The development of jobactive commenced in December 2012, with the contracts awarded in March 2015. A timeline of the development and implementation of the program is provided in Figure 1.1.

Figure 1.1: Timeline for the development of jobactive

Source: ANAO analysis of departmental documentation.

1.4 The key differences between the employment services model of Job Services Australia and jobactive are outlined in Table 1.1.

Table 1.1: Key differences between the Job Services Australia and jobactive models

 

Job Services Australia
(2012–15)

jobactive
(2015–20)

Objectives

  • Investing in the skills of unemployed Australians to help them find and keep a job
  • Ensuring that government assistance supports workforce participation and economic and social inclusion
  • Helping job-seekers find and keep a job
  • Helping job-seekers move from welfare to work
  • Helping job-seekers meet their mutual obligations
  • jobactive organisations delivering quality services

Contract length (years)

3

5

Number of job-seeker classification streams

4

3

Payment model (administration fees to outcome payments)

67:33

48:52

Administration fees paid every:

13 weeks

6 months

Outcomes payments at (weeks):

0, 13 and 26

4,12 and 26

Number of employment regions

110

51

Number of employment service providers

81

44

Total number of providersa

116

65

     

Note a: Contracted providers in the jobactive program include employment service providers (who deliver employment assistance to job-seekers), Work for the Dole Coordinators, New Enterprise Incentive Scheme providers, Harvest Labour Service providers and the provider of the National Harvest Labour Information Service.

Source: ANAO from departmental documentation.

1.5 Employment reported that the changes made to the employment services program have been designed to:

  • ensure that job-seekers better meet the needs of employers;
  • increase job-seeker engagement by introducing stronger mutual obligation requirements;
  • increase job outcomes for unemployed Australians; and
  • reduce prescription and red tape for service providers.6

The jobactive program

1.6 The jobactive program is delivered in 51 employment regions across Australia, as shown in Figure 1.2. Job-seekers who are eligible for employment assistance, but live outside of these employment regions, are to be serviced by the Community Development Programme (which is administered by the Department of Prime Minister and Cabinet).

Figure 1.2: The employment regions for the jobactive program

Note: Norfolk Island is not pictured, but as of 1 July 2016 was included in the North Coast employment region.

Source: Employment.

1.7 The jobactive program consists of five services, outlined in Table 1.2.

Table 1.2: The five services of the jobactive program

jobactive employment services – forecast average annual expenditure of $1.31 billiona

Delivered by 44 employment service providers to provide practical support to job-seekers to assist them in preparing for, securing and sustaining employment. Providers are required to maintain high levels of job-seeker and employer engagement, and facilitate and monitor Work for the Dole activities.

New Enterprise Incentive Scheme (NEIS) – forecast average annual expenditure of $115 million

A range of services to assist eligible job-seekers to establish and run their own small businesses, up to 7642 businesses were supported in 2016–17. The 21 NEIS providers deliver a range of services including, accredited small business training, business mentoring and advice for up to 52 weeks.

Work for the Dole Coordinators – forecast average annual expenditure of $34 million

Nineteen Coordinators, one per employment region, source suitable Work for the Dole places for job-seekers. Coordinators are responsible for sourcing places, liaising with hosts to secure places and connecting hosts with employment service providers.b Coordinators are not responsible for managing the Work for the Dole activity or job-seekers while they are in the activity; that is the role of employment service providers.

Harvest Labour Services (HLS) – forecast average annual expenditure of $3.39 million

Five HLS providers help supply supplementary out of area workers to agricultural growers in 11 harvest areas (these do not align to the employment regions).

The HLS is the only jobactive service for which providers are paid to place people who are not job-seekers on income support into employment (in April 2016, 92.9 per cent of placements were for people not receiving income support).

National Harvest Labour Information Service (NHLIS) – forecast average annual expenditure of $1.24 million

One provider coordinates and disseminates information regarding harvest related work, including areas not serviced by the HLS, via an electronic National Harvest Guide and a national telephone information service.

Note a: Department’s forecast funding for all five services for 2016–17, as at 31 December 2016.

Note b: Work for the Dole Coordinators started on 1 May 2015 to source places prior to the commencement of jobactive on 1 July 2015. In April 2017, the Government agreed to terminate Work for the Dole Coordinator contracts by 1 January 2018. Instead, employment service providers will source Work for the Dole placements and liaise with host organisations.

Source: ANAO analysis of departmental documentation.

1.8 Throughout this report, the term ‘provider’ refers to the 65 contracted providers in the five services of the jobactive program and the term ‘employment service provider’ refers to the 44 providers delivering jobactive employment services.

1.9 Employment is forecast to spend approximately $7.3 billion over the contracted five year period of the jobactive program. In 2015–16, expenditure was $1.02 billion, which was $157.8 million (13 per cent) below the total budget of $1.18 billion (as shown in Figure 1.3). The underspend was due, in part, to lower than expected outcome payments ($63.6 million lower than forecast) and Work for the Dole expenditure ($47.5 million lower than forecast). This resulted in a decreased budget forecast in 2016–17.

Figure 1.3: The jobactive program budgeted vs. actual expenditure, as at 31 March 2017

Source: ANAO analysis of departmental documentation and Budget papers.

Delivery of employment services

1.10 One of the objectives of jobactive is to ‘promote stronger workforce participation by people of working age and help more job-seekers move from welfare to work’.

1.11 Under the program, assistance is provided to job-seekers by an employment service provider through one of three streams (Stream A, B, or C).7 Stream A job-seekers (including volunteers) account for 43.8 per cent of the overall caseload and are considered to be the most competitive and ready for work. Those job-seekers in Stream B (38.8 per cent) have some assessed barriers that will need their employment service provider to play a greater role to become job ready, such as language barriers. The most disadvantaged job-seekers, those with multiple barriers to work such as drug and alcohol addiction, are assisted through Stream C (17.1 per cent).

1.12 Employment has sought to incentivise employment service providers to deliver high quality services and achieve sustained employment outcomes for job-seekers who need the most support to find and keep a job. The employment service providers receive higher outcome payments, at four, 12 and 26 weeks for the successful transition of regional long-term unemployed Stream C job-seekers off income support.8

Audit approach

1.13 The audit objective was to assess whether the Department of Employment effectively designed and monitors the progress of the jobactive program.

1.14 To form a conclusion against the audit objective, the Australian National Audit Office (ANAO) adopted the following high-level criteria:

  • the jobactive program was designed to support the achievement of the Government’s policy objectives; and
  • the department effectively monitors the progress of the jobactive program against the Government’s policy objectives.

1.15 The audit examined all five services of the jobactive program, but primarily focused on the jobactive employment services. The audit did not assess the department’s tender process conducted in 2014–15; review in-depth the transition process for implementing the jobactive program; or analyse the Department of Human Services’ payment of welfare benefits to job-seekers.

1.16 In undertaking the audit, the ANAO analysed departmental records and data, and interviewed key managers and personnel in Employment’s National and State Offices. The ANAO invited the three peak bodies and 65 providers to contribute to the audit. Two peak bodies and 23 of the 65 providers (35.4 per cent) provided a contribution, this included:

  • 16 of the 44 jobactive employment service providers (36.4 per cent);
  • eight of the 19 Work for the Dole Coordinators (42.1 per cent);
  • seven of the 21 New Enterprise Incentive Scheme providers (33.3 per cent); and
  • three of the five Harvest Labour Service providers (60 per cent).

1.17 The audit was conducted in accordance with the ANAO auditing standards at a cost to the ANAO of approximately $423 000.

1.18 The team members for this audit were Meegan Reinhard, Anne Kent, Veronica Clement-Jones and Deborah Jackson.

2. Program design

Areas examined

This chapter examines the reason for establishing the jobactive program, and the arrangements that the Department of Employment (Employment or the department) introduced to manage the design process, including: governance and oversight arrangements; stakeholder consultations; advice to government; and refinements to the program’s design over time.

Conclusion

The department effectively managed the design process for jobactive. There was a sound reason for redesigning the employment services model, the governance arrangements established by the department were comprehensive, stakeholders were adequately consulted and the Minister was briefed on a range of design and implementation topics. Since the implementation of jobactive on 1 July 2015, the department has reviewed and amended elements of the program’s design.

Was there a clear reason for establishing jobactive?

Employment identified significant weaknesses in the former Job Services Australia model, which was the key reason for developing the new employment services program, jobactive. These weaknesses were appropriately considered when designing the jobactive program.

2.1 In June 2014, the government agreed that a reform of employment services was needed to address the identified weaknesses of the Job Services Australia model, and to introduce the new Government’s election commitments. The weaknesses were identified through a series of discussion papers and consultation, which are summarised below.

2.2 In June 2011, the Minister for Employment Participation established an Advisory Panel on Employment Services Administration and Accountability (the Advisory Panel) to provide advice to Employment on future employment service programs, opportunities for streamlining administration and industry-supported solutions to reduce unnecessary administration. The Advisory Panel’s final report, which was released in September 2012, made 13 recommendations.9 The government’s response to the final report noted that:

The Advisory Panel found that, against the backdrop of high public expectations of fraud prevention and value for money, the employment services model is complex in design and administration, and that there was a level of ‘hyper specificity’ in administrative requirements, a risk-adverse culture and a practice of risk mitigation in order to meet those expectations.10

2.3 In December 2012, the Minister released a discussion paper, Employment Services—Building on success, to seek public feedback on areas of the employment services model that could be improved. The resulting feedback highlighted weaknesses in the system, including that: training should be linked to a job or placement, not just for its own sake; too many job-seekers with a disability were being placed into Job Services Australia and not Disability Employment Services; post placement support was often poor; and there were thousands of youth under 21 years of age who were not engaged in work or education that needed to be connected to government services.

2.4 In June 2013, the Government agreed that broad consultation with the community and stakeholders would be conducted in order to develop a new employment services model to operate from mid-2015. The following opportunities for improvement had been identified through various consultative forums (discussed further in paragraphs 2.12 and 2.13):

  • improving collaboration between employment service providers and other services (for example, housing and migrant services) would better leverage government investment in other social services to address non-vocational barriers to employment;
  • better collaboration between providers would improve the sharing of best practice;
  • increasing the awareness and use of employment services by employers of all sizes;
  • initial assessment processes that better identify job-seeker needs and vocational and non-vocational barriers;
  • addressing issues of training investment, which was poorly linked to local labour markets;
  • the need for better tailoring of services to job opportunities and job-seeker needs; and
  • addressing the concern that providers were investing significant, unnecessary resources in system compliance and reporting. It was noted that while transparency and accountability must not be compromised, the reduction in red tape needed to continue building on work underway to respond to the 2012 Advisory Panel’s report.

2.5 In September 2013, prior to the Federal Election, the Coalition released its Policy to Increase Employment Participation and Policy to Create Jobs by Boosting Productivity which included the following election commitments:

  • a renewed commitment to reinvigorating the Work for the Dole program, so that unemployed people on income support are active, engaging in mutual obligation work activities and building skills to ensure they are work ready;
  • introducing payments to long-term unemployed job-seekers who find and retain employment (the ‘job commitment bonus’) or who are willing to relocate for employment; and
  • reducing the regulatory burden for individuals, businesses and community organisations by establishing and meeting a red and green tape reduction target of at least $1 billion a year.

2.6 Employment conducted two evaluations of Job Services Australia—in October 2014 (for the first contract from 2009–12) and July 2016 (for the second contract from 2012–15). The evaluations aligned to the findings of the earlier consultations and research (as discussed above). The evaluations found that:

  • the two contracts achieved higher education outcomes than the preceding Job Network contract, but progressively lower employment outcomes; and
  • the estimate of red tape expenditure declined by nearly 20 per cent between the two contracts (from $321.9 million to $259.3 million per annum), but this was still significant and equated to 20.9 per cent of program funding and was largely borne by providers (84.5 per cent).

Was an appropriate governance framework established for the design and implementation of jobactive?

The governance arrangements established by the department for the design and implementation of the jobactive program were comprehensive, with strong senior executive engagement and support from the 16 working groups that had responsibility for developing the policy of specific subject areas.

2.7 The governance arrangements that Employment implemented for jobactive (at the time of design, the program was referred to as the Employment Services 2015 program) are shown in Figure 2.1.

Figure 2.1: Governance arrangements for the design of jobactive

Note: The arrows represent reporting arrangements.

Source: ANAO representation of departmental documentation.

2.8 The Employment Services 2015 (ES2015) Committee was the governing body responsible for providing strategic direction on the development and implementation of the 2015 Employment Services program. The ES2015 Committee was established as the central departmental decision-making body on policy development, and all aspects of implementation including project planning, risk management, issues management and overseeing stakeholder management.

2.9 The ES2015 Committee met regularly and received updates and made timely decisions on the topics raised (covering both policy development and implementation concerns), as recorded in its Decision Register and Action Items Register. The ES2015 Committee implemented the jobactive program on 1 July 2015. The ES2015 Committee effectively managed the oversight and design of the jobactive program. The ES2015 Committee was dissolved in October 2015 three months after the implementation of jobactive.

2.10 Sixteen working groups, each led by a Branch Manager, were established by the ES2015 Committee to support the implementation of the jobactive program through the development of specific policy and service delivery arrangements. Each of the working groups was responsible for developing and regularly reporting on project management plans, which included key tasks, risk plans, stakeholder identification, change management strategies, business process mapping and IT requirements.

2.11 The ANAO’s analysis of the closing reports for the 16 working groups found that the reports listed achievements against key deliverables and milestones, outstanding deliverables and/or ongoing functions that will be transferred to identified branches, outstanding risks and lessons learnt. The working groups achieved the majority of key deliverables. There were some minor policy decisions that remained unresolved at the time of closure (for example, the number of business days allowed to commence a job-seeker in an activity during the Work for the Dole phase) and some deliverables required technical solutions so they could be implemented (for example, errors with the job-seeker appointment booking system). There were several recurring lessons learnt across the various working groups, including: membership of the working groups and ownership of the issues; allowing appropriate lead times for internal clearances and testing; and the challenges of developing processes when the policy was not settled. Employment has assigned responsibility for building on the lessons learnt to the Employment Steering Committee (discussed in Chapter 3).

Were relevant stakeholders adequately consulted on the design of the jobactive program?

During the design of jobactive, stakeholders were consulted and Employment considered their views when designing the program.

2.12 During the initial design of the revised employment services program, there were multiple opportunities for stakeholders to provide feedback on the model, including:

  • Advisory Panel on Employment Services Administration and Accountability (the ‘Advisory Panel’)—in late 2011 the Advisory Panel consulted with representatives from large and small Job Services Australia providers, peak bodies, and government organisations in Australia and the United Kingdom, as well as job-seekers, on opportunities for streamlining administration and industry-supported solutions to reduce unnecessary administration. Twenty-nine submissions were received in response to the Advisory Panel’s discussion paper, with further consultations held with stakeholders in early 2012 before the panel’s final report was released in September 2012.
  • Employment Services—Building on Success, Issues Paper—following the release of the paper in December 2012, 37 consultation sessions were held across the country in early 2013, with 440 attendees representing around 300 organisations and businesses. The sessions were targeted at particular stakeholder groups including: employers; academics; job-seekers; and service providers and community organisations. In March 2013, a Key Stakeholder Forum was attended by approximately 60 peak body representatives, and 180 written submissions were also received in response to the paper.

2.13 While the public consultations in 2013 were not explicitly referenced in the June 2014 proposal to government, feedback that had been obtained by the department was incorporated in the earlier versions of the model and was aligned to the final design.

2.14 In August 2014, Employment also sought public feedback on the exposure draft to the Request for Tender, with amendments to the design made as a result of the 60 submissions received from potential providers, employers, peak bodies, and community sector representatives.

Was sound advice provided to the Government on design options and recommendations?

Employment provided sound advice to the Minister for Employment and the Assistant Minister for Employment on a range of design considerations and associated risks.

2.15 The ANAO reviewed the 141 briefings and correspondence from the department to the relevant Ministers for Employment, covering the period from February 2012 to December 2016. The department provided advice to ministers on key matters, including:

  • the Work for the Dole program;
  • proposed changes to the New Enterprise Incentive Scheme and Harvest Labour Services;
  • the performance framework for employment service providers, including Star Ratings and Indigenous Outcome Targets;
  • the reduction in the number of employment regions from Job Services Australia;
  • the redevelopment of the Quality Assurance Framework;
  • finalised evaluations of the Employment Pathway Fund; and
  • the publication of the evaluation strategy for jobactive.

2.16 The briefs outlined the benefits and risks for each option, and included a recommendation indicating the department’s preferred option. From October 2014 to December 2014, Employment provided monthly implementation status reports to the Minister, and fortnightly reporting from January to September 2015. Reports included a timeline for key activities and milestones, and the mitigation and monitoring strategies for identified high-level risks and sensitivities.

Has the program design been reviewed and adjustments made where appropriate?

In consultation with providers and peak bodies, Employment has made several changes to the design of the jobactive program. The department established working groups or external reviews to guide improvements to the program.

2.17 Employment has made several amendments to the jobactive program since the program commenced on 1 July 2015. For example, in October 2015 following feedback from providers and peak bodies, Employment suggested twelve refinements to the Minister, which were agreed. These amendments included that the department:

  • allow employment service providers to submit supplementary payslip information from employers if the payslips do not meet minimum requirements;
  • consider pre-existing employment arrangements for employment outcomes;
  • consider renegotiating sites with a low number of job-seekers on a case-by-case basis and where there are exceptional circumstances; and
  • consider options for using the Employment Fund for non-accredited training and capping the expenditure per job-seeker or per course.11

2.18 Employment has also made changes to the design of two specific components of jobactive—Work for the Dole12 and Harvest Labour Services.

Changes to Work for the Dole

2.19 In August 2015, Employment established a Work for the Dole Working Group with representatives from employment service providers, Work for the Dole Coordinators and peak bodies. The Working Group identified 12 issues with the new Work for the Dole model, including: the administrative impost on Lead Providers13, and the financial risks of upfront payments required for job-seekers on the caseload of a different employment service provider; insufficient time to commence a job-seeker in a Work for the Dole place; and poor collaboration between employment service providers and Work for the Dole Coordinators.

2.20 In response to the Working Group findings, Employment made 11 recommendations to the Minister for amendments to the model, which were agreed. The subsequent changes to the Work for the Dole model included:

  • allowing Lead Providers to reserve places in the system, before advertising them as available to other employment service providers;
  • introducing a ‘Lead Provider Payment’ of up to $100 per place to recognise the role the Lead Provider undertakes when managing job-seekers from non-lead providers in Work for the Dole activities14;
  • extending the timeframe for when job-seekers need to commence in an activity (from five days to 10 days); and
  • expanding the Employment Fund guidelines to allow providers to pay for police checks for its job-seekers.

Changes to Harvest Labour Services

2.21 In April 2016, the Employment Steering Committee agreed that a review of Harvest Labour Services would be undertaken.15 The scope of the Review included:

  • the exclusion of harvest placement fees to Labour Hire Companies;
  • the definition of ‘harvest work’, as planting and non-horticultural crops (for example, forestry) were excluded under the Deed; and
  • the ongoing viability of the Harvest Labour Service given the changes in the industry since the program was introduced in 1998.

2.22 The Review was finalised in June 2016 and made 11 recommendations. The Review concluded that the while the program is small (within the context of the jobactive program, and when compared to the overall number of seasonal workers required to support the agricultural sector) it is highly regarded by stakeholders and valued by the sector. The Review suggested that (subject to any Commonwealth Procurement Guidelines issues being satisfied) the Harvest Labour Service be amended so that providers would be able to claim placement fees: for placing workers with Labour Hire Companies; and, for a broader range of work including plant and animal cultivation, fishing and pearling, and tree farming and felling.16

2.23 The jobactive Deed was varied in April 2017 to expand the scope of eligible employers to include Labour Hire Companies and Contractors. Employment has not addressed the other suggestions made in the review.

3. Governance arrangements for ongoing management

Areas examined

This chapter examines the Department of Employment’s (Employment’s or the department’s) arrangements for the ongoing management of the jobactive program, including governance and oversight arrangements, the management of risk, and the guidance available to the providers of the five services.

Conclusion

Employment has established a suitable committee structure to oversee the jobactive program and the department has identified and managed risks at the program and provider level. The prioritisation of activities to deliver the program could be improved to ensure that required activities are completed. The principles-based guidelines do not always clearly articulate Employment’s expectations of providers.

Area for improvement

The ANAO made one recommendation aimed at improving the risk-based prioritisation of activities to deliver the jobactive program.

Have appropriate oversight arrangements for ongoing management of jobactive been established?

Employment has established an appropriate governance structure to provide oversight and make decisions about the ongoing management of jobactive. Improved prioritisation of the activities necessary for effective delivery of the jobactive program is needed to ensure that objectives are being achieved.

3.1 The committee arrangements that Employment implemented for the ongoing management of jobactive are shown in Figure 3.1. The three committees which oversee jobactive are the: Employment Steering Committee (ESC); Employment Branch Managers Committee (EBMC); and Program Integrity Sub-Committee for Employment Services (PISCES). Meeting minutes and papers are shared between the committees17, and Employment has documented the decision-making responsibilities for the various committees in each terms of reference. An ESC and EBMC Decisions Register is located on the department’s intranet and tracks issues as they progress through to the decision-making stage.

Figure 3.1: Committee arrangements for the ongoing management of jobactive

Source: ANAO representation of departmental documentation.

3.2 Various working groups consider issues and develop proposals to address identified issues for referral to PISCES, EBMC or ESC, but they do not have decision-making responsibility. The issue can be a long standing topic or an emerging issue. Members include Employment staff, or a combination of Employment and external stakeholders depending on the issue being considered. For example, the Performance Framework Working Group was established to resolve ongoing contract management, compliance and performance issues. It includes representatives from Employment, the peak bodies, employment service providers, and Work for the Dole Coordinators.

3.3 Fortnightly reports are provided to the Secretary and the Minister, including information on jobactive caseload, performance outcomes, jobactive and Employment Fund budget and expenditure, and Work for the Dole progress. The Department’s Audit Committee receives a quarterly update on departmental risks and treatments, and fraud and serious non-compliance investigations.

Employment’s prioritisation of program delivery activities

3.4 The jobactive program is administered by the Employment Cluster within the department, as shown in Figure 3.2.

Figure 3.2: Organisational arrangements for the Employment Cluster

Source: ANAO representation of Employment’s organisational chart, current as of 10 April 2017.

3.5 The program areas (located in Employment’s National Office)18 and the Delivery and Engagement Group (based around Australia)19 work together on policy, compliance and service delivery. ESC and the National Leadership Team oversee and manage the delivery of these services.20 As at 31 March 2017, resourcing of the Delivery and Engagement Group included 270 full-time equivalent staff, with 195 of those staff responsible for tasks related to jobactive.21

3.6 The Delivery and Engagement Group undertakes a range of functions, including business-as-usual, program specific and ad-hoc activities. The business-as-usual activities include regular engagement with providers, and assessing the delivery of provider service commitments. Program activities may be identified by the program areas as requiring action, and include provider performance reviews and discussions and program assurance activities. Ad-hoc activities may include procurement exercises, program evaluations, conferences, webinars, and learning and development activities.

3.7 National Office informs the Delivery and Engagement Group of required tasks by:

  • a Delivery and Engagement Group Calendar, updated quarterly, which details planned program specific and ad-hoc activities;
  • a teleconference held monthly, where program areas can provide information directly to the Delivery and Engagement Group about ‘hot issues’;
  • an email distributed weekly by National Office to communicate outstanding and upcoming tasks and communicate priorities;
  • joint representation on relevant working groups, committees, meetings, and teleconferences to share information; and
  • a shared central mailbox to facilitate the sharing of information.

3.8 Delivery and Engagement Group staff record the outcomes of many of their contract management activities in an information technology system—Provider 360.22 The information recorded in the system is available to National Office staff. According to Provider 360, in 2015–16 the required number of sites visits were not conducted (discussed in paragraph 4.10), and only 2183 of the 7593 service commitments were assessed (28.7 per cent).

3.9 In May 2016, Employment identified deficiencies in the processes for planning and prioritising work for the Delivery and Engagement Group, and agreed that business processes needed to be improved to ensure that activities were identified, resourced and managed in a consistent, timely and transparent manner. The Delivery and Engagement Group Calendar was identified as the primary tool for allocating tasks to the Delivery and Engagement Group and a revised format was released in March 2017, along with a new protocol aiming to support the consistent and timely identification, management and prioritisation of competing activities. However, the updated Calendar does not include business-as-usual activities, an assessment of the resources expected to be allocated to each task, or a risk-based prioritisation of the tasks to identify which activities may be deferred if necessary. In addition, the status of activities, including completed tasks, is not reported to the department’s executive in a timely manner.

Recommendation no.1

3.10 The Department of Employment should implement a risk-based approach to prioritising the activities required to effectively manage and monitor the delivery of the jobactive program.

Entity response: Agreed.

3.11 Risk and significance are key determinants for the department in prioritising activities required to effectively manage and monitor the delivery of the jobactive program. Noting the recent establishment of the Delivery and Engagement Group calendar and supporting protocol, the department is monitoring its effectiveness and will consider what additional information would be of value to add.

Is program risk appropriately managed?

Employment has established an appropriate framework to manage risks to the jobactive program. While the management of high-level program risks has been generally sound, there is scope for the department to improve the timeliness of monitoring and mitigation actions for identified provider risks. Further, the department should make greater use of compliance and performance data to identify and manage provider risks.

3.12 Employment has an established framework to guide its approach to risk management, with internal guidance and tools made available to assist staff.23 Employment manages the risk to the jobactive program through the jobactive, program area and provider risk plans.

Program risk management

3.13 The jobactive Risk Plan identifies twelve high-level risks, and 38 associated treatments for the jobactive program. The twelve risks consist of seven which are rated as ‘Medium’, and five which have been rated as ‘Low’. The medium-rated risks relate to:

  • fraud by providers;
  • injury resulting from providers not adhering to Work Health and Safety principles;
  • privacy breaches leading to reputational damage;
  • poor performance or service quality leading to reputational damage;
  • failing to respond to changed economic or policy environment in a timely manner;
  • inappropriate impact on job-seeker income support payments; and
  • financial viability of providers impacting on service delivery.
  • Other jobactive-program risk plans include the Performance Framework Risk Plan and the Payment Integrity Risk Plan.24

3.14 All ten program areas related to jobactive within the Employment cluster has a risk plan. The plans identify the risks relevant to that program area and the treatments established to mitigate or manage identified risks. The risks include program risks and implementation or delivery risks to the department. The ANAO reviewed the risk plans for the ten program areas, and found 127 unique identified risks and 364 associated treatments. The twelve risks identified in the jobactive risk plan are reflected in program area risk plans.25 All jobactive program and program area risk plans assign owners for each treatment and include a review date. Of the 608 treatments identified in the 13 risk plans, 590 had been reviewed by the due date. Eighteen (less than 3 per cent) were overdue for review, and none by more than a month.

Provider risk management

3.15 Employment manages the risks associated with outsourcing the delivery of the jobactive program with organisation level risk plans created and maintained for each provider. Where Employment considers that a risk would be better managed at the region or site level, risk plans can be created at that level.26 Providers are assessed against four standard risk categories (fraud, financial viability, governance and compliance) and two variable risk categories (service delivery and performance, and relationships).

3.16 The risk appetite for the department has been set at medium, with a preference to not accept extreme or high risk levels. Employment manages provider risks through risk actions in the risk plan.27 Since May 2016, risk plan ratings have been reported to PISCES, with reports including the date of the last update to the risk plan, overall organisational risk rating, and ratings against each of the categories. A number of overdue risk plans have been repeatedly reported to PISCES without the required review undertaken.28 For example, a provider risk plan with an extreme overall risk rating was only reviewed twice between September 2015 and November 2017, despite the department setting a monthly review period for extreme risks, and the overdue risk plan being reported monthly to PISCES.

3.17 Employment has developed 64 organisation-level risk plans for the 65 providers of the jobactive program.29 The ANAO’s analysis of the 64 risk plans found:

  • 28 (43.7 per cent) of the 64 plans had an overall rating of extreme or high;
  • 15 (53.6 per cent) of the 28 plans with an overall rating of extreme or high were due to perceived financial viability risk. Other high risk categories were service delivery and performance (35.7 per cent), compliance (28.6 per cent), governance (21.4 per cent), and fraud (3.6 per cent);
  • two (7.1 per cent) of the 28 plans with an extreme or high overall rating had no open risk actions recorded, contrary to the department’s risk appetite30; and
  • for approximately one-third of all the risk actions it is unclear how the actions will manage or mitigate the identified risk.31
Compliance and performance data in provider risk plans

3.18 Employment generates various compliance and performance data, including the Rolling Random Sample and Compliance Indicator (discussed further in Chapter 4) and the Star Ratings performance measure (discussed further in Chapter 5). In the risk plans for the 44 employment service providers there is some recording of compliance and performance data, with:

  • 21 (47.7 per cent) of the 44 plans mentioning the Rolling Random Sample—with 11 (52.4 per cent) of these including data;
  • 14 (31.8 per cent) of the 44 plans mentioning the Compliance Indicator—with 11 (78.6 per cent) of these including data; and
  • 31 (70.5 per cent) of the 44 plans mentioning the Star Ratings—with 28 (90.3 per cent) of these including data.

3.19 The department should make greater use of compliance and performance data to identify and manage provider risks.

Do the guidelines clearly articulate Employment’s expectations of providers?

Employment has established a principles-based approach for the jobactive guidelines to allow providers to deliver flexible solutions tailored to an individual job-seeker’s circumstances. On occasion, this has led to some inconsistency in the interpretation and application of the guidelines.

3.20 Guidance to providers in jobactive is principles-based. Employment advised that it changed from the prescriptive approach used in Job Services Australia to improve the servicing of job-seekers, with the intention that providers would be able to deliver flexible solutions based on a job-seeker’s circumstances.

3.21 When re-designing the guidelines, Employment advised the Minister that the Job Services Australia guidelines had been long and required providers to cross-reference to various sources. The department further advised the Minister that the new jobactive guidelines had been designed so that all relevant program information was consolidated.32 The ANAO’s review of the department’s guidance found that, in addition to the two jobactive Deeds, there are 31 guidelines that providers of the jobactive program need to adhere to, as well as additional information and factsheets. The guidelines cover broad topics, such as the Documentary Evidence Guideline, or more specific matters, such as Relocation Assistance to Take Up a Job Guideline. Each guideline contains a disclaimer which states:

This Guideline is not a stand-alone document and does not contain the entirety of Employment Service Providers’ obligations. It must be read in conjunction with the Deed and any relevant Guidelines or reference material issued by the Department of Employment under or in connection with the Deed.

3.22 The ANAO’s analysis of the guidelines found that multiple guidelines may need to be reviewed to answer a single question and, in some areas, the guidelines are unclear and/or inconsistent. For example, the requirement for New Enterprise Incentive Scheme (NEIS) participants to complete mutual obligation requirements (primarily monthly job searches). Relevant guidance about mutual obligations is provided as follows:

  • NEIS Training Guideline—states that participants do not have mutual obligation requirements while in NEIS training;
  • Mutual Obligation Requirements and Job Plan Guideline—states that participants will not generally have mutual obligation requirements for the period they are participating in NEIS, and separately states that providers must not enforce mutual obligations while the job-seeker is undertaking NEIS training or the NEIS program; and
  • Activity Management Guideline—advises that NEIS training is to be considered an activity like Work for the Dole (which does have mutual obligation requirements), but does not specifically mention mutual obligation requirements.33

3.23 Three of the four NEIS-only providers the ANAO interviewed reported that some employment service providers applied the guidelines inconsistently and required NEIS participants to complete their mutual obligation requirements while in the program.

3.24 Providers seeking further guidance from the department about the guidelines are directed to use Question Manager34, so that answers are visible to all and the department can gain insight into which areas the providers are requesting clarification. Employment considers that the centralisation of responses to guideline queries should also improve the consistency of department messaging.

3.25 As at 15 February 2017, there were 833 queries in Question Manager that related to the jobactive program.35 The ANAO reviewed 328 of the queries (39.4 per cent)36 and analysed 231 (27.7 per cent of the total)37, to assess which topics the providers were requesting clarity on, and the responses the department provided. The ANAO’s analysis found:

  • recurring query topics related to definitions of self-employment and permissible breaks, the feasibility of combining income from multiple jobs to achieve an outcome, clarifying if the provider could employ the job-seeker, and how to account for new, pre-existing or recurring employment with the same employer;
  • the majority of queries were escalated and answered by National Office (59.3 per cent), with 32.5 per cent answered by the Delivery and Engagement Group and 8.2 per cent answered by the State Office; and
  • in 26.0 per cent of the responses (60 of the 231) the department quoted the guideline and did not provide additional clarification.

3.26 Employment has not analysed the data contained within Question Manager since December 2015, despite noting in a paper to ESC that using the business intelligence within Question Manager would enable the Employment Cluster to consider: topics that require further policy clarification; the effectiveness of guidance material; workload management and planning; and identify training and education opportunities for departmental and provider staff.38

Rolling Random Sample

3.27 The Rolling Random Sample is a compliance activity that Employment uses to gain assurance that the claims submitted by providers are compliant with the requirements of the Deed(s) and guidelines (the Rolling Random Sample is discussed further in Chapter 4). As the jobactive guidelines are principles-based, it is important that the assessment undertaken for the sampled claims does not impose prescriptive elements that are not included in the guidelines. The ANAO’s review of the assessment guidance used by Employment staff, found that it was consistent with the Deed(s) and program guidelines.

3.28 As noted in paragraph 3.25, the ANAO found several examples where providers sought clarification on a guideline and Employment’s response was to quote the guideline, often caveating its response with ‘for any claims lodged, a jobactive provider must ensure that they have met all requirements outlined in the jobactive Deed and jobactive guidelines’, or similar. The employment service providers consulted by the ANAO were concerned that if the department interpreted the principles-based guidelines in a different manner than the providers when they submitted their claims for payment, this may result in poor compliance results in the Rolling Random Sample. For example:

  • in line with advice provided through Question Manager, in July 2015 a provider claimed $231 related to pre-employment medical costs;
  • as a result of the first quarter of the Rolling Random Sample, the claim was incorrectly recovered by the department on the basis that the claim related to ineligible medical costs; and
  • in April 2017, Employment advised the ANAO that this recovery should not have occurred as the claim was legitimate, and the incorrect recovery has been rectified.

Review of the guidelines

3.29 Since July 2015, Employment has released updated versions of the jobactive program guidelines on a quarterly basis (when deemed necessary to reflect changed or clarified policies).

May 2016 guideline review

3.30 In May 2016, Employment held a Stakeholder Working Group to review and streamline the jobactive guidelines, with an aim to reduce administration and drive practice improvement while ensuring that the basic requirements and principles were effectively communicated to the providers’ front-line staff.

3.31 The working group had three sub-working groups. Providers were represented on the sub-working groups, allowing them the opportunity to give feedback about the guidelines and input into the review. The sub-working groups made 79 suggestions for improvement, including formatting proposals, subjects that required further clarification, removing outdated Job Services Australia related documents from the Provider Portal39, and moving some information from the guidelines into the Deed(s). The Stakeholder Working Group agreed to consider 69 of the suggested changes (87 per cent).

3.32 As a result of the guideline review, Employment engaged an external consultant to redesign the guideline template and, in March 2017, began transitioning the guidelines to the new format over a number of quarterly publishing schedules. Employment has advised that the transition to the new format should be completed by late October 2017.

4. Program assurance of providers

Areas examined

This chapter examines the Department of Employment’s (Employment’s or the department’s) arrangements to manage provider compliance with contract requirements, with a focus on the activities for employment service providers.

Conclusion

Employment has obtained a reasonable level of assurance that the jobactive program is being administered as designed and expected. The Assurance Strategy for the jobactive 2015–2020 contract includes new program assurance elements that have strengthened the department’s monitoring of employment service provider’s compliance with contractual obligations. While the department is currently reviewing the operation of the Assurance Strategy, the review does not address some of the key elements of the program, including whether the strategy reflects the department’s preferred level of compliance.

Area for improvement

The ANAO made one recommendation addressing whether the compliance regime is structured to effectively and efficiently detect and manage non-compliance.

Has a provider compliance framework been established?

Employment has established a comprehensive framework to manage provider compliance that incorporates approaches to prevent, deter, detect and correct non-compliance.

4.1 Employment has established arrangements to provide assurance that the contracted services are effective and of suitable quality. Contractual arrangements are set out in the: Deed(s) (the jobactive Deed 2015–20 and the jobactive Work for the Dole Coordinator Deed 2015–20) and associated guidelines; Service Delivery Plans for each employment service provider40; Service Guarantees41; and the Joint Charter of Deed Management between Employment and the providers.42 The obligations for employment service providers broadly focus on:

  • the provision of employment services to job-seekers and employers;
  • meeting standards of service and behaviour;
  • claiming payments according to the Deed(s) and guidelines; and
  • the Service Delivery Plans submitted by each provider during the tender process.

4.2 Compliance with the terms of the Deed(s) is managed through the Assurance Strategy, as outlined in Figure 4.1. The Assurance Strategy is Employment’s framework for managing contractual compliance and aims to: encourage providers to voluntarily comply with requirements; describe the remedial actions that apply where non-compliance is identified; and state the target areas of risk. Employment employs four strategies to provide assurance that employment service providers are compliant with the terms of the Deed(s):

  • Prevention—making it easier for providers to comply;
  • Deterrence—making clear the risks and penalties of non-compliance;
  • Detection—processes are in place to identify non-compliance; and
  • Correction—acting on detected non-compliance.

Figure 4.1: Employment’s Assurance Strategy

Note: The Transition to Work and ParentsNext programs are outside the scope of this audit.

Source: Employment documentation.

Is employment service providers’ compliance with obligations effectively monitored?

Employment has a framework to monitor employment service providers’ compliance with contractual obligations. The framework enables the department to obtain a reasonable level of assurance about employment service providers’ compliance. There is scope to improve the analysis of complaints data, the provision of Rolling Random Sample results to providers in a timely manner, and the effective prioritisation of targeted assurance activities.

4.3 The key activities of the Assurance Strategy that Employment uses to monitor employment service provider compliance with the terms of the Deed include activities to monitor service provision and quality such as: Quality Assurance Framework certification; contract management activities including site visits; and tip-offs and complaints. Assurance activities to monitor the appropriateness of claims include: Rolling Random Sample reviews of key program elements; targeted program assurance projects of employment services; and data analytics to identify emerging risks. Each of these components is discussed below, and Employment’s management of identified non-compliance is discussed later in this chapter.

Quality Assurance Framework

4.4 The Quality Assurance Framework (QAF) was introduced following completion of a pilot program in 2014. The QAF was established in response to feedback from employment service providers and peak bodies that the quality framework used under Job Services Australia to assess employment service provider performance did not provide sufficient practical information to measure continuous improvement in service quality. The QAF aims to provide assurance that the services delivered by employment service providers are to an agreed standard, while encouraging continuous improvement.

4.5 Participation in the QAF process is mandatory for the 44 employment service providers, and involves a certification audit, a surveillance audit every two years and extraordinary audits at the department’s discretion. To obtain and maintain certification under the QAF, employment service providers are required to:

  • be assessed by an independent auditor against a quality standard deemed acceptable by Employment43; and
  • maintain adherence to seven overarching quality principles that are consistent with the QAF and program objectives, for the duration of the contract.44

4.6 The deadline for QAF certification was 1 July 2016. By the deadline, 41 of the 44 employment service providers were certified. Of the three that did not achieve certification by the deadline, one was certified in July 2016, one in September 2016, and one is not certified as at April 2017.45

4.7 Employment’s implementation of the QAF included one evaluation and two reviews of specific aspects of the QAF.46 The evaluation and reviews had similar findings, including:

  • the QAF has the potential to drive continuous improvement in the longer term for delivering services to job-seekers and, to a lesser extent, employers but is currently unable to measure the impact of the QAF; and
  • the time and costs associated with preparing for and conducting the certification audits was greater than expected.47

4.8 In December 2016, ESC endorsed streamlining and scope changes to the QAF process. The new approach is intended to be in place for the 20 re-certification audits and 24 surveillance audits scheduled from June 2017. The changes aim to address the concerns raised in the reviews and include: developing support materials for staff and providers; aligning similar practice requirements with the quality standards; streamlining and removing duplicated requirements; and clarifying terminology, including developing a glossary of terms.

Contract management activity

4.9 The primary purpose of contract management activities is to monitor and assess performance against the Service Delivery Plans and the Service Guarantees.48 Contract management activities include: provider site visits; review of providers’ self-assessments of performance required under the Deed (discussed in Chapter 5); and monitoring the delivery of employment services by the provider. Results of these activities assist to inform ongoing and targeted program assurance activity.

4.10 In October 2015, the Program Integrity Sub-Committee for Employment Services (PISCES) agreed that routine announced visits would be conducted in at least one site in each region where an employment service provider delivers services each financial year.49 With 44 employment service providers operating in between one and 29 employment regions each, this equates to a minimum of 202 visits each year. Employment reported to PISCES in June 2016 that ‘Provider site visit numbers are below expected benchmark due to competing priorities in the State Network. Site visit targets will be reviewed for next financial year’. The ANAO’s analysis of the number and location of site visits found that visits were conducted in 200 employment service provider regions during 2015–16. By 20 April 2017, Employment had completed 112 of the 202 targeted visits for 2016–17 (55.4 per cent).

4.11 Information from contract management activities are recorded in several systems including Provider 360 and/or TRIM. The ANAO’s analysis of the information gathered during routine site visits found gaps in the information recorded in Provider 360, with 12 of the 44 employment service provider files in Provider 360 not having a record of the required information. Intelligence and issues are also recorded in the Program Assurance Tool, the Rolling Random Sample assessment tool, and shared drives; however these systems are not accessible to all staff. With information not available or easily accessible in one source, the value of the information gathered from contract management activities is reduced and hinders Employment’s ability to identify non-compliance and trends in provider behaviour.

Tip-offs and complaints

4.12 Intelligence and feedback about providers and their activities can be provided to the:

  • Employment Services Tip-Off Line (ESTOL) by current and former employees of providers; and
  • Employment Services National Customer Service Line (NCSL) that provides an avenue for job-seekers, employers or other relevant parties.50

4.13 Reporting of tip-off data from ESTOL (including allegations and actions taken by Employment) is provided to PISCES on a monthly basis. Employment reported that 36 per cent of the 58 jobactive tip-offs received via ESTOL in 2015–16 related to business operations, and 27 per cent related to the Work for the Dole program.51 Of the 47 finalised complaints, 38 per cent were substantiated and resulted in ongoing monitoring of the provider’s operations or the provider being reminded of its obligations.

4.14 In 2015–16, the NCSL received 16 969 complaints about providers, which accounted for 30 per cent of the 57 293 contacts received about the jobactive program.52 Employment investigates individual complaints made against employment service providers, but does not analyse trends in the data received through the NCSL. Analysing this complaints data would assist Employment to detect employment service provider non-compliance.

Rolling Random Sample

4.15 The Rolling Random Sample is a major component of the Assurance Strategy introduced for the jobactive program.53 The principle objective of the Rolling Random Sample is to provide assurance that the jobactive program is being delivered as expected by providing: a comparative measure of employment service provider compliance; and intelligence to assist Employment to manage provider and program risks. The Rolling Random Sample involves reviews of a sample of claims or activities from each of the 44 employment service providers. Reviews are undertaken quarterly to verify: if the services were delivered; and the appropriateness of the documentary evidence that supports the claim or activity. The claims for the quarter are drawn randomly from within the focus areas being reviewed, with the sample stratified by transaction type.54

4.16 The Rolling Random Sample process is described below:

  • Employment selects the sample and requests evidence from providers to support the claims;
  • Employment staff around the country initially assess whether the sampled claims adhered to the Deed(s) and guideline requirements;
  • a portion of the sample is quality assured55, which involves verifying the initial assessment, a consistency check to ensure uniformity around the country, and a final review to check provider consistency;
  • if necessary, changes will be made to the assessment;
  • employment service providers are advised of the preliminary outcome of the assessment and have the opportunity to dispute an assessment by showing cause and providing additional evidence prior to the assessment being finalised; and then
  • Employment finalises the assessment and recovers payments where appropriate.

4.17 The finalised claim is categorised as: satisfies requirements; requirements mostly satisfied (no demerit); requirements partially met (demerit); or requirements not met (payment recovery). The demerit result is applied to the Compliance Indicator score (discussed from paragraph 4.27).

4.18 The ANAO analysed the results for the first three quarters of the sample, see Table 4.1, and found:

  • the number of quality assurance reviews undertaken in the third quarter decreased substantially from the earlier quarters (13 per cent of the total sample, compared to greater than 60 per cent in the first two quarters);
  • the number of decisions changed due to the quality assurance process remains high, with 67 per cent changed in quarter 356; and
  • a high number of decisions were amended following the show cause/dispute process, ranging from 88 per cent in quarter 1 to 59 per cent in quarter 3.57

Table 4.1: Rolling Random Sample results for quarter 1 to quarter 3, July 2015 to March 2016

Key statistics

Q1

Q2

Q3

Total

Total population of eligible claims for the quarter

34 748

76 375

142 336

253 459

Initial assessment

Number of claims sampled (as a percentage of total population)

3900
(11%)

3387
(4%)

5026
(4%)

12 313
(5%)

Claim result following initial assessment

Satisfactory requirement

1985
(51%)

1862
(55%)

2890
(57%)

6737
(55%)

Requirements mostly satisfied

963
(25%)

814
(24%)

1361
(27%)

3138
(25%)

Requirements partially met

288
(7%)

310
(9%)

480
(10%)

1078
(9%)

Requirements not meta

664
(17%)

401
(12%)

295
(6%)

1360
(11%)

Quality assurance process

Number of claims quality assured (as a percentage of claims initially assessed)

2362
(60%)

2175
(64%)

638
(13%)

5175
(42%)

Initial assessment changed due to quality assurance (as a percentage of quality assured claims)

831
(35%)

491
(23%)

430
(67%)

1752
(34%)

Show cause / dispute process

Number of claims disputed by employment service providers (as a percentage of claims initially assessed)

328
(8%)

338
(9%)

447
(9%)

1113
(9%)

Preliminary assessment changed due to show cause and dispute process (as a percentage of quality assured claims)

289
(88%)

253
(75%)

263
(59%)

805
(73%)

Final claim result (and outcome)

Satisfactory requirement

2286
(59%)

2010
(59%)

2993
(60%)

7289
(60%)

Requirements mostly satisfied (no demerit)

1064
(27%)

838
(25%)

1407
(28%)

3309
(27%)

Requirements partially met (demerit)

267
(7%)

285
(8%)

429
(9%)

981
(8%)

Requirements not met (demerit and recovery)

283
(7%)

254
(7%)

197
(4%)

734
(6%)

Recoveries

Total amount recovered

$66 321

$109 677

$106 341

$282 339

         

Note a: An initial assessment of ‘requirements not met’ includes claims assessed by the department as requiring additional documentation from the provider or the payment to be recovered.

Source: ANAO analysis of departmental data.

4.19 Following quarter 1, Employment identified that quality assurance was resulting in unforeseen delays and made refinements to the internal processes, including more guidance for quality assessors. As the quality assurance process is a critical step in instilling confidence in the Rolling Random Sample process internally and with providers, further consideration should be given to why such a high number of assessments continued to be changed as a result of the quality assurance process.

4.20 Employment advised that the focus of the Rolling Random Sample for the first year was on educating employment service providers. However, Employment has not published advice regarding lessons learnt until after the subsequent sample had been requested or provided the Rolling Random Sample outcomes to employment service providers in a timely manner.58 This makes it difficult for providers to apply learnings from the results.59

4.21 As shown in Table 4.1 the volume of claims assessed as ‘requirements not met’ improved over the first three quarters. However, Employment has not articulated a compliance target that signals the level of non-compliance it is willing to accept. Nor does it extrapolate the findings of the Rolling Random Sample across the population of similar claims. There would be value in setting a compliance target, estimating the volume of non-compliance, and assessing if it aligns to the department’s risk tolerance levels. For example, Table 4.2 shows the results for Employment Fund claims, if the Rolling Random Sample results were extrapolated to the total population.

Table 4.2: Employment Fund claims

Quarter

Total number of claims

Sampled claims not meeting requirements

Extrapolated claims not meeting requirementsa

1

26 234

206 (6.26%)

1478 to 1806

2

67 985

68 (5.18%)

3169 to 3873

3

119 883

80 (3.75%)

4054 to 4955

       

Note a: Given the margin of error, there is a 90 per cent likelihood that the number of claims in the population is within this range.

Source: ANAO analysis.

4.22 In December 2016, Employment undertook a review of the Rolling Random Sample assessment process. The review identified five main issues, including that it is unclear how the Rolling Random Sample fits into the overall assurance strategy.60 The department intends to seek further advice on implementing best practice, and expects to respond to the review by mid-2017.

Targeted assurance activities

4.23 Targeted assurance activities can be prompted by information gathered from a single source or a combination of monitoring activities including the Rolling Random Sample, site visits, tip-off line, complaints, or media. Targeted assurance activities may focus on: a specific element of the program (such as Work for the Dole or the Employment Fund); an individual provider or a group of providers; or a specific location or region. Targeted assurance activities can be undertaken by various areas across the department.

4.24 In September 2016, Employment reported that there was no standardised process for initiating and recording targeted activities. A ‘high level’ summary of targeted program assurance activity was reported to PISCES for the first time advising that work was underway to capture information on all assurance activities being conducted across the department to provide greater oversight.61 The department also developed a How to Undertake a Targeted Assurance Activity guidance document, to assist with consistency and standardising processes. In March 2017, the ANAO requested a consolidated list of all targeted assurance activities undertaken by Employment since 1 July 2015, which the department was unable to provide.62

4.25 While the development of guidance and recent reporting of targeted assurance activities to PISCES is an improvement, the approach to prioritising targeted assurance activities for the jobactive program is not coordinated. Further work is needed to ensure that the prioritisation process is risk-based to minimise duplication and direct resources to the areas of highest risk.

Data analytics

4.26 Employment is developing an approach to using data analytics with the aim of identifying areas of potential program integrity weakness and trends in the jobactive program.63 Employment has undertaken a limited number of data analytic activities to detect non-compliance and the analysis has been used by the department to inform other compliance activities, such as targeted activities. Employment should continue to develop and use data analytics to identify high-risk behaviour and increase the effectiveness of the Assurance Strategy.

Measuring provider compliance

4.27 Employment introduced a Compliance Indicator for employment service providers to measure the compliance of individual providers.64 The Compliance Indicator is designed to be calculated from the combined results of the previous 12 months of the Rolling Random Sample, targeted activities and contract management review outcomes. The June, September and December 2016 Compliance Indicator results were calculated using only Rolling Random Sample results.

4.28 A Compliance Indicator score of 95 or above (out of 100) is considered to meet Employment’s target level of compliance with the Deed(s).65 The average provider region Compliance Indicator score in December 2016 was 89.3 (with scores ranging from 73.6 to 99.6). While 89.6 per cent of the provider regions did not achieve the target score of 95, Employment had reported to PISCES in November 2016 that similar results in June and September 2016 meant that ‘providers generally have a high degree of compliance’.

Is non-compliance effectively managed?

Employment manages non-compliance through the Remedial Action Framework. As at November 2016, six incidents of non-compliance had been investigated under the framework. The department has not analysed the reasons for the low number of recorded non-compliance cases.

Remedial Action Framework

4.29 Employment has established a Remedial Action Framework (RAF) to assist with consistent, appropriate and timely management of non-compliance with the jobactive Deed(s) and guidelines. Non-compliance can relate to the payment of claims, or non-financial elements such as poor performance or not meeting service delivery obligations that results in a non-financial loss to the department, job-seeker, or employer.66

4.30 Employment’s arrangements to manage non-compliance under the RAF incorporate a range of remedial actions, including recovery of payments and referral to the Serious Non-Compliance and Investigations Unit (SNCIU).67 Employment may also consider departmental corrective actions, including adjusting policy settings, amending internal and external guidance and provider education.

4.31 The RAF outlines the process for escalating and managing remedial actions, which involves three steps.68 If fraud is suspected, staff are required to follow the fraud protocol and where appropriate continue the remedial action process alongside the fraud investigation.

4.32 There have been limited RAF matters managed under the jobactive program. As at April 2017, 14 incidents were being investigated, dating back to September 2015. Of the 14 cases:

  • none related to fraud, with 12 of the 14 incidents relating to human error or an issue with operational procedures;
  • all 14 incidents related to non-financial matters concerning privacy; and
  • seven finalised matters were rated as low impact non-compliance, and the seriousness of the remaining seven was still under consideration as part of the assessment process.

4.33 Examples of the remedial action taken by Employment include requests to review and update internal processes related to records management and privacy, and requests for provider staff to participate in extra training about privacy. These remedial actions are consistent with the RAF guidance.

Recovery of payments

4.34 Providers are paid for any claim for which they can supply true, complete and accurate documentary evidence sufficient to prove that they: are entitled to the payment; have delivered the services relevant to the claim; and have done so in accordance with the Deed(s) and guidelines.69 When it is determined that an invalid claim has been made, Employment recovers all payments, with Employment guidance stating partial recovery should only occur in exceptional circumstances.

4.35 The ANAO’s analysis of Rolling Random Sample data confirmed that for the first three quarters (covering the claim period from 1 July 2015 to 31 March 2016):

  • 734 claims were categorised as ‘requirements not met (recovery)’ (six per cent of reviewed claims)70;
  • the total amount recovered was $282 526;
  • the amounts recovered ranged from 19 cents to $5000;
  • 105 of the claims recovered were partial amounts (17.3 per cent of the 606 financial claims); and
  • nine recoveries (all in quarter 2) were for sums greater than the recorded claim amount (for an additional recovery total of $14 078).71
Referral to the Serious Non-Compliance and Investigations Unit

4.36 For matters assessed under the RAF as having a substantial or major impact, referral to the Serious Non-Compliance and Investigations Unit (SNCIU) is considered. As at April 2017, Employment had detected 16 serious non-compliance matters which were referred to the SNCIU. In summary:

  • one matter relates to employment service providers under the jobactive contract;
  • three cases relate to the Work for the Dole program; and
  • twelve of the matters relate to the previous employment services contract, four of which involved fraud. Five of the cases were being managed by the SNCIU, and nine of the cases were either with Commonwealth Department of Public Prosecutions for consideration or were before the courts.

4.37 Until February 2017, Employment’s guidance had not clearly identified the types of matters that were required to be referred to the SNCIU. As a result of the department’s Compliance Maturity Assessment72, in February 2017 Employment finalised three key guidance documents including the Serious Non-Compliance and Investigations Unit Operating Policy (includes roles and responsibilities, and high-level referral and assessment processes); Enterprise Compliance Framework (includes the high-level compliance principles and governance); and the Fraud Control Plan 2017–2019 (the regulatory framework for managing fraud). Promulgation of these documents would better place staff to manage referrals for serious non-compliance.

Review of the Remedial Action Framework

4.38 Employment planned to commence a review of the RAF in May 2017. The review, which is to be completed by August 2017, includes an assessment of the current framework, and will address identified deficiencies, including requirements relating to investigative processes and delegations, gaps in guidance, and the challenges with practical application of the RAF by staff.

4.39 The purpose of the RAF is to enable a consistent and appropriate approach to managing non-compliance, whether it occurs due to inadvertent errors or wilful actions. The department has not analysed whether the low number of recorded non-compliance cases is due to a high level of compliance, non-compliance not being identified, or non-compliance not being recorded in line with Employment processes. When undertaking the review, Employment should consider if all matters of non-compliance are being identified, managed and documented appropriately.

Has the effectiveness of the Assurance Strategy been assessed?

Employment has not assessed the effectiveness of the Assurance Strategy, but has commenced a review to identify best practice and focus resources on areas of highest priority.

4.40 Reductions to the cost of regulation under the jobactive program were expected to be achieved by reducing prescription and red tape for service providers.73 Over the first 18 months of the program, Employment had not assessed if the Assurance Strategy is achieving its intended aims, measured the cost of implementing the suite of compliance activities, or monitored savings or the cost of regulation for providers. Employment has reviewed two key program assurance components—QAF (as discussed in paragraph 4.8) and the Rolling Random Sample (as discussed in paragraph 4.22).

4.41 In February 2017, Employment contracted an external consultant to undertake a review of its Assurance Strategy to align Employment’s current approach with better practice with the aim of focusing resources on areas of highest priority. The review is scheduled to be completed in May 2017.

4.42 As the Assurance Strategy is Employment’s framework for managing contractual compliance, the ANAO considers that the department should:

  • assess if the delivery of the strategy reflects the department’s preferred level of compliance;
  • effectively guides the allocation of departmental resources towards areas of the highest risk; and
  • considers the level of compliance burden on providers.

4.43 These areas are not covered in the current review.

Recommendation no.2

4.44 The Department of Employment should assess whether the current compliance regime is structured to effectively and efficiently detect and manage non-compliance, and adjust as appropriate.

Entity response: Agreed.

4.45 The Assurance Review currently underway is considering how resources can be focused on the areas of highest priority and risk. A separate review of the Rolling Random Sample is also underway, and will improve its efficiency, reduce the compliance burden on providers and deliver more timely feedback.

5. Performance measurement, reporting and evaluation

Areas examined

This chapter examines the Department of Employment’s (Employment’s or the department’s) performance frameworks for the five services of the jobactive program, the utilisation of the Star Ratings performance measure, internal and external reporting on program performance and the department’s approach to evaluating jobactive.

Conclusion

The performance frameworks for the five jobactive services measure the performance of providers. The Key Performance Indicators developed by the department, which align with program objectives, have been developed for three of the five services, but performance targets have only been established for one of the services. Employment has an evaluation strategy for jobactive, but it does not address some aspects of the program, including contract management and the Star Ratings.

Area for improvement

The ANAO made one suggestion for Employment to consider how it will assess the use of Star Ratings by job-seekers and employers in the future.

Has an effective performance framework for providers been established?

Employment has performance frameworks in place for the five services which measure the performance of providers. Key Performance Indicators for employment service providers, New Enterprise Incentive Scheme providers and Work for the Dole Coordinators align to the objectives of the jobactive program, but performance targets have only been set for Coordinators.

5.1 The performance frameworks for each of the five jobactive services are outlined in the Performance Framework Guideline and in the jobactive Deed 2015–20 or jobactive Deed 2015–20: Work for the Dole Coordinators. Key performance indicators (KPIs) are stipulated in the Deeds for three jobactive services: jobactive employment services; New Enterprise Incentive Scheme (NEIS); and Work for the Dole Coordinators.74 The KPIs for each service cover three areas (efficiency, effectiveness, and quality and assurance). The providers in each service have different performance measures, as shown in Table 5.1.

Table 5.1: The performance measures for the three jobactive services with KPIs

Employment service providers

NEIS providers

Work for the Dole Coordinators

Efficiency KPI a

The average time taken from commencement to the achievement of the 26 week outcome.

The time taken for relevant participants to commence in a Work for Dole phase activity.

The number of NEIS places utilised.

The number, or rate, of participants that exit NEIS services prior to completion, and the reasons for the exit(s).

The number of Work for the Dole places that the Coordinator secures against any targets.

The extent to which the number of places secured is available to all employment service providers across the employment region(s), and which meet the employment service providers’ caseload needs.

Effectiveness KPI

The proportion of outcome payments claimed for the relevant participants.

The proportion of relevant participants that meet their Annual Activity Requirements.

The number of NEIS outcomes achieved.

The appropriateness of each Work for the Dole place that the Coordinator secures for a variety of eligible job-seekers and for the delivery of work-like experiences.

Quality and assurance KPI

Certification against the Quality Assurance Framework.

Compliance Indicator results.

Assessment of service delivery against the tender commitments.

Delivery of NEIS services in accordance with the Deed.

The number of validated complaints, the number resulting in ministerial correspondence and any Ombudsman complaints.

Input received from participants in the Post Programme Monitoring Survey.

Coordinator’s reports (including self-assessment quality reports).

Overall compliance with the Deed and guidelines.

Outcomes of any performance reviews undertaken by the department.

Any other basis that the department determines as relevant to the quality of services delivered by the Coordinator.

     

Note a: Employment’s efficiency KPI relates to time effectiveness of providers in delivering outcomes, which does not align to the standard definition of efficiency in the public service which is the unit cost of an output generated by an activity. Department of Finance, Resource Management Guide No.131: Developing good performance information, Finance, Canberra, April 2015, p. 48.

Source: ANAO analysis of the jobactive Deed 2015–20 and the jobactive Deed 2015–20: Work for the Dole Coordinators.

Performance framework for employment service providers

5.2 The performance measures for employment service providers, shown in Table 5.1, align with the four objectives of the jobactive program.75 Employment has not articulated targets for the performance indicators. However, data for the efficiency and effectiveness KPIs is used as the basis for the calculation of the publicly released comparative performance measure, the Star Ratings (discussed further from paragraph 5.14).

5.3 Employment service providers have six-monthly performance periods, with performance discussions held with employment service providers at the end of each period. The department also produces weekly reports on the performance of each employment service provider’s sites and provider-regions76, with a comparison to the employment region and national averages, although employment service providers are only able to see their own data.

5.4 The Deed states that if Employment considers the performance against the KPIs of an employment service provider to be less than satisfactory, then the department can reduce the employment service provider’s business share or close relevant sites (separate to the business reallocation process based on the Star Ratings, which is discussed further from paragraph 5.31). Employment uses a comparative assessment to determine whether the performance of employment service providers is satisfactory or unsatisfactory. There would be benefit in formalising a methodology for comparing the performance of providers to ensure that decision-making is robust and transparent.

Indigenous Outcome Targets

5.5 One of the stated objectives of the jobactive program is to drive better employment outcomes for Indigenous Australians, which is intended to be achieved through the introduction of specific Indigenous Outcome Targets. These targets are a standalone measure of performance. The Performance Framework Guideline states that, in each region where an employment service provider operates, it will achieve parity with the proportion of Indigenous job-seekers in its caseload for the 4, 12 and 26 week outcomes across the three streams. As the Performance Framework Guideline outlines:

For instance, if 5 per cent of the job-seekers on the Employment Provider’s Stream B caseload identify as Indigenous then at least 5 per cent of the Provider’s 4 Week, 12 Week and 26 Week outcomes in Stream B must be achieved for Indigenous job-seekers.

5.6 In March 2016, Employment assessed the outcomes achieved for Indigenous Australians during the first seven months of the jobactive program and found the program to be achieving less parity than under the Job Services Australia model. The ANAO’s analysis of employment service providers’ performance against the Indigenous Outcome Targets found that the majority of provider-regions had not achieved the targets during 2015–16 (see Table 5.2).77 The ANAO notes that 5 star providers were less likely to achieve the Indigenous Outcome Targets than those providers that performed mid-range in the Star Ratings.78

Table 5.2: ANAO analysis of employment service providers performance against Indigenous Outcome Targets, 2015–16

 

Performance Period 1 (July to December 2015)

Performance Period 2 (January to June 2016)

Provider-regions which achieved all of its targetsa

4 (2.0 per cent)

0

Provider-regions which achieved the targets on average

73 (36.1 per cent)

38 (18.8 per cent)

Provider-regions which achieved none of its targets

15 (7.4 per cent)

39 (19.3 per cent)

     

Note a: Performance Period 1 had targets for four and 12-week outcomes but not for 26-week outcomes. This resulted in six specific Indigenous Outcome Targets in Performance Period 1 (by outcome payment and job-seeker stream classification) and nine targets in Performance Period 2.

Source: ANAO analysis of Employment’s Indigenous Outcome Target performance data.

5.7 Employment advised that it is too early to assess the effectiveness of the introduction of Indigenous Outcome Targets. Nevertheless, any site not achieving the targets is considered for a Performance Improvement Plan. As a result of performance discussions during the first year of the jobactive program, a Performance Improvement Plan was developed for all employment service providers. The plans included improvement strategies related to the Indigenous Outcome Targets.

5.8 In September 2016, Employment reviewed whether the scoring of Indigeneity in the stream classification of job-seekers was accurately reflecting the employment barriers faced by Indigenous Australians. The department found that Indigenous job-seekers have a lower outcome rate overall relative to non-Indigenous job-seekers with the same assessment of disadvantage, and on average had higher rates of disadvantage compared to other job-seekers within their classification stream. The draft review recommended that the Job Seeker Classification Instrument process increase the impact that Indigeneity has on a job-seeker’s classification to more accurately reflect the employment disadvantage experienced.79

Performance framework for New Enterprise Incentive Scheme (NEIS) providers

5.9 For the jobactive program, a performance framework was introduced for NEIS providers to improve provider performance. As a result, the Deed included three KPIs for NEIS providers (see Table 5.1). The performance measures of the KPIs are aligned to the objectives of the jobactive program. While the framework is well-founded, Employment has not set performance targets for the NEIS KPIs.

5.10 The ANAO found low utilisation of NEIS placements in the first year of the program, with only 19 (21.1 per cent) of the 90 NEIS provider-regions80 and four (7.8 per cent) of the 51 employment regions using all of its allocated places, for a total utilisation of 80 per cent nationally. In the first six months of 2016–17, 11 (12.2 per cent) provider-regions and two (3.9 per cent) employment regions used more than 50 per cent of their annual places, and total utilisation was 36 per cent nationally.

Performance framework for Work for the Dole Coordinators

5.11 Employment assesses a Work for the Dole Coordinator’s performance against the KPIs listed in Table 5.1 at the end of the six-monthly performance period. The KPIs align to the program objectives, however, the effectiveness KPI may be difficult for the department to assess.81 The Deed states that Coordinators are to submit performance reports following every performance period, which are to include a self-assessment of the Coordinator’s progress against the KPIs. All progress reports were submitted in the required format for the first three performance periods (May 2015 to December 2016).

5.12 The ANAO found improvements in the results for the three KPIs for the first three performance periods (May 2015 to December 2016). While the aggregate performance of the Work for the Dole Coordinators improved over the first 19 months of the jobactive program, there were five Coordinators (26.3 per cent) with an increase or constant number of ‘not met’ assessments in at least one region. Due to the reliance of employment service providers on Coordinators to source sufficient activities, it is important that Employment proactively manages under-performing Coordinators.82

Performance framework for Harvest Labour Service and National Harvest Labour Information Service providers

5.13 Employment has not set KPIs for the five Harvest Labour Service (HLS) providers and the National Harvest Labour Information Service (NHLIS) provider. The Deed stipulates that HLS and NHLIS providers will self-report on progress quarterly and annually against requirements detailed in the Performance Framework Guideline. All required reports were submitted to Employment for the five quarters from July 2015 to September 2016. For the HLS providers, 41 of the 55 quarterly reports addressed all the requirements from the guidelines (74.5 per cent), and two of the five providers included all the required information in the 2015–16 annual report. In 2015–16, the NHLIS provider submitted an annual report but not the fourth quarter report.

Are the Star Ratings being used as designed?

The Star Ratings were designed to be used by job-seekers and employers to inform their choice of employment service provider, and by Employment for business reallocation. The department does not have a strategy in place to assess whether the Star Ratings are used by job-seekers or employers to influence their choice of employment service provider. The department has used the Star Ratings for the first round of business reallocations.

5.14 The Performance Framework Guideline states that the Star Ratings are an assessment of an employment service provider’s performance against two of the service’s KPIs (efficiency and effectiveness), and will be used to inform job-seeker and employer choice as well as the department’s business review and reallocation processes.

5.15 Employment designed the Star Ratings to provide:

  • Continuous improvement—4 and 5-star ratings are more attainable, encouraging providers to strive to increase performance;
  • Value for money—15 to 20 per cent of contracts are in scope for business reallocation (providers rated as 1 or 2-star), allowing the department to regularly review the poorest performers;
  • Job-seeker choice—there is sufficient spread of ratings to allow job-seekers to differentiate between the providers in their employment region based on performance; and
  • Flexibility—Star Rating bandwidths can be adjusted at specified points throughout the Deed (with providers to be advised of changes via updates to the guidelines).

Calculating the Star Ratings

5.16 An employment service provider’s Star Ratings are calculated at both the site and employment region levels, and are a relative comparison to the national average. The calculations are based on a two-year rolling assessment period, and are calculated quarterly. The process for calculating the Star Ratings is described in Figure 5.1.

Figure 5.1: The process for calculating the Star Ratings

Source: ANAO analysis of Employment’s Performance Framework Guideline.

5.17 The Star Ratings for each job-seeker classification stream are calculated using the same performance measures and weightings, which are:

  • 26 week outcome—all job-seekers (50 per cent);
  • 26 week outcome—Indigenous job-seekers (10 per cent);
  • 26 week outcome—time to placement (10 per cent);
  • 12 week outcomes (10 per cent);
  • Work for the Dole phase—participation (10 per cent); and
  • Time to commence in Work for the Dole/Annual Activity Requirement (10 per cent).83

5.18 As the ratings are a national comparison, Employment’s regression analysis intends to account for differences between the local labour markets and job-seeker populations by calculating the actual outcome for each measure compared to the department’s predicted outcome of what an employment service provider who was achieving the national average would have attained. The predicted outcomes are not shared with the providers, and several stakeholders consulted by the ANAO described the regression model of the Star Ratings as a ‘black box’.

5.19 Following the regression step, a standardisation step was re-introduced for the jobactive program.84 Standardisation results in a more uniform distribution of scores and produces a wider distribution of Star Ratings. The standardisation approach adopted by Employment is two-fold. Firstly, the regression scores are compared to the target average and standard deviation; and secondly, the outlier scores are reset to fixed minimum and maximum scores.

5.20 After these calculations, the scores for each measure are combined using the weightings listed in paragraph 5.17 to generate a performance score for each job-seeker classification stream (‘star score’). The distance of the star score from the national average for each stream is termed the ‘star percentage’. The distribution of the star percentages into bandwidths is subsequently used to generate the Star Ratings, as shown in Table 5.3. Employment narrowed the bandwidths compared with those used under the Job Services Australia model, because of a convergence at the 3 star rating under the previous model.85

Table 5.3: Bandwidths for the distribution of Star Ratings

Star Percentage

Star Rating

30 per cent or more above the national average

5 star

Between 15 and 29 per cent above the national average

4 star

Between 14 per cent above and 14 per cent below the national average

3 star

Between 15 and 39 per cent below the national average

2 star

40 per cent or more below the national average

1 star

   

Source: Employment’s Performance Framework Guideline.

5.21 Lastly, as a reflection of the different assistance required for the job-seeker streams, a provider’s Overall Star Rating for each site is calculated by aggregating the star percentages from each stream with the following weightings:

  • Stream A—25 per cent;
  • Stream B—35 per cent; and
  • Stream C—40 per cent.
  • The overall star percentages are then converted to Star Ratings as per Table 5.3.

Star Ratings results

5.22 The first publicly released ratings for the jobactive program, based on the first 12 months of operation (the June 2016 Star Ratings), were published in August 2016.86 The September 2016 Star Ratings were released in November 2016, and the December 2016 Star Ratings released in February 2017.87

5.23 The publicly released Star Ratings do not include the Star Percentage or the effect of the Compliance Indicator score on the Star Rating.88 For employment service providers, the department has created comparative infographics for each quarterly release available on the Qlik application89, which illustrates how the sites and regions compare to the national average for each of the six performance measures.

5.24 The ANAO’s analysis of the Star Ratings data from June to December 2016 is presented in Figure 5.2. While more releases will be required to confirm if the standardisation process introduced by Employment will limit convergence to the centre, early indications suggest it has been successful given the wide distribution of the Star Rating results. However, the number of sites rated as 1 or 2 stars is higher than Employment forecast when the Star Ratings were designed (32.8 per cent, compared to the forecast maximum of 20 per cent).

Figure 5.2: Percentage of sites by Overall Star Rating, June to December 2016

Source: ANAO analysis of published Star Ratings.

5.25 While nationally the employment provider sites have a reasonable distribution of Overall Star Ratings, there are some employment regions across the country where few employment service providers are performing well in the Star Ratings. For example, in three of the eight employment regions in Western Australia, all 23 sites are rated as 1 star, and in a different Western Australian employment region 10 of the 11 sites are rated as 1 star. Additionally, between the June and December 2016 releases, only 21 (20.2 per cent) of the 104 sites in Western Australia recorded an improvement in the star percentage, with an average change of minus 9.7 (and a range from 37 to minus 45).90

Interaction with the Compliance Indicator

5.26 The Compliance Indicator was first incorporated into the Star Ratings calculation in September 2016.91 In the September 2016 Star Ratings, the star percentage was reduced for 27 provider-regions (13.4 per cent) once the Compliance Indicator penalty had been applied (with four provider-regions losing a whole Star Rating as a result). This improved in the December 2016 Star Ratings, with no provider-regions losing a star and 24 provider-regions (11.8 per cent) receiving a star percentage reduction as a consequence of the interaction with the Compliance Indicator.

5.27 Seven of the 18 employment service providers and peak bodies the ANAO consulted raised the interaction between compliance and performance. They stated that it is time consuming to comply with the numerous compliance requirements (discussed in Chapter 4) and this is impacting on their ability to secure performance outcomes for job-seekers. The ANAO analysed the relationship between a provider-region’s Star Percentage and Compliance Indicator score. As shown in Figure 5.3, there is no discernible correlation between performance and compliance, with less than 0.3 per cent of the variance in the Compliance Indicator score attributable to the Star Percentage result.

Figure 5.3: Relationship between Star Rating and Compliance Indicator, by provider-region, December 2016

Note: Where an employment service provider’s Compliance Indicator score is below 83 (dotted line on graph), penalties are to be applied to the employment service provider’s Star Percentage. The Star Percentage is reduced by half a percentage for every point that the Compliance Indicator is below 83.

Source: ANAO analysis of December 2016 Star Ratings and Compliance Indicator results.

Use of the Star Ratings

5.28 The Star Ratings are designed to be used by job-seekers and employers to inform their choice of employment service provider, and by the department in the business reallocation process.

Informing job-seekers and employers

5.29 Employment reported to the ANAO that during the first 15 months of jobactive (July 2015 to September 2016), the vast majority of job-seekers (94.7 per cent) had chosen their own employment service provider (compared to 90.5 per cent during Job Services Australia). Employment has not evaluated the reasons driving provider choice and the impact of Star Ratings on a job-seeker’s decision, or whether employers use Star Ratings to inform their choice of employment service provider.

5.30 Employment has advised that as the Star Ratings data has only been publicly available since August 2016 it is too early to examine whether Star Ratings are a factor in job-seeker or employer choice of employment service provider. Given that use of the Star Ratings by job-seekers and employers was a design feature of the Star Ratings, the department should consider how it will assess use of the Star Ratings in the future.

Business Reallocation

5.31 Business reallocation is the process whereby the department has discretion to reduce an employment service provider’s market share in an employment region (to zero if necessary), or to notify the provider that they must discontinue delivering services at a particular site. The Performance Framework Guideline states that there will be two performance-based business reallocations for employment service providers, at the 18 and 36 month points of the Deed. Employment service providers who have regions or sites that have a Star Rating of 1 or 2 will be in scope for business reallocation.92 When designing the Star Ratings, the department had anticipated that 15 to 20 per cent of provider-regions would be in scope for business reallocation.

5.32 Employment advised the Minister and employment service providers that business reallocation 18 months into the Deed will be based on the December 2016 Star Ratings. The ANAO’s analysis of the December 2016 Star Ratings found 32.8 per cent of sites (559 of 1704 sites) were in scope for business reallocation as they were rated as 1 or 2 Stars. Employment has advised that it is in the process of reallocating places and that the changes will take effect on 3 July 2017.

Is progress towards the achievement of jobactive program outcomes monitored and reported?

Employment monitors progress against the targets in the Portfolio Budget Statement and Corporate Plan through performance reporting to the relevant governance committee. The department publicly reported on progress against the targets in its 2015–16 Annual Report.

Internal reporting

5.33 Progress of the jobactive program is reported to Employment Steering Committee (ESC), the committee that oversees performance, among other things. In December 2015, the department noted that it was not on track to meet several jobactive outcome targets from its Portfolio Budget Statements and Corporate Plan.93 As a result, a regime of monthly performance reporting to ESC was implemented.

5.34 In May 2016 this approach was revised so that only five underperforming measures would be provided in the monthly reports, with all remaining performance targets reported on a quarterly basis from September 2016. The five measures to be reported to the committee monthly were: 4 and 12 week conversions for Indigenous job-seekers; Indigenous job placements; appointment attendance; and overall job placements.

5.35 The performance of the first year of jobactive was reported to the Joint-ESC meeting in July 2016. Employment assessed that the jobactive program met or exceeded 12 of the 19 targets set for 2015–16.

External reporting

5.36 Employment reported on its performance against the four objectives of the jobactive program in its Annual Report 2015–16 (see Table 5.4).94 Employment reported against five of the six performance indicators from the Portfolio Budget Statements, it did not report on the cost per employment outcome. The data was largely consistent with internal reporting previously provided to ESC.95

Table 5.4: Employment’s reporting on jobactive program performance, Annual Report 2015–16

Performance indicator

Target

Reported result

Objective 1: Helping job-seekers find and keep a job

Count of job placements (total)

380 000

346 324

Count of job placements (Indigenous)

38 000

28 443

Proportion of job placements sustained to 4 weeks

68%

56.8% overall (50.2% Indigenous)

Proportion of job placements sustained to 12 weeks

44%

48.3% overall (40.3% Indigenous)

Proportion of job placements sustained to 26 weeks

24%

31.6% overall (24.8% Indigenous)

Proportion of job-seekers employed three months following participation in jobactive

45% overall

47.3% overall

  • Stream A

55%

56.6%

  • Stream B

35%

39.5%

  • Stream C

25%

25.5%

Objective 2: Help job-seekers move from welfare to work

Proportion of job-seekers moving off income support, or with significantly reduced reliance on income support, six months following participation in jobactive

40%

43.0%

Objective 3: Help job-seekers meet their mutual obligations

Proportion of Work for the Dole participants who report increased motivation to find a job

75%

73.8%

Proportion of jobactive appointments attended (appointments for activity-tested job-seekers, excluding appointments where the job-seeker had a valid reason for not attending)

90%

71.4%

Proportion of job-seekers (with mutual obligation requirements) who are actively looking for work

95%

94%

Objective 4: jobactive organisations deliver quality services

Proportion of employers satisfied with the assistance provided by a jobactive organisation

80%

84%

Proportion of jobactive organisations that meet the service delivery requirements

80%

91%

     

Note: Performance indicators from the Portfolio Budget Statements reported in the Annual Report 2015–16 include: count of job placements; proportion of jobs sustained to 4, 12 and 26 weeks; and the proportion of job-seekers moving off income support, or with significantly reduced reliance on income support, six months following participation in jobactive.

Source: Employment Annual Report 2015–16.

5.37 External reporting on the performance indicator ‘proportion of jobactive organisations that meet the service delivery requirements’ is not accurate. The Annual Report stated that ‘in 2015–16, 91 per cent of jobactive organisations met the service delivery requirements that were assessed’, against a target of 80 per cent. The ANAO found that:

  • nine of the 44 employment service providers had no commitments assessed during 2015–16, and were omitted from the analysis;
  • only 2183 of the 7593 service commitments (28.7 per cent) had been assessed;
  • Employment used an 80 per cent benchmark to determine whether individual employment service providers had achieved its service delivery requirements, but this benchmark is not outlined in the Deed or guidelines; and
  • ‘met’ commitments included those that were ‘partially met’. This approach is not explained in the report. If the calculation included only commitments which were assessed as being ‘met’, then 62.9 per cent (22) of the 35 providers would have achieved the 80 per cent benchmark.96

Has a sound approach to evaluate the jobactive program been established?

Employment has developed an evaluation strategy for two of the five services of the jobactive program, including jobactive employment services, which adopts a staged approach to evaluation. The strategy does not cover three of the services or other key elements of the jobactive program, such as contract management and the Star Ratings.

5.38 Employment developed the Evaluation Strategy for jobactive in late 2014 and published it on the department’s website in December 2016. The evaluation strategy states that the department will produce an Interim Evaluation Report (scheduled to the completed in December 2016)97, a Meeting Employer Needs Report (March 2018), and the Final Report (July 2019). The strategy outlines the four overarching evaluation questions, the proposed evaluation methodology, and the various data sources to be used in the evaluations.

5.39 The evaluation strategy covers two of the five services of the jobactive program (jobactive employment services and Work for the Dole Coordinators).98 The strategy does not cover the New Enterprise Incentive Scheme, Harvest Labour Services, or the National Harvest Labour Information Services. Also excluded from the evaluation strategy is a review of contract management and the Star Ratings.

Appendices

Appendix 1 Entity response

Appendix 2 Contracted providers for the five jobactive services

Table A.1: Providers contracted to deliver jobactive services and the number of employment regions where contracteda, funding for 2015–16

Trading name

jobactive employment services

New Enterprise Incentive Scheme

Work for the Dole Coordinator

Funding for 2015–16

ABS Institute of Management

-

2

-

$0.8m

Advanced Personnel Management

9

-

-

$22.2m

AMES Australia

5

-

-

$20.2m

Atwork Australia Pty Ltd

2

-

-

$14.7m

Auctus Business Training and Consulting

-

1

-

$0.1m

BEST Employment Limitedb

3

-

1

$9.9m

Box Hill Institute

-

7

-

$1.4m

BRACE Education and Training

-

2

-

$0.3m

Business Foundations Inc

-

8

-

$0.8m

BUSY At Work Employment Services

-

-

2

$0.5m

Campbell Page Limited

1

-

-

$6.1m

Centacare Employment

1

-

-

$1.9m

Central NSW Business Enterprise Centre

-

1

-

$0.2m

CoAct

7

-

1

$32.8m

Communicare Inc.

2

-

1

$10.7m

Complete Personnel

1

-

-

$4.2m

CVGT Australia Ltd

3

-

2

$11.4m

Direct Recruitment Pty Ltd

2

-

-

$11.7m

Djerriwarrh

-

-

1

$0.6m

Employment Directions

-

-

1

$0.4m

Employment Services Group

6

-

-

$30.2m

Enterprise & Training Company Limited

3

2

-

$14.8m

Eworks Employment Solutions

2

-

-

$6.6m

G-Force Staffing Solutions

1

-

2

$3.0m

Global Skills

2

-

-

$7.9m

HELP ENTERPRISES LIMITED

3

-

-

$12.8m

Holmesglen Institute

-

8

-

$1.2m

Job Prospects (aka SYC Ltd)

8

-

1

$40.2m

Joblink Plus

4

-

-

$17.7m

Jobs Australia Enterprises Ltd

1

-

-

$4.2m

Jobs Statewide Employment Solutions

3

-

-

$18.0m

MADEC Australiabc

6

-

-

$20.4m

MatchWorks

8

-

5

$30.4m

MAX Employment

29

-

14

$153.0m

MBC Employment Services

2

-

-

$8.5m

Mission Providence Pty Ltd

8

20

6

$47.2m

MTC Australia

-

2

-

$0.5m

My Pathwayd

1

4

3

$5.4m

My Pathwayd

-

-

1

$0.6m

National Business Advisory Services

-

1

-

$0.1m

NEATO Employment Services Pty Ltd

6

-

-

$35.6m

NetGain

-

3

-

$0.5m

New Hunter Business

-

1

-

$0.3m

NORTEC Staffing Solutions

4

4

-

$19.8m

OCTEC Limited

-

-

2

$1.4m

Olympus Solutions Limited

1

-

-

$4.8m

Parkes Forbes Business Enterprise Centre

-

1

-

$0.04m

PeoplePlus

3

1

-

$9.2m

Q.I.T.Eb

1

-

-

$5.3m

RMIT University

-

3

-

$0.6m

Sarina Russo Job Accessb

11

13

-

$68.3m

SETS-South East Training Services

-

2

-

$0.6m

Skill Hire WA

4

-

1

$7.2m

St Laurence Community Services Inc

3

-

-

$5.5m

Status Employment Services

1

-

-

$3.6m

Sureway Employment and Training Pty Ltd

7

-

1

$26.9m

Sydney Business

-

4

-

$1.0m

The Job Shopb

-

-

-

$0.4m

The Salvation Army Employment Plus

13

-

-

$56.6m

Tursa Employment & Training

7

-

3

$51.7m

VERTO Ltd

1

-

-

$3.1m

Western District Employment Access Inc

4

-

-

$9.5m

WISE Employment Ltd

4

-

-

$14.6m

Workskil Australia

8

-

3

$38.6m

Workskills Incorporated

1

-

-

$5.6m

Total

202

90

51

$943.3me

         

Note a: The harvest areas for Harvest Labour Services and National Harvest Labour Information Service do not align to the 51 jobactive employment regions, so the providers with these contracts are reported separately.

Note b: These five providers are contracted as Harvest Labour Service providers.

Note c: This one provider is contracted as the National Harvest Labour Information Service provider.

Note d: This provider signed two separate Work for the Dole Coordinator contracts due to sub-contracting arrangements.

Note e: The total amount of funding to providers in 2015–16 was $943.3m, due to rounding errors the total when adding the individual provider totals is $944.3m.

Source: ANAO analysis of departmental documentation. Financial values provided by Employment in June 2017.

Appendix 3 Mutual Obligation Requirements

1. Mutual Obligation Requirements are a range of requirements a job-seeker can be compelled to fulfil under Social Security Law in return for income support payments. For example, attending provider appointments, minimum job search requirements, and meeting their Annual Activity Requirement. The Annual Activity Requirement is the number of hours and the timeframe over which a job-seeker must undertake suitable activities. A job-seeker must satisfy their Annual Activity Requirement during the Work for the Dole Phase. This phase is the period during which Work for the Dole, or another approved activity (for example, study or volunteering), is undertaken in conjunction with job search requirements.

2. In addition to the Work for the Dole phase, a job-seeker will progress through a Case Management phase, and those in Stream A (general) will have a Self Service and Job Activity phase for their first six months under the jobactive program.

3. An individual job-seekers’ Mutual Obligation Requirements will vary depending on their age and stream placement, an example of the requirements for job-seekers up to 30 years can be found below in Table A.2.

Table A.2: Mutual Obligation Requirements – job-seekers up to 30 years

Period of Service

Stream A (general)

Stream B (general)a

Stream C

0–6 months

Self Service and Job Activity

  • appointments
  • job search (20 per month)
  • other suitable activities

Case Management

  • appointments
  • job search (20 per month)
  • other suitable activities

Case Management

  • appointments
  • job search (depends on capacity)
  • other suitable activities

6–12 months

Case Management

  • appointments
  • job search (20 per month)
  • other suitable activities

Case Management

  • appointments
  • job search (depends on capacity)
  • other suitable activities

12–18 months;

24–30 months; etc.

Work for the Dole phase

  • appointments
  • job search (depends on capacity)
  • Annual Activity Requirement: 650 hours over 26 weeks (50 hours per fortnight), Work for the Dole as the principle activity

18–24 months;

30–36 months; etc.

Case Management

  • Appointments
  • job search (20 per month)
  • other suitable activities

Case Management

  • appointments
  • job search (depends on capacity)
  • other suitable activities
       

Note a: From 1 October 2016, Stream A and Stream B job-seekers with Stronger Participation Incentives have the same Mutual Obligation Requirements as those in Stream B (general).

Source: Mutual Obligation Requirements and Job Plan Guideline.

Appendix 4 Determination of breaches and possible remedial actions

Table A.3: Determination of breaches and possible remedial actions

Type of breach

Threshold

Examples of possible remedial actions

Low impact breaches

A breach that has resulted in a nil or very low negative impact on job-seekers or employers or Employment, or the reputation of the program or employment services industry, and has resulted in a nil or minimal advantage to the provider.

Financial loss <$50 000.

Number of invalid claims <2% of all claims reviewed for the year in the RRS.

Minimal impact on job-seeker or employer.

No evidence of undue benefit.

No reputational damage to program or the sector.

Expected performance (not part of star ratings) is <10% below established benchmark.

Provider to investigate and/or rectify.

Education.

Provider to review and report on specific internal processes and policies.

Provider to evaluate and report on the effectiveness of its training and quality assurance.

Substantial impact breaches

A breach that has resulted in a moderate negative impact on Employment, job-seeker or employer, or has resulted in a moderate advantage to a provider.

Financial loss is between $50 000 ≤$200 000.

Number of invalid claims is >2% but <5% of all claims reviewed for the year in the RRS.

Has negative effect on job-seeker, or on more than one employer.

Scope of breach: evidence that the breach is happening across one to two sites within an employment region, and has happened more than once.

Undue benefit to the provider (including impact on the star rating).

Temporary or potential for there to be a negative effect on the reputation of the program or sector.

Performance is below department’s expectations or performance is >10% but ≤30% below the established benchmark

Provider to develop and implement specific strategies to prevent reoccurrence of the breach.

Provider to undertake in-depth self-audit activities and report findings back to Employment.

Additional reporting requirements.

Additional financial or performance requirements on the provider.

Additional conditions on payment of fees or use of the Employment fund including restricting the use of the Employment Fund.

Additional conditions on claiming.

Reducing or not paying specific payments that would otherwise have been payable.

Reducing the total amount of any payments temporarily.

Liquidated damages if the number of requirements not met (recovery) claims ≥100 in a financial year.

Suspending further referrals until the breach has been rectified.

Suspending any payments under the Deed, in whole or in part.

Applying a minor reduction in the provider’s business share.

Reducing referrals to the provider or transferring job-seekers to another provider.

Suspending or restricting the providers access to Employment’s IT system.

Major impact breaches:

A breach that has resulted in a major negative impact on one or more job-seekers, or one or more employers, or the department, or the employment industry, or has resulted in a major advantage to a provider.

Financial loss ≥$200 000.

Number of invalid claims ≥5% of all claims reviewed for the year in the RRS.

Widespread, large or ongoing impact on job-seeker or a number of employers were involved, or if provider continues to commit further breaches.

Scope: as for substantial breach AND the breach occurred at more than two sites across an employment region, or in multiple employment regions, or evidence that the breach behaviour is widespread across the provider.

Undue benefit: resulted in widespread, large and/or ongoing advantage to the provider (including impact on the star rating).

Widespread, large and/or ongoing negative effect of the breach on the reputation of the program.

The Performance level is below department’s expectations or >30% below the established benchmark.

Terminating the provider’s access to Employment’s IT system.

Applying a substantial business share reduction.

Deferring payments in a region until evidence proves the legitimacy proves the legitimacy of claims.

Reducing the number of NEIS claims in a region.

Reducing the number of payments permanently.

Reducing the scope of the Deed such as closing a site.

Terminating the Deed for default.

     

Note: The list of remedial actions is provided as examples; it is not a complete list.

Source: Employment documentation.

Footnotes

1 A complete list of the 65 providers for jobactive is provided at Appendix 2.

2 A complete list of the 65 providers for jobactive is provided at Appendix 2.

3 Job-seeker mutual obligations are described in Appendix 3.

4 Disability Support Pension recipients with a work capacity of more than eight hours per week are generally eligible for Disability Employment Services (administered by the Department of Social Services).

As at 6 February 2017, 26 per cent of job-seekers on the jobactive caseload self-reported a disability.

5 As at 6 February 2017 there were 10 557 volunteer job-seekers (1.4 per cent) in jobactive, of the 750 000 total participants on the jobactive caseload.

6 Department of Employment, Annual Report 2014–15, Employment, Canberra, September 2015, p.11.

7 Stream placement is largely determined by the Job Seeker Classification Instrument, a principally self-reporting tool administered by the Department of Human Services. In order for a job-seeker to be placed in Stream C, or referred to Disability Employment Services, an Employment Services Assessment is undertaken by the Department of Human Services to determine an individual’s work capacity.

8 For example, an employment service provider could receive an outcome payment of $6250 if a Stream C job-seeker who has been unemployed for greater than 60 months and lives in a regional location is eligible for a 26-week outcome. Alternatively, if the same job-seeker lived in a non-regional area the outcome payment to the employment service provider would be $5000, or if the job-seeker was in Stream B the outcome payment would be $3938.

9 The 13 recommendations included: establishing an Industry Consultation Forum; consolidating the stream structure and extending contract periods; encouraging greater innovation; establishing an industry-standard accreditation scheme for providers; and introducing a risk-based assurance framework.

10 Department of Education, Employment and Workplace Relations, Initial Government Response to the Advisory Panel on Employment Services Administration and Accountability: Final Report, DEEWR, Canberra, September 2012, p.2.

11 The Employment Fund refers to funding which is held by the department, but may be accessed by employment service providers for reimbursement of purchased goods or services. The employment service provider is allocated credits for each job-seeker that joins its caseload, but those credits are not tied to individual job-seekers.

12 Additionally, in April 2017, the Government agreed to terminate Work for the Dole Coordinator contracts by 1 January 2018.

13 When an employment service provider manages multiple places in a single activity with job-seekers from other employment service providers then it is considered to be the ‘Lead Provider’ and will manage the relationship with the host organisation and Work for the Dole Coordinator.

14 Work for the Dole fees are paid by Employment to the employment service provider and are set at $1000 per individual hosted activity place and up to $3500 per group-based activity place. The employment service provider, Work for the Dole Coordinator and activity host organisations then decide how the fees will be allocated. The Lead Provider fee is allocated from within the available fees, and is only permissible when the Lead Provider manages job-seekers from more than one employment service provider in activities involving more than one place.

15 Employment advised the Review that Harvest Labour Services had not been evaluated or audited since it was formed in 1998. The Review that was undertaken was not an evaluation or audit of the service, rather it sought views from stakeholders and made recommendations based on their feedback.

16 The Review suggested that the definition of ‘harvest work’ be aligned to the Migration Regulations 1994 (Cth) Working Holiday Visa—Definitions of Specified Work and Regional Australia 2016/087, so that any work undertaken while on a Working Holiday Visa would be eligible for a placement fee to the Harvest Labour Service provider.

17 All discussion and decision papers are to be presented to EBMC prior to ESC consideration (unless exempted by the Secretary), with EBMC agreement or considerations summarised for inclusion in the papers to ESC.

18 Program areas relevant to jobactive include the branches of: Assessments, Services and Outcomes; Job-Seeker Activation and Cluster Support; Work for the Dole; Employment Services Reporting and Analysis; Programme Assurance; Providers; Purchasing; Specialist Programmes and Indigenous Employment Taskforce; Incentives and Investments; Youth Employment; and Employer Mobilisation.

19 Employment has regional offices in all six state capitals, plus Bendigo, Canberra, Darwin, Mackay, Newcastle, Orange, Rockhampton, and Townsville.

20 The National Leadership Team is chaired by the Group Manager of the Delivery and Engagement Group, with membership consisting of the seven State Managers.

21 The 195 staff may also have responsibilities for other programs, including Transition to Work.

22 Provider 360 is the information system that the department uses to record, track and report on contract management information about providers.

23 Employment is an auditee in the ANAO performance audit The Management of Risk by Public Sector Entities which is due to table in mid-2017. The audit focuses on the risk policies, frameworks and cultures of the designated entities.

24 The Performance Framework Risk Plan covers the five services of the jobactive program; the Payment Integrity Risk Plan covers four of the five jobactive services (not National Harvest Labour Information Service) as well as the Transition to Work, ParentsNext and Empowering YOUth Initiative programs.

25 One risk in the jobactive Risk Plan (‘injury resulting from providers not adhering to Work Health and Safety principles’) is not included in the NEIS program area risk plan, but is included in the other program area risk plans.

26 Employment has created region-level risk plans for five jobactive providers. The ANAO examined the risk plans and found they were not always fit-for-purpose. For example, of the 13 region-level risk plans created for a particular provider, only one plan contained concerns specific to that region.

27 Examples of risk actions include: discussions between the provider and Employment to address the risk concerns; the requirement to provide audited financial statements to Employment; desk-top monitoring to review adherence to the Deed; and conducting an impromptu review of provider files.

28 The review periods set by the department for each level of risk are: at least monthly for plans rated as extreme; at least quarterly for plans rated as high; and at least annually for plans rated medium or low.

29 Employment has one risk plan for a consortium provider that has two Work for the Dole Coordinator contracts (as the subcontractors were different in various regions).

30 Employment’s guidance states that ‘the agreed risk appetite means that any risks or assessments that are identified as High or Extreme are deemed unacceptable and are required to be managed and mitigated, through treatments, to a manageable or acceptable level’.

Department of Employment, Risk Management for Employment Services Contracted Entities: Guiding Principles, Employment, Canberra, June 2016, p.18.

31 Holding meetings with providers was a frequently applied risk action; which would not necessarily mitigate the risk identified. For example, Employment was to hold a teleconference with a provider identified as having a high financial viability risk.

32 Employment advised that the new guidelines had been developed with input from the Transition Working Group whose membership includes the industry peak bodies and a sample of providers from Job Services Australia.

33 A further guideline, NEIS Referral and Eligibility Guidelines, does not refer to mutual obligation requirements.

34 Question Manager is an online system that supports the lodgement, transmission and resolution of policy and operational orientated questions, and facilitates the escalation of questions where required from Delivery and Engagement Group staff, to State Offices, and finally to the National Office.

35 This equates to an average of two queries a day (based on a five day working week) since the start of jobactive.

36 The ANAO reviewed jobactive queries relating to outcomes and the Employment Fund, for the period 1 July 2015 to 27 July 2016.

37 Queries that the ANAO reviewed without analysing, included those: identified as duplicates; reportedly answered via a different method, for example telephone; or concerning information technology or processing.

38 Employment analysed Question Manager data for the period from 1 July 2015 to 3 December 2015 (the first five months of jobactive) and reported the results to ESC. The report noted that: 65 per cent of queries were answered within the ten business day requirement stipulated in the Joint Charter for Deed Management (with an average duration of 9.9 business days); queries were resolved within 0 to 63 days; more than a third of the queries related to outcomes (80 of the 237 queries submitted); and National Office answered the majority of the queries (54 per cent).

39 The Provider Portal is a repository for information relating to programs delivered by contracted providers. It includes information on the deeds and contracts, operational guidelines and other reference material, and any important new information that needs to be communicated to the contracted providers.

40 An employment service provider’s Service Delivery Plan sets out the services job-seekers and employers can expect to receive, as detailed in the Deed. Employment also identifies service offer commitments, being any other commitments core to the overall servicing strategy that should be monitored and assessed throughout the Deed period.

41 The jobactive Service Guarantee sets out what: job-seekers in each of the different service streams can expect from providers in terms of services, courtesy, and confidentiality; and the requirements job-seekers need to meet while looking for employment as outlined in an individual job plan.

42 The Joint Charter of Deed Management sets out the two-way commitments made between Employment and jobactive providers regarding service expectations.

43 The three acceptable standards for the QAF certification are: International Quality Management Systems Standard, ISO 9001; Employment Services Industry Standards; and National Standards for Disability Services.

44 The overarching quality principles are: governance; leadership; staff; participants; labour market, employers and community; operational effectiveness; and, continual improvement.

45 This provider was granted an extension to 30 June 2017, as the organisation had undergone a number of changes to its operations and its QAF report was no longer current.

46 These were: Job Services Australia Quality Standards Pilot Evaluation (finalised on 3 November 2014); Reviewing the Quality Assurance Framework: Implementation and support materials (finalised in June 2016); and Review: Employment’s Quality Framework team review of the QAF (finalised in December 2016).

47 Costs advised by providers varied significantly, and the financial costs associated with preparing for the certification audit (median cost of $47 500) were higher than the cost of completing the audit (median cost of $36 000). Similarly, there were significant variations in the reported hours spent preparing and supporting the audit process, with a median of 500 hours required.

48 Contract management activities also assist with stakeholder management and identifying provider risks.

49 In October 2016, the ANAO observed three site visits undertaken by Employment and found the visits contributed to stakeholder relationships via the sharing of jobactive program messages, high-level monitoring of the 7593 service commitments, and performance management issues.

50 The NCSL also handles calls relating to Disability Employment Services and Community Development Programme participants (on behalf of the Departments of Social Services and Prime Minister and Cabinet). Complaints can also be made through the Commonwealth Ombudsman.

51 In 2014–15, ESTOL received 212 tip-offs about providers operating under the Job Services Australia contract.

52 In 2014–15, the NCSL received 12 210 complaints about providers, accounting for 37.6 per cent of the 32 431 contacts about Job Services Australia.

53 Under Job Services Australia claims were manually selected on an ad-hoc basis and reviewed by Employment staff at provider sites.

54 The focus areas reviewed in quarters 1 to 4 (1 July 2015 to 30 June 2016) included outcome payments, wage subsidies, and Employment Fund reimbursements. Claims verified by data held by the Department of Human Services are ineligible for the Rolling Random Sample.

55 Employment has set a mandatory target of claims to be quality assured of 20 per cent of the total sample each quarter. This includes a random selection of claims and those deemed by Employment to be high risk.

56 Employment advised that the majority of changes during quality assurance process were ‘a result of assessors’ interpretation of the guidelines and their interaction with the assessment tools and the overall bedding down of the Rolling Random Sample process’.

57 Employment advised that the majority of decision changes during the show cause/dispute process were a result of new evidence or information submitted by the employment service providers.

58 For example, learnings for providers from quarter 1 were published on the provider portal on 16 March 2016, two months after the quarter 2 claims were sampled in January 2016. Additionally, the finalised results for quarter 1 were sent to providers on 17 June 2016 (nine months after the claims were sampled), and after quarter 3 sample claims had been requested on 6 April 2016.

59 In June 2017, Employment provided detail of timeframes for quarters 4 to 6. The initial results for the claims of quarter 6 were sent to employment service providers on 20 April 2017, less than three months after the claims were sampled. Final result letters were provided to employment service providers in early June (five months after the claims were sampled), which is an improvement on the nine months required for the first quarter.

60 The other four issues identified were: defining the objective and purpose; documenting resource implications, including a cost-benefit analysis; distinguishing between results based on binary and judgmental responses to questions during the assessment process; and establishing a screening process for the inclusion of a focus area.

61 In September 2016 it was agreed that high-level reporting would occur biannually. In April 2017, targeted assurance activities for the period September 2016 to February 2017 were reported to PISCES.

62 The April 2017 reporting to PISCES listed 15 targeted assurance activities which had been reported centrally, but the report did not include timeframes, rationale or follow-up required.

63 Data analytics uses available data to identify patterns of provider behaviour which may result in follow-up with a provider and/or targeted assurance activity. Examples include building, testing and deploying predictive models to inform compliance activity, and trialling data matching with other agencies.

64 The interaction of the Compliance Indicator with the Star Ratings performance measure is discussed in Chapter 5.

65 The Compliance Indicator has a maximum permitted margin of error of 12 per cent.

66 To assist with readability, the term non-compliance is used in this report. Employment also uses the term ‘breach’, which it defines as a failure to satisfy any obligation or requirements under the Deed(s) which has been substantiated through evidence. A financial breach results in a financial loss to Employment and includes invalid claims.

67 Other remedial actions include: suspending or terminating access to the department’s IT systems; imposing special conditions on the making of payments, the accessing of the Employment Fund or the management of records; reducing, not paying or recovering specific payments; termination of the Deed with costs; and reduced provider business share.

68 The three steps are: an assessment of the impact of the non-compliance on stakeholders as low, substantial or major; followed by consideration of the circumstances of the non-compliance; and then selection of a remedial action with consideration given to appropriateness and consistency. Thresholds for assessing low, substantial and major non-compliance are summarised in Appendix 4.

69 With the exception of payments verified by Department of Human Services data in accordance with any guidelines.

70 There were 134 claims categorised as ‘requirements not met (recovery)’ for which no payments were recovered as the claims related to non-financial Work for the Dole or Job-Seeker Classification Instrument claims.

71 Recoveries may include associated claims for the same activity. Eight of the nine additional recoveries in quarter 2 related to relocation assistance for job-seekers.

72 In August 2016, a Compliance Maturity Assessment was conducted to capture best practice compliance strategies across the department and identify areas that may require further support from the SNCIU. One of the potential gaps identified was limited guidance on those matters that should be referred to the SNCIU.

73 In June 2014, Employment’s Regulation Impact Statement for the jobactive program estimated that the regulatory cost of the Job Services Australia operations was in the order of $259.3 million per year. In comparison, the regulatory cost of the jobactive model was estimated to be approximately $197.1 million per year (including the impact of regulation on employment service providers, employers and individual job-seekers but not the cost to the department).

Savings were expected due to a number of reasons, including: more provider discretion on service interventions for job-seekers; information technology that is more user friendly; simplified outcome definitions and evidentiary requirements; streamlined administration fee claims; three service streams for job-seeker rather than four streams to reduce complexity; the introduction of Work for the Dole Coordinators to promote Work for the Dole and gather suitable places; and more concise guidelines.

74 Employment has not stipulated KPIs for the Harvest Labour Service or National Harvest Labour Information Service.

75 As outlined in Chapter 1, the four objectives of jobactive are to: help job-seekers find and keep a job; help job-seekers move from welfare to work; help job-seekers meet their mutual obligations; and jobactive organisations deliver quality services.

76 Under jobactive, the 44 individual employment service providers may be operating in more than one of the 51 employment regions (the largest provider is delivering employment services in 29 employment regions). The term ‘provider-region’ is used in this report to denote the 202 combinations of employment service provider and employment regions.

77 Feedback from employment service providers to the ANAO about Indigenous Outcome Targets focused on the difficulty of achieving a target which is always moving dependent on other performance outcomes, the lack of granularity in the results with the result being either a pass or fail, and the poor performance of the entire sector reflecting how hard the parity target is to achieve. Employment service providers welcomed the stronger focus on ending disadvantage for Indigenous Australians.

78 For instance, 23 per cent and 26 per cent of employment service providers with Star Ratings of 2 and 3 respectively met the Indigenous Outcome Targets, compared to 8 per cent of 5 star providers.

79 Employment advised that the recommendation to review the Job Seeker Classification Instrument is being considered in the context of ongoing program improvements and future policy developments.

80 There are 21 NEIS providers across the 51 employment regions, resulting in 90 provider-regions.

81 The effectiveness KPI (appropriateness of Work for the Dole places secured for a variety of eligible job-seekers and delivery of work-like experiences) is measured by two performance measures which do not provide a measure of appropriateness. The measures are: percentage of places recorded in department’s IT system which meets Deed and guideline requirements; and the spread of places by industry, occupation and task reflects the local labour market.

82 In April 2017, the Government agreed to terminate Work for the Dole Coordinator contracts by 1 January 2018. Instead, employment service providers would source Work for the Dole placements and liaise with host organisations.

83 Two of the measures (26 week outcome—time to placement and Time to commence in Work for the Dole/Annual Activity Requirement) relate to the efficiency KPI, while the other four measures relate to the effectiveness KPI.

84 Standardisation was used in the Star Ratings under the former employment services model Job Network, but was removed from the Job Services Australia calculations of Star Ratings.

85 At the first Job Services Australia Star Ratings release in June 2010, 58 per cent of contracts were rated as 3 star. By June 2014, the percentage of 3 star rated contracts had risen to 86 per cent.

86 In May 2016, each employment provider was provided with its own preliminary employment region-level Star Ratings based on the data available from July 2015 to March 2016. These calculations were not shared with other employment service providers or the public.

87 As at 9 May 2017, the Star Ratings for each employment service provider site were available from the jobactive website <http://jobsearch.gov.au> or the Department’s website <http://www.employment.gov.au/jobactive-star-ratings-and-performance>.

88 Employment has set a target level of compliance with the deed and guidelines, with a Compliance Indicator score of 95 and above considered to meet the department’s target. Due to the 12 per cent margin of error in the Compliance Indicator calculation, the department will only apply penalties to an employment service provider’s Star Percentage when its Compliance Indicator score is less than 83. The Star Percentage will be reduced by half a percentage for every point that the Compliance Indicator is below 83.

89 Qlik is a reporting application that Employment has provided to providers, which allows for more dynamic representation of the data than the previous spreadsheet reporting allowed.

90 Seven of the 104 sites had not commenced sufficient job-seekers in all three job-seeker classification steams for a complete Star Rating to be calculated. Instead, an estimation of the Star Ratings was generated using available site and region data for the employment service provider. The seven imputed sites had an estimated star percentage change between June and December 2016 ranging from minus 15 to minus 2.

91 Employment’s guidance states that 90 per cent of provider-regions need to have a Compliance Indicator calculated before the Compliance Indicator will be incorporated into the calculation of the Star Ratings. In June 2016, as only 77 of the 202 provider-regions had a Compliance Indicator score (38 per cent), Employment did not include it in the Star Ratings calculation.

92 Employment has also stated that providers with poor performance against service delivery commitments, the Compliance Indicator, or Indigenous Outcome Targets, may also be in scope for business reallocation.

93 As of mid-December 2015, there was only five months of data available to review, so there was no data available for four of the six Portfolio Budget Statement measures, or seven of the additional thirteen targets from the Corporate Plan.

94 The Annual Report is the only forum Employment uses to report on the performance of the jobactive program.

95 There were some minor discrepancies between the internal and externally reported data. For example, the Annual Report included 1135 more job placements than the internal reporting (a 0.3 per cent variation).

96 Of the 2183 commitments assessed, 1538 were assessed by the department as being ‘met’ (70.5 per cent).

97 In March 2017, the draft Interim Evaluation Report was presented to the EBMC for discussion.

98 The two services covered by the evaluation strategy are the areas of major change from the previous employment services model, Job Services Australia.

Related documents: