Audit snapshot

Why did we do this audit?

  • Social services programs require close management of risks to ensure program outcomes are being achieved.
  • Each year there are around 17,000 humanitarian entrants to Australia who are eligible to access the Humanitarian Settlement Program.
  • This audit has reviewed whether this program is being delivered effectively.

Key facts

  • The Humanitarian Settlement Program provides services to assist humanitarian entrants to become self-reliant and active members of the Australian community.
  • Services are delivered by five providers across 11 contract regions. Services commenced in October 2017.
  • There are three needs-based tiers. Clients receive orientation services and support to establish themselves in the community.
  • Management of the program and contracts moved from the Department of Social Services to the Department of Home Affairs on 1 July 2019.

What did we conclude?

  • The delivery of the Humanitarian Settlement Program has been partially effective.
    • Overall management of the HSP contracts has been partially effective, although relationship management was strong.
    • The HSP IT system does not fully support the delivery of the program.
    • The lack of reliable performance information means the department cannot determine the effectiveness of the program.
    • Only some of the intended benefits of moving to the HSP have been realised.

What did we recommend?

  • The Auditor-General made three recommendations to the Department of Home Affairs in relation to contract management, the IT system, and performance information.
  • The Department of Home Affairs agreed to all recommendations.

$129.8 m

allocated for the HSP in 2018–19.

15,158

clients received services through the HSP in 2018–19.

408,958

claims were processed in 2018–19.

Summary and recommendations

Background

1. The Humanitarian Settlement Program (the HSP) assists eligible clients to settle in Australia and is part of the Government’s broader humanitarian program. HSP clients include humanitarian entrants to Australia and some onshore visa holders that meet the HSP eligibility requirements.

2. Humanitarian entrants to Australia were previously supported through the Humanitarian Settlement Services (HSS) and Complex Case Support (CCS) programs. Following a 2015 evaluation, the programs were merged into the HSP, with an increased focus on English, education, and employment outcomes. The Department of Social Services (DSS) commenced operation of the HSP in October 2017. The program was transferred to the Department of Home Affairs in July 2019.

Rationale for undertaking the audit

3. The HSP provides initial settlement support to people who have recently arrived in Australia on refugee or humanitarian visas and to some eligible people onshore. The objective of the HSP is to build people’s skills and knowledge for social and economic well-being through a tailored, needs-based, case management approach. Over 17,000 clients are eligible for the HSP each year and the program is allocated over $120 million annually.1 The ANAO has not previously audited the HSP or the prior versions of this program but audited settlement services grants in 2009. This audit assessed whether DSS and Home Affairs have managed the HSP effectively following the commencement of the HSP in October 2017.

Audit objective and criteria

4. The objective of this audit was to examine whether the HSP is being delivered effectively. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

  • Are the HSP contracts well designed and managed?
  • Is the performance of the HSP effectively managed to achieve program outcomes?

Conclusion

5. Delivery of the Humanitarian Settlement Program (HSP) has been partially effective.

6. The HSP contracts are largely well designed but contract management has only been partially effective. Service provider contracts contain appropriate provisions supported by strong stakeholder engagement by DSS. Key contract management elements were not undertaken, including establishment of abatement regimes, contract management plans or adequate assurance activities. The HSP IT System only partially supported program delivery due to issues obtaining reliable information from external systems, challenges with accessing performance information, and the administrative burden of using the system.

7. The departments have been partially effective in managing HSP performance to achieve program outcomes. While HSP risk management processes have been appropriate, performance management and reporting has not been appropriate and available data does not allow the departments to determine if the objective of the HSP is being met. Despite some improvements having been made to the program, with more planned, not all intended benefits of the new HSP delivery model have been achieved.

Supporting findings

Contract design and management

8. The HSP contracts reflect the HSP outcomes and contain appropriate provisions. The contract documents are consistent with the HSP Program Outcomes Framework and include the common contract provisions in the Australian Government Contract Management Guide. The contracts have 71 service payment points. There are caps on the number of times some services can be used (without separate departmental approval) to support probity.

9. DSS’ management of the HSP contracts has been partially effective. DSS had appropriate governance arrangements for the program, including strong engagement with and examination of issues raised by service providers. However, an abatement regime in the contracts has not commenced, contract management plans were not established, and assurance activities since July 2018 have only examined a small percentage of clients through desk based reviews. Home Affairs is finalising its HSP governance arrangements and plans to commence a new assurance and compliance strategy for the HSP at the start of 2020.

10. The HSP IT system partially supports contract management and the delivery of the HSP. While the HSP system manages payments effectively, not all data (such as health information) has been migrating correctly from external systems, adding risk to the delivery of essential services. Reporting against key performance indicators and on client outcomes was delayed partly due to IT system issues. The increased administrative burden of using the HSP IT system from October 2017 to April 2019 led to DSS making additional payments to service providers. Over 100 system enhancements and fixes have been implemented since the system went live which largely resolved the administrative burden issue. Earlier user acceptance testing may have assisted more timely identification and mitigation of these issues. DSS had spent $16.2 million on the HSP IT system as at 30 June 2019, which is $3 million more than the original estimate. Some originally planned functionality has still not been implemented. Home Affairs and DSS are developing a memorandum of understanding regarding future management of the HSP IT System, including establishing priority system enhancements.

Program performance management

11. The departments have employed an appropriate process for HSP risk management. DSS’ and Home Affairs’ risk management activities for the HSP comply with the Commonwealth Risk Management Policy. Consideration of risk is evident in key program documents and four risk assessments have been conducted. Mitigation strategies were updated as the program continued and Home Affairs is reviewing its risk appetite following transfer of the HSP. In the September 2019 risk assessment, the post treatment risk rating is low for four risks and medium for two risks.

12. HSP program performance has not been appropriately managed or reported, and the department was not able to determine if the objective of the HSP is being met. Client outcome data is not being tracked and Key Performance Indicators (KPIs) in the contracts have not been effectively implemented. A KPI ‘relief period’ has been in place since the program started, and non-binding reporting of a limited set of KPIs commenced in the July to September 2019 quarter. It showed that all three KPIs were met by only one of the five providers during the quarter. There has also been limited internal and external reporting on program performance, however DSS did prepare the first six-monthly report on the HSP for the period up to 31 December 2018 based on reports from the five service providers.

13. Continuous improvements are being made to the delivery of the HSP, with further work required. A number of benefits were intended to be gained when moving from the previous programs to the HSP. These improvements were largely reflected in the HSP contracts but the benefits have only been partially achieved. For example, shifts to outcomes focused contracting, improved data and a new IT system have only been partly achieved. Other improvements to the HSP have been identified via a number of reports and reviews. Flowing from this, Home Affairs prepared a high-level schedule with seven elements to improve the HSP. This includes finalising and implementing the recommendations of the 18 month review of the HSP.

Recommendations

Recommendation no. 1

Paragraph 2.31

Home Affairs should finalise changes to contract management arrangements for the HSP to ensure full alignment with the Australian Government Contract Management Guide.

Department of Home Affairs response: Agreed.

Recommendation no. 2

Paragraph 2.92

That Home Affairs (in consultation with DSS) continue to prioritise identifying, planning for and resolving HSP IT system issues that are either causing risks to program delivery, or impacting Home Affairs’ ability to manage program performance.

Department of Home Affairs response: Agreed.

Recommendation no. 3

Paragraph 3.71

Home Affairs should finalise the review and update of HSP performance information including by:

  1. refining the KPIs in the HSP contracts to reflect relevant elements of the HSP Program Outcomes Framework;
  2. providing additional guidance to service providers on how to measure attainment of the orientation competencies; and
  3. developing a Data Management Plan to help track if the HSP Program is meeting its objective.

Department of Home Affairs response: Agreed.

Summary of entity response

14. As the Accountable Authority for the Humanitarian Settlement Program the Department of Home Affairs provided a formal response to the proposed audit response — a summary of which is provided below with further information at Appendix 1. The Department of Social Services also provided a responses to the proposed report — which is included at Appendix 1.

Department of Home Affairs

The Department of Home Affairs (the Department) welcomes the ANAO’s findings that the Humanitarian Settlement Program (HSP) contracts are largely well designed, that risk management has been managed appropriately and that stakeholder engagement has been strong. The Department is committed to building on the sustained success of settlement services for humanitarian entrants through the effective delivery of the HSP and is actively making improvements to realise efficiencies and improve outcomes for humanitarian entrants. The HSP will be reorientated to move from an outputs measured model to an outcomes based model providing greater flexibility to respond to client needs and achieve outcomes.

Significant progress has already been made in addressing all of the improvement areas identified in this audit. Contract management and performance information has been enhanced, and further improvements to the HSP IT System are being implemented.

Contract management activities for the HSP were recently centralised within the Department. This streamlined administrative processes and provided greater consistency in approach. The Department is already well advanced in finalising individual contract management plans for each provider, which once in place, will mean that the contractual arrangements are fully compliant with the Australian Government Contract Management Guide.

The Department also identified a range of system enhancements, building on the 123 system improvements already implemented. We will continue to work closely with the Department of Social Services (DSS) on implementation of these enhancements, which will further mitigate risks and improve performance management.

In terms of performance management, the Department considers that it already has good performance information to assess program performance and client progress against outcomes. This includes data on client achievement against the orientation outcomes which directly links to the HSP Outcomes Framework, service provider performance reports, key performance indicators, research, internal data reports on service levels and client outcomes. Notwithstanding this, the Department is implementing a new performance framework which will further strengthen performance reporting. This includes building on existing key performance indicators to better align to the HSP Outcomes Framework and our HSP data management plan to improve tracking of client progress towards outcomes and achievement of the program objective.

The Department will continue to improve guidance to service providers on measuring attainment of orientation outcomes and work to better link the HSP to the Adult Migrant English Program and other settlement programs to provide a more holistic approach to supporting clients to achieve settlement outcomes.

A new Assurance and Compliance Strategy has also been recently finalised to provide assurance that settlement services are delivered appropriately, support client progression towards outcomes, and tests integrity of service provider claims. This includes gaining insights directly from clients on their overall satisfaction of settlement services and where improvements could be made.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Performance and impact measurement

1. Background

Introduction

1.1 Australia’s Humanitarian Program comprises up to 18,750 places in 2019–20.2 The Humanitarian Settlement Program (HSP or the program) provides support to the majority of the clients in the Humanitarian Program as shown below.3 Since the HSP commenced in October 2017 the largest numbers of clients have come from Iraq, Syria, Myanmar, the Democratic Republic of the Congo and Afghanistan.4

Figure 1.1: Onshore and offshore visas granted and HSP client numbersa 2010–11 to 2018–19

 

Note a: This figures includes humanitarian entrant numbers in years prior to the commencement of the HSP in 2017. Actual HSP clients for any year may not equal visas granted in that year due to the continuation of clients from the previous year.

Note: The Department of Home Affairs (Home Affairs) manages the client intake, which has two key components:

  1. offshore resettlement: offered to people overseas who are in the greatest need of resettlement; and
  2. onshore protection: offered to people already in Australia who are found to meet Australia’s protection obligations.

Note: Most off-shore humanitarian entrants are automatically eligible to access supports from the HSP. Onshore, high-needs humanitarian entrants can access the HSP subject to an assessment.

Source: ANAO analysis of Home Affairs reports.

The Humanitarian Settlement Program

1.2 Until October 2017, humanitarian entrants to Australia were supported through the Humanitarian Settlement Services (HSS) and Complex Case Support (CCS) programs which operated from 2011 and 2008 respectively. A 2015 evaluation proposed merging the HSS and CCS programs into a single program to deliver administrative efficiencies.5 The Settlement Services Advisory Council also made recommendations to improve the English language, education and employment outcomes of the programs.

1.3 The HSP commenced in October 2017, replacing the HSS and CCS. The objective of the HSP is to:

Build skills and knowledge for social and economic well-being of eligible clients through a tailored, needs-based, case management approach.6

1.4 The Department of Social Services (DSS) had responsibility for the HSP until 30 June 2019. Under the Administrative Arrangements Order of 29 May 2019 settlement programs, including the HSP, moved to Home Affairs from 1 July 2019.7 In this report:

  • if ‘DSS’ is used, the described activity was conducted prior to 1 July 2019; and
  • if ‘Home Affairs’ is used, the described activity was conducted after 1 July 2019.

1.5 HSP services are intended to be needs-based and there are three service tiers. The highest need clients are in tier three: Specialised and Intensive Services (SIS).8 HSP services are provided to clients until they achieve the settlement outcomes, decide to stop receiving services, or until Home Affairs decides the services should cease. It is expected that most tier one and tier two clients will be provided HSP services for up to 18 months. The duration of delivery of SIS to tier three clients will vary according to the client’s needs.

Eligibility

1.6 Figure 1.2 shows the visa classes that are eligible for the HSP. The offshore Humanitarian Program has two components. Entrants who arrive in Australia under the refugee component (Visa Subclasses 200, 201, 203 and 204) are eligible for the HSP. Entrants who arrive under the Special Humanitarian Program component (Subclass 202) are also eligible for the HSP.

1.7 There are different rules for the Community Support Program (CSP) component of offshore visas. The CSP allows individuals, communities and businesses to support people in humanitarian need through the visa application and settlement processes. People granted a Subclass 202 visa under the CSP are not generally eligible for the HSP and receive settlement services from their Australian supporters coordinated through their Approved Proposing Organisations. However, subject to departmental approval, CSP entrants with complex needs are eligible for HSP SIS (tier three).

1.8 The onshore component of the Humanitarian Program is for non-citizens who are in Australia (other than illegal maritime arrivals), and meet the criteria for the Onshore Protection visa. Some Onshore Protection visa holders who have complex needs are eligible for SIS.

Figure 1.2: HSP eligibility by visa type (visas granted in 2018–19)a

A diagram that shows the different categories of humanitarian entrants to Australia and their eligibility for the Humanitarian Settlement Program (HSP). In 2018-19, 18,762 visas were granted as a part of Australia’s humanitarian program. Of these, 17,112

Note a: DSS HSP Fact Sheet, August 2018 [Internet] available from: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-factsheet.pdf, [accessed 29 November 2019].

Source: ANAO and HSP documentation. Refugee visas are visa sub-classes 200, 201, 203 and 204.

Current HSP providers and budgeted expenditure

1.9 In July 2017 DSS signed contracts with five service providers for 11 regions across Australia to deliver settlement services to HSP clients (see Table 1.1).9

Table 1.1: HSP Service Providers

Service Providers

State / Territory

Contract regions

(11)

Settlement locations

(25)

Settlement Services International Limited (SSI)

NSW

Sydney

Sydney

Regional NSW

Newcastle; Coffs Harbour; Armidalea

Australian Red Cross (ARC)

ACT/NSW

Canberra and surrounds

Canberra; Wollongong; Wagga Wagga; Albury

WA

Western Australia

Perth

AMES Australia (AMES)

Vic

Melbourne

Melbourne

Regional Victoria

Mildura; Shepparton; Geelong; Wodonga

SA

South Australia

Adelaide; Mount Gambier

Tasmania

Tasmania

Hobart, Launceston

Multicultural Australiab

Qld

Brisbane and surrounds

Brisbane; Gold Coast; Logan, Toowoomba

North Qld

Cairns, Townsville

Melaleuca Refugee Centre

NT

Northern Territory

Darwin

       

Note a: On 11 August 2017 Armidale was announced as a new settlement location for humanitarian entrants. SSI was given a one-off payment for establishing the services in Armidale.

Note b: Formerly MDA Ltd.

Source: Humanitarian Settlement Program Fact Sheet, Department of Home Affairs, [Internet], available at: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-fa... [accessed 23 November 2019].

1.10 The budget allocation for the HSP is outlined in Table 1.2.

Table 1.2: Humanitarian Settlement Program budget

Program feature

Resources

2018–19 allocation for the Humanitarian Settlement Program

$129.814 million

2019–20 allocation for the Humanitarian Settlement Program

$124.713 million

   

Source: ANAO summary of DSS documents.

Rationale for undertaking the audit

1.11 The HSP provides initial settlement support to people who have recently arrived in Australia on refugee or humanitarian visas and to some eligible onshore people. The objective of the HSP is to build people’s skills and knowledge for social and economic well-being through a tailored, needs-based, case management approach. Over 17,000 clients are eligible for the HSP each year and the program is allocated over $120 million annually.10 ANAO has not previously audited the HSP or the prior versions of this program but audited settlement services grants in 2009.11 This audit assessed whether DSS and Home Affairs have managed the HSP effectively following the commencement of the HSP in October 2017.

Audit approach

Audit objective, criteria and scope

1.12 The objective of this audit was to examine whether the HSP is being delivered effectively. To form a conclusion against the audit objective, the ANAO adopted the following high-level criteria:

  • Are the HSP contracts well designed and managed?
  • Is the performance of the HSP effectively managed to achieve program outcomes?

Audit methodology

1.13 The audit methodology included:

  • examining program documentation including contracts, program guidance, performance information, risk management documentation, program reviews and reporting information;
  • examining HSP IT system documentation and performance;
  • interviewing relevant departmental officers, including staff in regional offices;
  • interviewing representatives of the five HSP providers; and
  • interviewing representatives from the Settlement Council of Australia.

1.14 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $418,080. The team members for this audit were Katherine Lawrence-Haynes, Deanne Allan, Joel Smith, Hugh Balgarnie, Jocelyn Watts, Samuel Painting and David Brunoro.

2. Design and management of the HSP contracts

Areas examined

This chapter examines if the Humanitarian Settlement Program (HSP) contracts are well designed and well managed.

Conclusion

The HSP contracts are largely well designed but contract management has only been partially effective. Service provider contracts contain appropriate provisions supported by strong stakeholder engagement by DSS. Key contract management elements were not undertaken, including establishment of abatement regimes, contract management plans or adequate assurance activities. The HSP IT System only partially supported program delivery due to issues obtaining reliable information from external systems, challenges with accessing performance information, and the administrative burden of using the system.

Areas for improvement

The ANAO made two recommendation for Home Affairs to:

  • align contract management with the July 2019 Australian Government Contract Management Guide; and
  • resolve the HSP IT system issues which are impacting program outcomes and reporting.

2.1 In order to assess the design and management of the HSP contracts, the ANAO examined whether the HSP:

  • contracts align with program outcomes and have appropriate provisions — because effective contract design should include a clear line of sight from relevant contract deliverables to program objectives, and contract provisions help protect Government interests;
  • contracts are well managed — because robust contract governance, assurance activities, reporting and engagement with providers is needed to ensure a program is delivered effectively; and
  • whether the HSP IT system supports contract management — because the IT system was purpose built, is used by both departmental staff and providers, and facilitates payments and data collection.

Do the contracts reflect the HSP outcomes and have appropriate provisions?

The HSP contracts reflect the HSP outcomes and contain appropriate provisions. The contract documents are consistent with the HSP Program Outcomes Framework and include the common contract provisions in the Australian Government Contract Management Guide. The contracts have 71 service payment points. There are caps on the number of times some services can be used (without separate departmental approval) to support probity.

2.2 The Humanitarian Settlement Services (HSS) and Complex Case Support (CCS) programs transitioned to the Humanitarian Settlement Program (HSP) in October 2017. DSS intended to gain a number of benefits when moving to the new contract arrangements which included moving towards an outcomes focus. An assessment of whether these benefits have been realised is included in Chapter 3.

The Program Outcomes Framework and the services

2.3 The objective of the HSP is to ‘build skills and knowledge for social and economic well-being of eligible clients through a tailored needs-based case management approach’. DSS developed an HSP Outcomes Framework (see Table 2.1) which is reflected in the contracts (it is included as an appendix and it also aligns with the services in the contracts).

Table 2.1: The Program Outcomes Framework (excluding the innovation target outcomes)a

Type

Stage

Deliverable/outcome

Immediate deliverable

Pre arrival to Australia

A client has a Pre-Arrival Assessment in place prior to their arrival

A client has suitable accommodation on arrival

On arrival (within 12 hours)

A client is met and welcomed on arrival to Australia

A client’s immediate cultural and religious, nutritional, hygiene, clothing and footwear needs are met

A client is transported to suitable accommodation

A client knows what to do in an emergency situation

A client’s immediate health needs are met

Post arrival

A client is registered with Centrelink, Medicare and bank within three days of arrival

A client attends critical health appointments

A client knows where their immediate services are

A client can utilise translating and interpreting services

A client has a Case Management Plan in place

Intermediary outcome

 

 

 

 

 

 

 

 

Foundation

 

 

 

 

 

 

 

 

Housing

A client is in secure and suitable accommodation A client can navigate the housing market to secure and maintain suitable rental accommodation

Physical and Mental Health and Wellbeing

A client can use services and implement strategies to improve or maintain physical and mental health and lifestyle

Managing Money

A client can manage and access finances

Community Participation and Networking

A client can develop and maintain links with local communities and continue cultural practices

A client knows about Australian way of life and values

A client can safely use transport

Family Functioning and Social Support

A client can access government services and other support

Justice

A client knows that they have legal rights and responsibilities and how to obtain legal assistance

Language Services

A client participates in AMEP and other English language learning

Education and Training

A client has sought recognition of pre-arrival skills and qualifications

A school-aged client is enrolled in mandatory education

A client can apply for and participate in education and training

Employment

A client can engage with employment services and seek and apply for jobs

A client is on a pathway to achieve their employment goals

A client can access support services for establishing a business

   

Note a: The Program Outcomes Framework also included Innovation Target Outcomes. They had not been activated as at July 2019.

Source: ANAO Summary of DSS documents.

2.4 Consistent with the Program Outcomes Framework, the HSP deliverables in the contract are divided into ‘immediate deliverables’ (within three weeks of arrival to Australia) and ‘foundation services’ (delivered following the initial three weeks). A key feature of the foundation services is the delivery of an onshore orientation program to build clients’ knowledge about the Australian way of life. Services then focus on longer term needs including assistance to source long-term accommodation, access services and connect with the community and employment.

2.5 The HSP service providers employ case managers to assess the needs of HSP clients and services are provided in line with client needs, with reference to the client’s tier. Tier one clients generally need minimal assistance to settle into the community; tier two clients are likely to have experienced significant challenges and need more support; and tier three (Specialised and Intensive Services) clients face multiple and complex settlement barriers.

2.6 Program Outcomes Framework immediate deliverables are linked to specific provisions in the HSP contracts and to related payments in the payment schedule. An example is shown in Table 2.2.

Table 2.2: Links between the outcomes framework and contracts — example

Program Outcomes Framework

Contract provisions

Immediate deliverable

Contract Statement of Requirement

Payment point description

A client is registered with Centrelink, Medicare and bank within three days of arrival

The service provider must arrange appointments for the group to register with Centrelink, Medicare and open a bank account within three business days of the group arriving in Australia.

Payment point: the service provider has supported the group with their Centrelink, Medicare and bank registrations.

   

Source: ANAO analysis of the HSP contract.

Common contract provisions

2.7 The Australian Government Contract Management Guide (July 2019) states that contracts should include 11 common contract provisions.12 The HSP contracts contain the 11 common contract provisions which cover: access and records; insurance, liabilities, guarantees, indemnities and warranties; intellectual property rights; key personnel capabilities; and sub-contracting. There are also other standard provisions including ones relating to contract variation and termination, fraud, confidentiality and conflict of interest.

The performance regime and reporting

2.8 The HSP performance regime centres on an abatement regime and nine Key Performance Indicators which are discussed in the next Chapter.13 Service providers must submit an annual business plan and six-monthly reports for departmental approval (details in Table 2.4).

Payment provisions

2.9 The HSP Payment Schedule has two categories of payments:

  • Monthly Operating Charge (MOC) — a fixed amount payable each month to cover direct and indirect costs to maintain the base infrastructure14; and
  • Settlement Service Charge (SSC) — there are 71 SSCs (or payment points) which have prescribed prices for a completed service (for example, orientation services, supporting a client to manage health appointments, or reimbursement of a cost).

2.10 Many of the SSCs are client-specific (for example, completion of a case management plan) and others are for the client group (for example, provision of a mobile phone for a family). Some services are capped based on: a price cap linked to the number of clients in the group (for example, Basic Household Goods) or a cap on the number of times a service can be claimed before departmental approval is required (for example, health appointments).

2.11 The payment levels for most of the SSCs vary between providers, reflecting the outcome of the tendering process.15

Service-related payment points

2.12 A June 2016 DSS financial compliance review of the HSS recommended more payment points to increase transparency and minimise compliance risk. DSS agreed with this and the 35 payment points in the HSS contracts were increased to 71 SSCs in the HSP contracts.

2.13 DSS also stated to the ANAO that the additional payment points can better reflect the costs when a client moved between regions. Claim points were intended to align with outcomes and to provide data on the cost and timeliness of each component service.

2.14 As discussed in paragraphs 2.52–2.56, the large number of payment points and some caps on the number of times a service can be provided (without separate approval) were reviewed after service provider input. There is a recommendation in a draft departmental report reviewing the operation of the first 18 months of the HSP to increase the number of times some services can be claimed before the cap is reached.16 Home Affairs also plans to explore consolidation of payment points (refer to paragraph 3.46).

Pricing considerations

2.15 ANAO review of the tender and contract pricing considerations showed:

  • DSS provided guidance to tenderers to help them establish their costings; and
  • the contract pricing for some regions was below the DSS price assumption.
Guidance on tender pricing

2.16 DSS provided guidance to tenderers on pricing, including clarification of what the MOC was to cover in May 2017. The MOC was principally to capture costs that would enable the tenderer to maintain a minimum capacity given uncertain client numbers.17Figure 2.1 shows the variability in the arrivals for the HSP and prior programs. Despite the MOC, this variability was raised as an issue by one service provider (refer to paragraph 2.61).

Figure 2.1: The pattern of arrivals of humanitarian entrants by month

 

Source: ANAO analysis of DSS data.

2.17 DSS also provided guidance to tenderers on the other pricing components. A May 2017 Tenderer’s Pricing Response Schedule had instructions for tenderers to complete an embedded Pricing Schedule for all program costs.

2.18 Clarifications on pricing, including administration costs, were provided in three question and answer documents for tenderers issued in March and April 2017. For instance, one answer stated that the $150 allowance for a mobile phone ‘could be for the phone, SIM, credit, and any other associated costs’.

Overall contract pricing

2.19 To review tender pricing, DSS developed price assumptions for each contract region for all payment components. DSS ranked tenders according to seven evaluation criteria (capability, organisation capacity, service delivery capacity, Indigenous Participation Plan, pricing, financial viability / capacity and risk).

2.20 DSS compared prices in each tender against its price assumptions. The preferred tenderers then participated in contract negotiations in June 2017. DSS sought revised tender prices where tender prices were above its price assumptions. Over the five year life of the contract, there was a 24.6 per cent (average) reduction between the original tender pricing and the final contract pricing across all contract regions.

2.21 Prior to the contract negotiations, DSS noted that the pricing for one preferred tenderer (Service Provider K) was low and testing of its service delivery assumptions was required.18 Service Provider K provided an overview of its model for case management and orientation costs and pricing was adjusted as a result.

2.22 Although Service Provider K’s overall tender prices were below the DSS final price assumptions for each region it tendered for, prices in seven priority areas were higher than DSS expected. The outcome of the price negotiation was that the final contract price for each of Service Provider K’s regions was lower than its original tender price. Three other service providers had a region where the final contract price was below the DSS price assumption.

Are the contracts well managed?

DSS’ management of the HSP contracts has been partially effective. DSS had appropriate governance arrangements for the program, including strong engagement with and examination of issues raised by service providers. However, an abatement regime in the contracts has not commenced, contract management plans were not established, and assurance activities since July 2018 have only examined a small percentage of clients through desk based reviews. Home Affairs is finalising its HSP governance arrangements and plans to commence a new assurance and compliance strategy for the HSP at the start of 2020.

2.23 To examine how well the HSP contracts were managed five areas were examined:

  • program governance;
  • contract management, including the abatement regime;
  • assurance and compliance activities;
  • service provider reporting; and
  • engagement with service providers.19

Program governance

2.24 DSS’ program governance of the HSP has been appropriate. DSS had a range of HSP oversight and advisory committees for management of the HSP, including to manage the transition to the HSP.

2.25 After the HSP Transformation Board (comprising senior executive staff) ceased in June 2018, DSS planned that a HSP Board would commence in May 2019. With the transfer of the program to Home Affairs, this did not occur. Home Affairs stated that it is reviewing the governance structures for the HSP with a view of establishing a Settlement Board to look at settlement in a wider context than just the HSP.

2.26 There are two advisory groups for the HSP. The Senior Officials Settlement Outcomes Group, comprising senior settlement officials in all levels of government, meets bi-annually to consider settlement data and programs. It is now chaired by Home Affairs. There is also a Settlement Services Advisory Council appointed by the Minister. It will cease in late 2019, with a new advisory council to be agreed by the Prime Minister.

2.27 In DSS, HSP operations were delivered by the DSS Community Grants Hub regional network. Policy and other coordination activities were managed by the national office. In Home Affairs, the HSP contracts as well as associated policy and coordination activities are managed by the National Office.

HSP contract management

Contract management plan

2.28 DSS’ Contract Management Guide 2017 lists contract start-up activities and states that the development of a contract management plan would ‘usually be required’. The July 2019 Australian Government Contract Management Guide (released post commencement of the HSP) requires contract management plans for programs like the HSP.20

2.29 DSS did not use a contract management plan to manage the HSP contracts.21 As evidence of contract management, DSS provided a PDM ‘Manage Function’ diagram for the Community Grants Hub (responsible for the HSP). This has reference to management functions including planning and assurance. It notes roles of the Service Assurance Centre for Excellence, including HSP service cap increases.

2.30 The detail on how the HSP was managed in practice is not evident from this diagram, including how claims for payment were assessed. Home Affairs should adopt the Australian Government Contract Management Guide in managing the HSP.

Recommendation no.1

2.31 Home Affairs should finalise changes to contract management arrangements for the HSP to ensure full alignment with the Australian Government Contract Management Guide.

Department of Home Affairs response: Agreed.

2.32 Contract management activities for the HSP were recently centralised within the Department. This streamlined administrative processes and provided greater consistency in approach. The Department is already well advanced in finalising individual contract management plans for each provider, which once in place, will mean that the contractual arrangements are fully compliant with the Australian Government Contract Management Guide.

The abatement regime

2.33 The HSP contracts contain an abatement regime where service failures incur ‘failure points’ which can lead to a reduction in the Monthly Operating Charge.22 The contracts state that the abatement regime would be inactive for the first 12 months of the HSP, from October 2017 to October 2018.

2.34 The abatement regime has not been activated. Home Affairs stated that it will review the abatement regime during the KPI transition period up to the end of 2019 (see Chapter 3, paragraph 3.29).

HSP expenditure

2.35 ANAO analysed data in the HSP system on expenditure since the program commenced in October 2017 to 30 June 2019.

2.36 The ANAO analysis shows that expenditure under all contracts in 2017–18 was less than anticipated at the start of the contracts (due to the fact that the contracts only ran for eight months in that year and the budget was for a full year). In 2018–19 overall HSP expenditure was 1.8 per cent below the cost estimated for that year at program commencement.

2.37 Over the life of the HSP the largest expenditure line item is the Basic Household Goods packages (further discussed at paragraph 3.55). Development of case management plans, assisting with health appointments and accommodation costs are the next highest categories.

2.38 The analysis also looked at average payments per client per tier (excluding payments made to a family group such as the Basic Household Goods package). The greatest overall expenditure is for tier two clients (93.5 per cent of clients) but the highest average expenditure per person is for tier three clients who have the highest need levels. The analysis showed that the average spend per client was around $1,300 for tier one, $2,000 for tier two and $3,000 for tier three.

Assurance and compliance activities

2.39 Compliance and assurance activities for the HSP have been undertaken in three phases23:

  • transition-in monitoring (October 2017 — June 2018);
  • interim monitoring (from July 2018); and
  • a new Assurance and Compliance Strategy (planned to commence at the start of 2020).
Transition-in monitoring

2.40 The transition-in monitoring consisted of case management plan reviews, client interviews, and observations, including client arrivals to verify the required on-arrival processes. A total of 88 transition-in monitoring activities were undertaken, for the 7,579 clients who transitioned in to the HSP and arrived in this period. This is on the low side for a sample size.24 Client interviews have not been conducted since July 2018.

Interim monitoring

2.41 The interim monitoring involved a review of 195 client case management plans and invoices (1.7 per cent of the 11,317 HSP clients who arrived in this period). DSS based the sample size on the number of HSP clients and the assessed risk level in each region (not statistical significance). A January 2018 internal report recommended statistically valid sampling with around 1000 reviews per year for a confidence level of 90 per cent (around five times the number of reviews that were conducted).

2.42 Table 2.3 shows the average result data in the May 2019 Monitoring Summary Report.

Table 2.3: Summary of HSP compliance results, July 2018 to April 2019

Activity

Average

Client Management Plan (CMP) developed for each HSP client

98%

Client was involved in the development of the CMP

67%

CMP developed within the required time (three weeks for tier one and two, five working days for tier three)

86%

CMP included the required elements

67%

Orientation level/s claimed with sufficient supporting evidence

87%

Education, English and Employment claims made with sufficient supporting evidence

87%

Housing services claimed with sufficient supporting evidence

70%

  

Source: ANAO summary of DSS data.

2.43 All of the elements in Table 2.3 are mandatory requirements in the HSP contracts. While nearly all providers were creating case management plans for HSP clients, some were not developed in a timely manner and did not contain the required elements.25 The DSS Monitoring Summary Report noted that the findings may be affected by factors including an inconsistency between assessors as to what constituted sufficient evidence and the low numbers of desktop reviews.26

2.44 The report recommended enhanced evidence to support claims and staff training towards more consistent desktop reviews. DSS had provided desktop review training to over 20 delivery network staff in March 2019 informed by findings from recent reviews, including on consistency.

A new Assurance and Compliance Strategy

2.45 DSS commissioned a review of the quality assurance process to inform development of a potential new HSP Assurance and Compliance Strategy in May 2019.

2.46 A July 2019 draft of the Strategy included:

  • a client interview for verification and feedback on the provision of a range of immediate settlement services three months after the client’s commencement in the HSP;
  • a follow-up client interview to seek client verification and feedback on the provision of a range of settlement services six to seven months after their commencement in the HSP; and
  • a desktop review based on case management plans and other documentary evidence uploaded by service providers to the HSP IT system to support HSP claims.

2.47 Home Affairs stated that it plans to finalise and implement the new strategy at the start of 2020 and estimates that a sample size of 787 clients per annum will provide a statistically valid sample for the future performance monitoring processes.

Reporting by service providers

2.48 HSP service providers have submitted the majority of the reports required under the contracts, except one provider has not reported on Indigenous participation and there is no evidence of reporting on compliance with the Workplace Gender Equality Act 2012 (see Table 2.4). Home Affairs could consider offering some guidance to staff on assessing the business plans and six-monthly reports.

Table 2.4: HSP reporting requirements

Report

Has it been submitted?

ANAO comment

Annual Business Plan

 

The service providers have submitted Annual Business Plans for 2017–18, 2018–19 and 2019–20. The HSP contract stipulates that this milestone payment is made once the department approves the report. There are no guidelines for staff on how to assess these reports.

Six Monthly Reports

 

The service providers have submitted all required six-monthly reports up to the end of June 2019. The HSP contract stipulates that this milestone payment is made once the department approves the report. There is no guidance on how the reports should be assessed.

Indigenous Participation Plan (IPP)

Partially

The HSP contract requires service providers to submit a quarterly report on its compliance with the IPP. As at July 2019, no standalone reports had been submitted by any provider although one provider commented on this in the six-monthly report. Home Affairs stated that service providers are required to report this to the National Indigenous Australians Agency. As at July 2019, four Service Providers had submitted all required reports. Home Affairs is working with the remaining provider to update reporting.

Workplace Gender Equality (WGE) Act

No

The HSP contract requires service providers to submit a report 18 months after the contracts commenced on compliance with the Act, and then annually. Home Affairs stated it will require service providers to submit this information with their annual business plan. The 2019–20 Business Plans that were provided to the ANAO do not include this.

Fraud Control Plan

N/A

A Fraud Control Plan is to be produced and submitted to the department by providers on request. At July 2019 a request had not been made.

Critical Incident Reporting

N/A

HSP service providers must report critical incidents to the department (within 24 hours). The department maintains a register of incidents and compiles six-monthly summary reports. No critical incidents were reported in the first 18 months of the program.

   

Source: ANAO analysis of DSS records.

2.49 The contracts require that the six-monthly reports and business plans are assessed and approved by the department before related payments are made to service providers. DSS produced an overall assessment of providers’ six-monthly reports for the period ending 31 December 2018 (see paragraph 3.47). This stated that the reports were found to be ‘comprehensive’.

2.50 Home Affairs has received the six-monthly reports for 1 January to 30 June 2019 and stated that it is summarising these to identify trends and strategies to improve service delivery.27

Engagement with service providers

2.51 DSS established appropriate arrangements for engagement with service providers (see Table 2.5). Service providers considered that DSS’ relationship management was a key strength of the HSP. Home Affairs stated that the DSS engagement arrangements will largely continue under its administration of the HSP.

Table 2.5: Engagement with HSP service providers

Title

Who attends?

Timing

ANAO comment

HSP Provider Advisory Group

Executive staff from all service providers and Home Affairs

3 times per year

Attendance is required under the HSP contract. This group met three times since July 2017. Home Affairs stated that it will continue to meet while the program is administered by Home Affairs.

HSP Community of Practice (COP)

Service providers

Every two months

The COP first met in June 2018 to address key areas of importance. Home Affairs stated that it will continue to meet under Home Affairs.

HSP working group

HSP service providers and Home Affairs

Quarterly from July 2019

This reports to the Provider Advisory Group. This group is continuing under Home Affairs and prior to July 2019 it had met four times.

Orientation Conference

Service providers and Home Affairs

Once per year

Attendance is required under the HSP contract. Home Affairs held a HSP workshop with service providers in November 2019 which considered orientation.

    

Source: ANAO analysis of DSS meeting records and Home Affairs advice.

Issues raised by HSP service providers

2.52 Key issues identified by service providers were communicated to DSS in a 2018 ‘Community of Practice’ issues paper. The issues paper made 16 recommendations. IT system issues were noted. Service providers said that the number of Settlement Service Charge points had created an administrative burden that they did not anticipate.

2.53 The issues paper stated that service caps and the process for requesting increases was overshadowing the intended move towards outcomes instead of outputs.28 It also stated:

The tension point between changing from outputs to outcomes requires some inbuilt flexibility within the program to enable Service Providers to make some judgement calls that reflect individualisation of services to clients rather than strictly adhering to service line items.

2.54 The Settlement Council of Australia (a peak body for settlement service providers with members including the HSP service providers) surveyed its members and produced a HSP Online Survey Report in October 2018. The report was submitted to DSS and noted an over-reliance on outputs (attached to the payment model) detracts from outcomes.

2.55 The Community of Practice issues paper stated that clients in crisis (including those experiencing reportable critical incidents such as family violence) can require extensive case management and administration, ‘a large portion of which is non-claimable.’ In February 2019 DSS and service providers noted that this issue mainly affected tier three clients.

2.56 Many of the issues raised in the Community of Practice paper were noted in DSS’ reviews of the program and in planned responses to these (see paragraph 3.46). DSS noted that an external evaluation of the program design and outcomes is planned in 2020–21. Home Affairs is planning to consider a number of potential changes to HSP management, further discussed at paragraph 3.45.

HSP service provider financial viability

2.57 In July 2018, Service Provider K raised concerns to DSS about additional (unfunded) costs across its contract regions, noting a higher than expected administrative burden. DSS and Home Affairs have considered the Service Provider K viability issues in detail.

2.58 A January 2019 DSS briefing to the Minister sought approval to increase Service Provider K’s payment points under the contract. The briefing noted that:

During HSP contract negotiations, despite attempts by the Department to encourage Service Provider K to reconsider and appropriately cost its price points, Service Provider K assured the Department that it could deliver services for the tendered prices. Unfortunately, this is now resulting in viability issues for a number of Service Provider K’s services and their sub-contractors.

2.59 The briefing did not state that Service Provider K’s contract pricing was lower than the DSS price assumptions, but said that its average cost per client was below the national average.

2.60 The Minister did not approve the proposed increase in payments to Service Provider K, stating that more needed to be understood about the issues before releasing additional funds to an organisation that tendered a contract price.

2.61 A February 2019 briefing to the Minister proposed five options on additional funding (over three months) to Service Provider K and its subcontractors. On 6 March 2019 the Minister approved one-off funding over three months from March to May to Service Provider K’s sub-contractors. The briefing included advice from a Service Provider K subcontractor on the impact of high rents and uncertain arrivals in a particular region. It noted that DSS would respond to this by ensuring a more sustainable and consistent flow of clients to that region. Home Affairs stated that it anticipates a smoother rate of referrals to that region throughout 2019–20.

2.62 An April 2019 consultant report to DSS on subcontracting focussed on Service Provider K and concluded that the majority of subcontractors who provided data were running at a loss for the HSP.

2.63 Home Affairs’ September 2019 HSP risk assessment noted the March 2019 payment to Provider K’s subcontractors (for a three month period) and Home Affairs plan to manage the viability risk by the potential consolidation of payment points and the HSP recalibration project (see paragraph 3.46). Home Affairs stated that it continues to work closely with the Service Provider K on contract deliverables and financial viability.

Does the HSP IT system support contract management and the delivery of the program?

The HSP IT system partially supports contract management and the delivery of the HSP. While the HSP system manages payments effectively, not all data (such as health information) has been migrating correctly from external systems, adding risk to the delivery of essential services. Reporting against key performance indicators and on client outcomes was delayed partly due to IT system issues. The increased administrative burden of using the HSP IT system from October 2017 to April 2019 led to DSS making additional payments to service providers. Over 100 system enhancements and fixes have been implemented since the system went live which largely resolved the administrative burden issue. Earlier user acceptance testing may have assisted more timely identification and mitigation of these issues. DSS had spent $16.2 million on the HSP IT system as at 30 June 2019, which is $3 million more than the original estimate. Some originally planned functionality has still not been implemented. Home Affairs and DSS are developing a memorandum of understanding regarding future management of the HSP IT System, including establishing priority system enhancements.

Key features of the HSP IT system

2.64 The HSP IT system (HSP system) was developed to facilitate client referrals, travel, and service management (including claims submitted by providers and tracking of client outcomes). As at 8 November 2019, the HSP system had processed and paid over 769,489 claims across 36,950 clients, and has over 800 users across Home Affairs and the five HSP provider organisations.29

2.65 DSS relied on Home Affairs’ Humanitarian Entrants Management System (HEMS) to manage the HSS programs since 2013. A June 2015 evaluation of HSS and CCS noted that HEMS had ‘limited case management functionality’ leading to ‘duplication and lack of visibility of client information and analytical capability.’ It noted that an improved HSP system could lead to:

  • decreased burden on providers by having one system to replace both the Home Affairs HEMS system and providers’ own case management systems (this would also reduce the need for provider reporting, as data would be more accessible);
  • efficiencies in departmental processes by allowing easier reporting, better quality assurance activities, and easier invoicing; and
  • the collection of more robust client information.

2.66 The evaluation’s recommendation to review the IT system was agreed by DSS. DSS also noted two additional reasons for a system redesign, including: issues with ongoing operational stability as the existing systems were not scalable, flexible, or sustainable; and, a high risk of unrecoverable failure and limited ongoing technical support or capacity for enhancements.

2.67 While the new HSP system has successfully processed payments across a large number of clients, the other benefits of developing a new HSP system have not been realised.

Developing the HSP IT system

2.68 DSS finalised a Project Management Plan to support the implementation of the new IT system in February 2017, before the commencement of the HSP program in October 2017.

2.69 The HSP system functionality was to be delivered in two phases, as summarised in Table 2.6, at a total cost of $13.2 million. The system was built in-house by the Information Technology and Communications Group (ITCG) on one of DSS’ central IT platforms (Siebel).

Table 2.6: HSP system phased functionality and cost estimates

Phase

Functionality

Cost

Delivery deadline

Phase one

Basic case management functionality

Contract management

Client referrals, claims and payments

Migration of historical data and a live data exchange of HEMS client pre-arrival information

Monitoring and reporting

$6.5m

July 2017

Phase two

Enhance phase one functionality

Automation of some functions

Enhance analytical capability

$6.7m

June 2018

Total

 

$13.2m

 

    

Source: DSS HSP Project Management Plan, February 2017.

Implementation of the HSP system

Governance

2.70 DSS established or used existing governance boards to oversee and manage the implementation of the HSP system. This included the HSP Transformation Board which managed the transition from the HSS and CCS to the HSP, and the HSP IT Project Board which supported the delivery of IT system enhancements and fixes after the HSP Transformation Board was disbanded. The boards met regularly in line with established terms of reference and maintained records. The boards considered achievements, issues, decisions, priorities, and risks.

2.71 DSS established guidance for HSP system users, including regularly updated system manuals and task cards, monthly communiques, and a GovDex site for users, which housed the guidance.

Adjustment of the Minimum Viable Product

2.72 In March 2017, DSS received legal advice signing-off on the release of the request for tender. The legal advice noted that delays to the delivery of the HSP system risked the department’s ability to meet its contractual obligations. In the same month the HSP Transformation Board agreed:

  • that the system would go live on 30 October 2017, in line with contract commencement; and
  • to a Minimum Viable Product (MVP) — the minimum system functionality needed to support the HSP.

2.73 Delivery of the MVP by 30 October 2017 was again confirmed at the HSP Transformation Board meeting on 26 July 2017.

2.74 On 6 September 2017 the HSP Transformation Board was advised by the program manager that the MVP could not be delivered by the end of October and a ‘revised and reduced scope’ MVP had been developed in order to deliver a system on time. The HSP Transformation Board was informed that the removed functionality would now be delivered in phase 2. The delivery risk was upgraded from amber to red and the reduced scope HSP system was delivered on 30 October 2017.

2.75 The delayed functionality and a description of its purpose, the proposed work around and the current implementation status is outlined in the Table 2.7.

Table 2.7: Delayed HSP system functionality and current status

Function name

Description

Proposed work around

Current status at time of publication

Tier Three Referrals

Automating referral information for clients referred to HSP through channels other than HEMS.

Manually entering client data and using non-purpose built functionality.

Half full Harvey ball

Functionality was implemented in March 2019 however Home Affairs is unable to access the fields for reporting purposes.

Client reporting

The ability for providers to report client outcomes through the HSP system including at formal reviews and exit reviews. These reports track client outcomes in relation to health, housing, family functioning, education and employment.

Manual forms.

Half full Harvey ball

Implemented in February 2019, and manual reporting was used in the interim. Home Affairs is only able to access client reporting from system launch to February 2019 by looking at the forms attached to client files.

Additionally, while client outcomes can now be entered in to the system, Home Affairs cannot access the data for reporting purposes. No release date or cost estimate have been determined for reporting.

The Australian Cultural Orientation (AUSCO) Program

A data exchange for information on whether the client had attended off-shore orientation services with AUSCO.

Requiring providers to ask the client.

Empty Harvey ball

AUSCO data exchange has not been implemented and no release date for a fix or cost estimate has been determined.

Critical incidenta reporting

The ability for providers to report critical incidents through the HSP system.

Manual forms.

Half full Harvey ball

Critical incident reporting can occur through the HSP system using manual forms. Implementing the full functionality was estimated to cost $250,000, but was removed from the list of fixes in January 2019 because of this work around.b

Key:

Empty Harvey ball Not implemented

Half full Harvey ball Partially implemented 

    

Note a: The HSP Case Management Guidelines define a critical incident, discussed further in Chapter 3.

Note b: Implementing the full critical incident reporting functionality was also recommended in a 2018 review and was estimated to cost $250,000.

Source: ANAO analysis.

Key HSP system issues

2.76 There have been ongoing issues impacting the effectiveness of the HSP system. These have included: an increased administrative burden on providers and limited user acceptance testing; issues with the data exchange with external systems; and the system’s ability to monitor key performance indicators. These issues relate to the proposed benefits of the system.

Administrative burden

2.77 The HSP system was proposed partly to reduce the administrative burden on providers, to replace providers’ own systems, and to reduce duplication of effort. These benefits have not been fully realised.

2.78 Providers reported various issues with the HSP system in mid-2018.30 Two issues related to the functionality that was removed when DSS agreed to a revised MVP, including:

  • manually reporting DEX scores; and
  • manually reporting critical incidents.

2.79 Providers continued to use their own case management solutions alongside the HSP system due to the limited HSP system functionality. They advised DSS that this was leading to duplication.

2.80 DSS acknowledged that the functionality of the HSP system was resulting in a ‘considerable additional administrative burden for providers’. In January 2019 the Minister for Families and Social Services approved a one-off administrative payment to service providers. The amount was to be based on an additional four hours of administrative work per client, and providers were advised that DSS ‘expected’ providers to pass a proportion of the funding to sub-contractors. The payment totalled $1.95 million for the period 30 October 2017 (HSP commencement) to April 2019.

2.81 User acceptance testing (UAT) is used to ensure an IT solution is working well. It allows users to access the solution and provide feedback before the system is finalised. Providers, the majority of HSP system users, participated in UAT on 9–10 October 2017, however there was insufficient time available to address the concerns raised before the system commenced operation on 31 October 2017.

2.82 A second round of UAT with the providers occurred in November 2018, over 12 months after the HSP commenced. Providers completed acceptance agreements for the December 2018, January 2019, and February 2019 updates to the HSP system. The enhancements and fixes in these releases were largely to resolve the administrative burden issues before the end of the ‘one off administrative payment’ period. DSS surveyed users in the five provider organisations after these releases and found a reduction in administrative burden for most survey participants across seven different system functions.31

Data exchange between HEMS and the HSP system

2.83 Replacing HEMS and improving the stability of the system supporting settlements was identified as a reason to develop the HSP system. This benefit has not been fully realised. Establishing a data exchange was also identified as a risk and was monitored throughout implementation by the HSP Transformation Board. The migration of historical HEMS data into the HSP system and the ongoing exchange of live data was implemented at commencement of the new IT system on 30 October 2017.

2.84 DSS has experienced issues with the data exchange which impacted the availability of information, including:

  • critical health information not transferring to the HSP system;
  • visa classification types listed in the HEMS system changing when imported to the HSP system;
  • clients not being created in HSP system automatically from HEMS data; and
  • personal information such as languages spoken, dates of birth, or arrival dates not transferring to the HSP system.

2.85 Home Affairs data is sent through to DSS via spreadsheets which were then loaded in to the HSP system. DSS had established protocols for actioning errors in processing the data. Providers are continuing to liaise with Home Affairs to investigate missing or incorrect data in the system.

2.86 DSS and Home Affairs have resolved a number of data exchange issues, but have also estimated it would cost $2.7 million to implement a web services data exchange to resolve multiple data issues. The planned benefit of the new system to reduce risk to operational stability was not fully realised.

Monitoring of key performance indicators

2.87 The collection of more robust client information and easier reporting were identified as a benefit of developing the HSP system. DSS uses a software program called Qlik Sense to access and visualise data from DSS’ central Siebel system, which is where HSP is housed. DSS stated to the ANAO that it had the potential to use the Qlik Sense software to access and visualise data from the HSP since implementation.32 Home Affairs began developing reports in Qlik Sense only in July 2019 (refer to Chapter 3 on key performance indicators). These were previously delayed due to system issues.

HSP system enhancements and fixes

2.88 DSS spent $141,382 on planning and scoping the HSP system. It estimated that the development and implementation would cost $13.2 million and be delivered by June 2018. DSS expended a total of $16.2 million on the HSP as at 30 June 2019 with some key functionality still missing. At September 2019, there have been 123 enhancements and fixes to the HSP system.

2.89 The system enhancements and fixes were identified via feedback from DSS staff, other user groups, including providers, and a 2018 review of the HSP. Early enhancements and fixes in 2018 related to issues such as user interface design. Later fixes were more substantive such as for tier three referrals.

2.90 As of September 2019, DSS had identified 105 further enhancements and fixes, including eight critical and 23 high priority. Release dates have been estimated for one critical and two medium priority.

2.91 The settlement services function was transferred to Home Affairs in July 2019. Home Affairs has signed a memorandum of understanding with DSS which includes responsibilities for managing the HSP System. Home Affairs and DSS are also establishing priority system enhancements and a release schedule.

Recommendation no.2

2.92 That Home Affairs (in consultation with DSS) continue to prioritise identifying, planning for and resolving HSP IT system issues that are either causing risks to program delivery, or impacting Home Affairs’ ability to manage program performance.

Department of Home Affairs response: Agreed.

2.93 The Department has identified a range of system enhancements, building on system improvements already implemented. We will continue to work closely with DSS on implementation of these enhancements, which will further mitigate risks and improve performance management.

3. Management of HSP performance

Areas examined

This chapter examines whether the performance of the Humanitarian Settlement Program (HSP) has been effectively managed to achieve overall program level outcomes.

Conclusion

The departments have been partially effective in managing HSP performance to achieve program outcomes. While HSP risk management processes have been appropriate, performance management and reporting has not been appropriate and available data does not allow the departments to determine if the objective of the HSP is being met. Despite some improvements having been made to the program, with more planned, not all intended benefits of the new HSP delivery model have been achieved.

Areas for improvement

The ANAO made one recommendation aimed at improving HSP performance information.

3.1 In order to assess the overall program performance of the HSP, the ANAO examined:

  • program risk management processes — because effective management of risks supports program planning and delivery;
  • performance measurement and reporting — because complete and accurate performance information is needed to determine whether objectives are being met; and
  • whether continuous improvements are made to the program — because the HSP was intended to improve settlement services compared to previous similar programs and the program has been reviewed a number of times since commencement.33

Is there an appropriate HSP risk management process?

The departments have employed an appropriate process for HSP risk management. DSS’ and Home Affairs’ risk management activities for the HSP comply with the Commonwealth Risk Management Policy. Consideration of risk is evident in key program documents and four risk assessments have been conducted. Mitigation strategies were updated as the program continued and Home Affairs is reviewing its risk appetite following transfer of the HSP. In the September 2019 risk assessment, the post treatment risk rating is low for four risks and medium for two risks.

Compliance with the Commonwealth Risk Management Policy

3.2 The Commonwealth Risk Management Policy has nine elements and requires entities to establish and maintain risk management controls.34 While the elements take an entity-wide view, ANAO assessed DSS’ risk management for the HSP and found it aligned with the policy.35 Home Affairs’ overall approach to risk management is also consistent with the policy and it is updating the risk documentation for the HSP in line with its Risk Framework. The following sections cover key elements of HSP risk management.

The initial approach to program risk management

3.3 Reports were made to the HSP Transformation Board on the IT system and program transition risks before and after the program commenced in October 2017.36

3.4 The risk management section of the HSP contracts says that service providers must advise the department about any contract performance issues, report critical incidents and outline risks and mitigation strategies in their annual business plans. Program risks are also communicated to the department in the service providers’ six-monthly reports.

Risk management after the HSP contracts commenced

3.5 The DSS Contract Management Guide required a finalised risk management plan at commencement of a contract. DSS stated that this risk assessment was commenced in February 2017. The contracts commenced in October 2017 but the risk assessment was not finalised until April 2018. Table 3.1 shows that at four points, risk-related documents were prepared (or drafted) and risk assessments were conducted (equivalent to a risk management plan).

Table 3.1: Risk management documents and assessments

Risk document

Date of document preparation

Risk Assessment

Date of risk assessment

DSS HSP Risk Management Guidelines — has categories of program risk and risk management responsibilities.

March 2018

DSS HSP risk assessment (which included the DSS required content for a risk management plan).

Approved in April 2018

Ernst and Young (EY) report to DSS HSP Business Improvement Future State.

September 2018

This EY Report for DSS included a HSP risk assessment and proposed risk improvements.

September 2018

DSS HSP Risk Management Plan — references the Enterprise Risk Management Framework.

March 2019

 

The February 2019 DSS risk assessment used the EY report and updated treatments.

February 2019

Home Affairs HSP Risk Management Plan Home Affairs is developing this.

Not yet finalised

Home Affairs updated the HSP risk assessment.

September 2019

    

Source: ANAO analysis of DSS and Home Affairs documents.

DSS April 2018 HSP risk assessment

3.6 The April 2018 risk assessment listed nine risks, sources for each risk, current controls, a control effectiveness rating, and risk ratings before and after planned treatments. The risk assessment included the service delivery risks identified in the DSS Contract Management Guide (except for fraud).37

3.7 Sources of risk for ‘service provider failure to deliver services’ included service provider or sub-contractor insolvency and sub-contractors having insufficient resources.38

3.8 The initial risk rating for the HSP IT system was ‘high’ but the target was ‘medium’ given planned treatments (note Table 2.7 on the HSP IT system).

DSS February 2019 HSP risk assessment

3.9 DSS updated the risk assessment in February 2019 to take into account September 2018 risk management recommendations in a consultant report which also identified fraud as an additional risk.39 Treatments were also updated.

3.10 A March 2019 HSP Risk Management Plan stated that high and extreme risks were generally considered unacceptable but could be accepted in some circumstances.40 In the February 2019 risk assessment two risks remained high post treatment: client settlement outcomes not being achieved (Risk 1) and compromised well-being of client (Risk 4).

3.11 Given the Service Provider K financial viability issues, a new source of risk was added: ‘service provider or subcontractor withdrawal’. A February 2019 minute to the Group Manager for the HSP noted the high post treatment rating for Risk 1. It said that the proposed treatments (especially for Service Provider K) were temporary and longer-term solutions were required.41 It was noted that Risk 4 was rated high as the consequence will always be severe.

3.12 In line with the HSP Risk Management Plan, the medium risks were reported to the Branch Manager Settlement Support and the high risks were reported to the Chief Risk Officer.

Home Affairs September 2019 HSP risk assessment

3.13 Home Affairs prepared a HSP Strategic Risk Assessment in September 2019. The risk categories reflected those in the DSS February 2019 assessment. For the six risks, there was an initial risk rating, treatments (with owners) and a target risk rating post treatment (see Table 3.2).

3.14 The post treatment risk was medium for two risks and low for four risks (see Table 3.2). The Home Affairs Risk Management Policy states that Home Affairs has a low appetite for risk to the delivery of refugee and humanitarian programs but this predates the transfer of the HSP to Home Affairs. HSP risk is to be reviewed in line with the Department’s Risk Framework.

Table 3.2: Risk ratings September 2019 risk assessmenta

Risk

Initial risk rating

Post treatment risk rating

Risk 1: Client settlement outcomes not being achieved

High

Medium

Risk 2: HSP enablers do not support program deliveryb

High

Low

Risk 3: HSP is not well governed

Medium

Low

Risk 4: Compromised well-being of client as a result of the HSP

Medium

Medium

Risk 5: Personal information of clients / employees is compromised

Low

Low

Risk 6: Service providers make false or exaggerated claimsc

Medium

Low

   

Note a: Colour coding is based on Home Affairs colours for the risk level — there are no severe (red) risk ratings.

Note b: HSP IT system, systems, processes and policy etc. The post treatment risk rating for HSP enablers (principally related to the HSP IT system) fell from medium in February 2019 to low. A Memorandum of Understanding between DSS and Home Affairs on the HSP IT system (under negotiation) was noted as a treatment.

Source: Home Affairs HSP Risk Assessment September 2019.

3.15 Home Affairs has stated that it will establish a Settlement Board (as an alternative to the HSP Board that DSS planned) to look at overall settlement issues, including the HSP. Home Affairs stated that, in the interim, the September 2019 risk assessment was reported to the Assistant Secretary and First Assistant Secretary. It was also reported to the Audit Committee in October 2019.

Client settlement outcomes risk

3.16 The post treatment risk for client settlement outcomes not being achieved (Risk 1) was high in the February 2019 risk assessment. In the September 2019 risk assessment, the initial high risk rating for this risk dropped to medium as a result of the planned treatments.

3.17 These treatments for Risk 1 included the March 2019 one-off payment to Service Provider K (which was for three months). As noted in paragraph 2.63, treatments still to be potentially implemented for this risk (which are relevant to Service Provider K) include consolidation of payment points and additional payments for tier three clients.

Client well-being risk

3.18 In Home Affairs September 2019 risk assessment, the post treatment risk rating for ‘compromised well-being of client’ was medium, although it had been high in the February 2019 risk assessment. This change was a result of a clarification of the definition of critical incidents, which are used to monitor this risk. In December 2018 the critical incident definition was updated to specify that reporting of critical incidents should only occur when they are ‘as a result of the HSP service’.

3.19 The half-yearly HSP Incident reportsstate that, in addition to reporting critical incidents, providers are encouraged to report non-critical incidents. No critical incidents (in line with the updated definition) were reported. The half-yearly reports note that there were 67 non-critical incidents between July and December 2018 and 72 between January and June 2019.42

Is program performance appropriately managed and reported?

HSP program performance has not been appropriately managed or reported, and the department was not able to determine if the objective of the HSP is being met. Client outcome data is not being tracked and Key Performance Indicators (KPIs) in the contracts have not been effectively implemented. A KPI ‘relief period’ has been in place since the program started, and non-binding reporting of a limited set of KPIs commenced in the July to September 2019 quarter. It showed that all three KPIs were met by only one of the five providers during the quarter. There has also been limited internal and external reporting on program performance, however DSS did prepare the first six-monthly report on the HSP for the period up to 31 December 2018 based on reports from the five service providers.

3.20 This section examines the overall program performance management and reporting, as opposed to the management of the specific contract issues covered in Chapter 2. The aggregated picture for the program was examined by reviewing:

  • client settlement outcomes;
  • key performance indicators; and
  • internal and external reporting on HSP performance.

3.21 Longitudinal scores on client attainment in areas including health and employment were recorded but not reviewed by the department, although this data could reveal program performance and potential areas for improvement. Data is only being collected on three of the nine KPIs in the contracts. Data on all contract KPIs could be aggregated to assess the overall program status but this is only partially possible given the use of only three KPIs and potential data issues. In line with the limited performance data, there has been limited internal reporting on program performance.

Client settlement outcomes

3.22 To track progress of client outcomes the HSP Case Management Guidelines include provisions on Data Exchange Reporting (DEX scores). DEX scores were designed to cover client outcomes across 13 outcome domains, such as health, housing, education and employment.

3.23 Under the contracts, service providers must assess and record DEX scores for each client at the start of the program, formal review points (varies according to client tier), and when exiting from the HSP.

3.24 Client outcome data DEX scores are not being tracked, as:

  • DSS scoped out the ability to record DEX scores from the HSP IT system Minimum Viable Product (October 2017) — DSS created spreadsheets for manually recording DEX scores;
  • DSS chose not to extract the data from the individual client spreadsheets and has not summarised or reported on the DEX scores for any HSP clients — Home Affairs stated that it will add a review of recording of DEX scores to the desk top reviews; and
  • a February 2019 IT system upgrade allowed service providers to report DEX scores in the HSP system. However, this data cannot be extracted from the HSP IT system (for instance, to provide data on the areas covered by the six inactive KPI — see Table 3.3).

Key performance indicators

3.25 The HSP contract lists nine key performance indicators (KPIs) which were due to commence with signing of the contracts. If a service provider fails to meet a KPI consistently over four quarters, the department may terminate, suspend or reduce the scope of the contract. Failure to meet KPIs is not linked to the abatement regime (discussed in paragraphs 2.33–2.34).

3.26 HSP IT system issues at the commencement of the HSP hampered KPI data collection. A planned alternative manual system for reporting KPIs and holding service providers accountable for the results was not implemented. Instead, in November 2017, DSS instituted a ‘KPI relief period’ during which service providers were exempt from meeting the KPIs. DSS advised the providers in May 2018 that the KPI relief period was being extended and that “Service Providers must continue to meet or exceed KPIs,” however they noted that formal monitoring had not commenced. The relief period was extended multiple times (Figure 3.1).

Figure 3.1: Approval and extensions of the KPI relief period, timeline

A timeline showing the key dates related to the key performance indicators (KPIs) for the Humanitarian Settlement Program (HSP). After the HSP started, a KPI relief period was approved to all service providers until 1 February 2018. The relief period was

Source: ANAO analysis of DSS documents.

3.27 DSS commenced a review of the KPIs in mid-2019, which is being continued by Home Affairs. Initial findings of the review found that six of the nine KPIs do not effectively measure the intended outcomes and were therefore considered not fit for purpose by DSS (see Table 3.3).

Table 3.3: Current status of the HSP contract key performance indicators

KPI reference

KPI description

Current status

1

70% of clients aged 15 years and over achieve Key Outcomes of HSP Orientation at the ‘knowledge’ level (as set out in the Orientation Facilitator’s Guide) within seven months of arrival to Australia.

Transitional reporting commenced 1 July 2019

2

60% of clients aged 15 years and over achieve Key Outcomes of HSP Orientation at the ‘application’ level (as set out in the Orientation Facilitator’s Guide) within twelve months of arrival to Australia.

Transitional reporting commenced 1 July 2019

3

Clients aged 18 or over are supported to attend English language lessons within six months of arrival to Australia.

Reporting not implemented

4

Clients (aged 18–65 years) are supported to enrol in education and training within twelve months of arrival to Australia.

Reporting not implemented

5

Clients (aged 18–65 years) are supported to engage in employment services within twelve months of arrival in Australia

Reporting not implemented

6

Clients have received all Settlement Services, identified as a need through their case management plan prior to exit.

Reporting not implemented

7

Tier one clients (aged 18–65 years who are seeking work and available for work) are employed at exit.

Reporting not implemented

8

Tier two clients (aged 18–65 years who are seeking work and available for work) are employed at exit

Reporting not implemented

9

95% of mandatory data is entered into the HSP system within 14 days of the delivery of a settlement service

Transitional reporting commenced 1 July 2019

   

Source: ANAO summary of DSS data.

3.28 Reporting was not implemented for KPIs 3 — 8 as shown in Table 3.3 above and DSS analysis was that:

  • the service was not required (KPIs 3, 4 and 5) — service providers advised DSS that clients engaged in education themselves and humanitarian entrants are now exempt from entering jobactive for 12 months;.
  • the data could not be accessed or monitored (KPI 6) — client needs and case management plans cannot be tracked in the HSP system;
  • two KPIs were not a contractual requirement or within service provider control (KPIs 7 and 8) — the service providers are not accountable for clients being employed although they are required to support clients to attend employment services.

3.29 Home Affairs implemented internal reporting against KPIs 1, 2 and 9 from 1 July 2019 to 1 January 2020 as part of a ‘transitional period’ to ‘test and refine the [KPI] reports’.43

3.30 If Home Affairs commences formal KPI monitoring from 1 January 2020 as intended, enforcement of any KPI failures through provider contracts would be possible from 1 January 2021.

Quality of the key performance indicators

3.31 Two of these KPIs relate to the orientation program which delivers ‘ten core settlement topics’.44 Clients develop comprehension at three progressive levels: awareness, knowledge and application. The orientation program is delivered to each HSP client aged 15 years and over if the case manager determines that the client needs this assistance.

3.32 Home Affairs stated that the targets of 70 per cent and 60 per cent account for clients who may not need this service, despite the department’s stated expectation that all eligible clients should participate in HSP orientation. However, if a client genuinely does not need the orientation service and it is therefore not delivered, this is counted as ‘not achieved’ by the HSP IT system. This could introduce a negative bias to the results.

3.33 Home Affairs stated that service providers must assess whether a client has achieved the outcome level. The Orientation Facilitators’ Guide says that a client meets the achievement standard for orientation if they demonstrate a majority of: attendance and participation; self-assessment of results; and an understanding of the topic and ability to apply it independently. Some providers stated to the ANAO that they have developed internal guidance to assist their staff to assess comprehension levels.45 Home Affairs stated to the ANAO that there is an opportunity to provide additional guidance to service providers on how to assess achievement of orientation levels.

3.34 Home Affairs also stated that there is quality assurance for orientation claims via the desk top reviews, but it intends to expand assurance activities to reach a higher volume of clients and include client interviews from the start of 2020 (refer to Chapter 2, paragraphs 2.45–2.47 on the new assurance and compliance strategy).

3.35 The KPIs in each contract only provide a partial picture of performance. The two orientation KPIs (KPIs 1 and 2 in Table 3.3) are relevant to the overall objective of the HSP but they do not provide the information necessary to assess whether or not clients have achieved the intermediary outcomes listed in the HSP Outcomes Framework.

3.36 Although the third transitional KPI (KPI 9 in Table 3.3) covers administration from the service providers’ side, there are no KPIs that consider the effectiveness of the delivery of the program by the department (for example, related to the accuracy of data in the HSP system and whether payments are made on time).

3.37 Potential improvements to KPIs are outlined at the end of this Chapter.

Reporting on performance

Reporting on key performance indicators

3.38 A dashboard was provided to the Home Affairs executive in October 2019. It included performance data for KPIs 1, 2, and 9, for each provider for the July to September 2019 quarter. Home Affairs stated to the ANAO that the dashboard will be reported to the executive monthly in 2019 and quarterly from January 2020. KPI performance varied across the service providers. In summary, the report found:

  • KPI 1, orientation outcomes at the ‘knowledge’ level46: met by only one of the five providers for all three months in the quarter.
  • KPI 2, orientation outcomes at the ‘application’ level: met by only one of the five providers for all three months in the quarter.
  • KPI 9, data entered in to the HSP system within 14 days: met by only two of the five providers for all three months in the quarter.

3.39 One provider (Service Provider Z) only met one of the three KPI’s, for one month in the quarter. Home Affairs met with Service Provider Z in October 2019 and Service Provider Z noted it was implementing strategies to improve performance including a new case management system and a quality management system.

3.40 Home Affairs has organised meetings to discuss performance with each provider in October and November 2019 based on this performance information.

Reporting on service provision

3.41 Six HSP services have to be delivered within specified time periods. For each of these services, the July 2019 dashboard shows the percentage of service instances in July that met the timeframe requirement.47 These outcomes varied across providers, with Service Provider Z having much lower percentages than other providers. The overall results for all providers were:

  • health appointment (80 per cent of clients);
  • client registered for health follow up (76 per cent of clients);
  • client registered with English language opportunities (100 per cent of clients);
  • school enrolment (69 per cent of clients);
  • case management plan developed (79 per cent of clients); and
  • essential registrations (for example, Medicare) (66 per cent of clients).
Other internal and external reporting

3.42 Overall, there has been limited internal and external public reporting on HSP outcomes, most of which has occurred from 2019. Reporting has consisted of:

  • a May 2019 internal report to the HSP Working Group on transfers between regions;
  • the July to September 2019 dashboard reported to the Home Affairs Executive in October 2019, with monthly reporting planned for the rest of 2019 and quarterly reporting in 2020 (see paragraph 3.38);
  • a report to the Minister on HSP data and outcomes in July 201948;
  • external reports to the Provider Advisory Group in December 2018 and July 2019 (numbers of services provided and numbers of clients reaching orientation outcomes);
  • based on reports from the five service providers, DSS prepared the first six-monthly report on the HSP for the period up to 31 December 2018 (see 2.47);
  • KPI data from July to December 2018 and for the July 2019 KPI report was given to service providers after June 2019; and
  • Home Affairs provides monthly de-identified HSP arrivals and referrals data for planning purposes to the Senior Officials Settlement Outcome Group (inter-governmental forum).

3.43 There is very limited public reporting on the HSP via the DSS Annual Reports. The 2018–19 report notes that the program moved to Home Affairs in July 2019 and each annual report shows the number of individuals assisted under the HSP for the last three years.49

Are continuous improvements made to the delivery of the HSP?

Continuous improvements are being made to the delivery of the HSP, with further work required. A number of benefits were intended to be gained when moving from the previous programs to the HSP. These improvements were largely reflected in the HSP contracts but the benefits have only been partially achieved. For example, shifts to outcomes focused contracting, improved data and a new IT system have only been partly achieved. Other improvements to the HSP have been identified via a number of reports and reviews. Flowing from this, Home Affairs prepared a high-level schedule with seven elements to improve the HSP. This includes finalising and implementing the recommendations of the 18 month review of the HSP.

3.44 In order to assess whether continuous improvements were being made to the HSP the ANAO considered:

  • if the recommendations made before transitioning to the HSP were implemented;
  • the status of HSP reviews and improvement projects; and
  • the availability of performance information.

Improvements to be made via transitioning to the HSP

3.45 DSS intended to achieve a number of benefits from moving to the HSP (see Table 3.4). The contract design largely reflects these but the benefits have only been partially delivered.

Table 3.4: Achievement of the intended benefits of moving to the HSP

DSS — intended benefits of the HSP

Was this achieved via the HSP?

 

Fewer service providers and contract regions

Full Harvey ball

There were 16 HSS and 33 CCS service providers across 23 contract regions. The HSP has five service providers across 11 contract regions.

Move towards an outcomes focussed contract

Half full Harvey ball

There is a Program Outcomes Framework. HSP contract payments relate to outputs rather than outcomes. DSS cannot monitor progress against many of the outcomes (see paragraph 3.35). Home Affairs’ schedule for potential future improvements notes development of a new funding model with outcomes based options.50

Provide more flexibility to deliver needs-based services

Full Harvey ball

HSP case management plans are made for individuals (there were family-group level plans in the HSS and CCS). The Program Outcomes Framework and contracts reflect the needs-based focus.

Focus on the ‘three E’s’: English, Education and Employment and improve the orientation/

foundation services

Half full Harvey ball

The HSP contract requires Service Providers to coordinate access to English, education and employment. The department has not tracked client outcome DEX scores in these areas which could illuminate clients’ progress (see paragraph 3.23). DSS developed KPIs but there is no direct line of sight oversight over how orientation levels are met (see paragraph 3.33).

Provide incentives for the service provides to innovate

One quarter full Harvey ball

DSS included ‘Innovation targets’ in the contracts but these have not been implemented.

Improve data collection and reporting on the program

One quarter full Harvey ball

DSS did not use the data from the HSP system to report on KPIs until July 2019 (see paragraph 3.29). Home Affairs stated that data collection has improved compared to the HSS and CCS programs and there will improvements to HSP data.

Provide a new bespoke HSP IT system

Half full Harvey ball

DSS created a bespoke IT system, but it has experienced issues since released and requires additional funding to meet the intended original functionality (see Chapter 2, paragraphs 2.64–2.91).

Key:

One quarter full Harvey ball Planned but not implemented

Half full Harvey ball  Partially implemented

 

Three quarter full Harvey ball Largely implemented

Full Harvey ball Fully implemented

    

Source: ANAO analysis.

Reviews and reports on the Humanitarian Settlement Program

3.46 There have been a number of reports and reviews on the HSP and some changes were made in response. In August 2019 Home Affairs provided the ANAO with a high-level schedule of planned improvements to the HSP drawing on reviews to date. This work has seven elements:

  • progressively implement recommendations from the 18 month review from October 2019;
  • Stage one recalibration: consolidation of payment points and options to improve employment outcomes (by early 2020);
  • Stage two recalibration which involves a new outcome based and tier three funding model (by 1 July 2020);
  • commence the HSP Assurance and Compliance Strategy (1 January 2020) — linked to this there is a September 2019 version of the draft strategy and initial and follow-up client interview questionnaires have been developed;
  • transition in KPIs 1, 2 and 9 (by 1 January 2020) and consider potential further KPIs;
  • develop options to manage service provider financial viability (these are noted in Chapter 2, paragraph 2.63 and include additional payments for tier three clients); and
  • review HSP sub-contracting and implement recommendations (by 30 December 2019).
The Humanitarian Settlement Program Six-Monthly Reports

3.47 Based on reports from the five service providers, DSS prepared the first six-monthly report on the HSP for the period up to 31 December 2018. Issues to be actioned included:

  • considering the Provider K subcontractor risks (later action is detailed in paragraph 2.61)
  • engaging with providers on the abatement regime before it was to go live in July 2019 (the abatement regime was not active as at September 2019); and
  • liaising with other agencies to resolve issues (for example, the National Disability Insurance Agency and Centrelink as forms were not client-friendly and there was difficulty linking clients to the National Disability Insurance Scheme).51

3.48 As noted in Table 2.4, providers have now submitted their six-monthly reports for January to June 2019. Home Affairs has not yet prepared a consolidated report on these.

Business Improvement Future State Report

3.49 To support the Government’s streamlining and deregulation agenda, DSS commissioned Ernst and Young (EY) to develop an HSP — Business Improvement Future State Report. This included a risk management plan (see paragraph 3.9), a high-level assurance and compliance framework, and 69 recommendations.

3.50 EY proposed that assurance should be based on desk-top reviews and recommended eliminating client visits, which ceased in July 2018. Client interviews are planned under the new draft Assurance and Compliance Strategy due to commence at the start of 2020.

3.51 When it received the report in September 2018, DSS decided that 29 recommendations would be implemented over 18 months, 20 had been recently implemented, 12 related to the HSP IT system and would be dealt with separately given funding considerations and an alternative approach would be taken for eight recommendations.

3.52 In February 2019 DSS consolidated the EY recommendations to focus on six high priorities for 2019. Home Affairs advised that the six priorities did not proceed as planned, but that the priorities are planned to be actioned by:

  • developing a HSP monitoring strategy (now in the draft Assurance and Compliance Strategy);
  • centralising client referral functions into the National Office which occurred in November 2019;
  • formalising HSP governance forums and structures — Home Affairs stated that it is currently reviewing HSP governance structures;
  • updating standard operating procedures for internal governance — now to be undertaken by Home Affairs on an ongoing basis;
  • updating standard operating procedures for service provider risk assessments — now to be undertaken by Home Affairs on an ongoing basis; and
  • providing training to staff and service providers — will be incorporated into Home Affairs’ training strategies over 2019–20.
DSS report on the first 18 months of the program

3.53 As at September 2019 Home Affairs was finalising an internal report on the first 18 months of the HSP from 30 October 2017 to 30 April 2019. The draft report states that the HSP was ‘mainly working well’ but there is an opportunity for improvements. The draft has 13 recommendations (Table 3.5 below), to be progressively implemented from October 2019.

Table 3.5: Recommendations in the September draft of the 18 month report on the HSP

Number

Recommendation

1

Progress enhancements to the HSP System to address risks and functionality.

2

Review service provider reporting requirements to reduce administrative burden.

3

Consider all recommendations of the review into HSP subcontracting arrangements.

4

Improved claiming for clients who enter the HSP via tier three.

5

An Orientation workshop to clarify how to assess achievement of orientation outcomes.

6

Undertake an independent review of Orientation in line with the Orientation Guidelines.

7

Reconsider the policy position on crisis accommodation and relevant payment points.

8

Publish a model Case Management Plan to illustrate the requirements for these.

9

Update Case Management Guidelines — further information on evidence for claims

10

Increase the cap for the number of times a service can be claimed without separate departmental approval, for example, for health appointments.

11

Update service providers and staff on the assessment criteria for Tier 3 referrals.

12

Consider additional payments given the additional support required to Tier 3 clients.

13

Update critical incident policy and remind service providers of the correct process for reporting incidents.

  

Source: September draft of the First 18 months of the HSP Report

Review of the Basic Household Goods package

3.54 Under the HSP, clients can be given a Basic Household Goods (BHG) package (furniture and household goods) to help establish them in their on-going accommodation.52 From 31 October 2017 to 31 May 2019, expenditure on BHG ($31.7 million) accounted for 26 per cent of total HSP expenditure (the highest category of expenditure).

3.55 DSS conducted a review of BHG pricing and invoicing one year after the HSP commenced. It examined 809 BHG claims and found that in two contract regions (one each for Service Providers K and R) there were much higher rates of maximum claims (95 per cent and 60 per cent respectively). The review then examined six invoices from November 2017 to February 2018 for each contract region (and 12 for the regions with the high maximum claims).

3.56 The review found that invoicing varied across the contract regions; some invoices were not clear; some invoices lacked itemisation; some invoice totals were higher than what had been claimed; and some clients were residing together but were each claiming a BHG package.

3.57 In the regions with the high maximum claim levels, a single BHG package supplier was used (Package Supplier P). It invoiced a single price for the whole BHG package of goods. DSS requested an itemised breakdown of the BHG prices in the two regions with the high levels of maximum claims.

3.58 Service Provider R responded with the itemised invoices. Service Provider K advised DSS that it could not provide the itemised amounts. From the information Service Provider R submitted, DSS concluded that the price Package Supplier P charged for some items (for example, toasters) appeared to be above what would be considered to be reasonable market prices.

3.59 The review proposed obtaining the breakdown of individual BHG prices from Service Provider K, investigation of discrepancies across three providers and that service providers be reminded of the appropriate use of tax-payer funds in purchasing BHG packages. A broader review of BHG was also proposed.

3.60 Recommendations from the 2018 review are yet to be fully actioned. Home Affairs stated to the ANAO in July 2019 that it was considering the recommendations and had already implemented some changes such as enhanced guidance to contract managers.53

Performance information

Employment

3.61 Home Affairs’ high-level schedule of future potential improvements states that it is planning to consider options to improve employment outcomes. A 2017 briefing to the Assistant Minister for Social Services and Multicultural Affairs noted that barriers to obtaining work included limited English, physical or mental health status and difficulties in recognising overseas qualifications.

3.62 The Australian Bureau of Statistics (ABS) Australian Census and Migrants Integrated Dataset 2016 has data on employment outcomes: 48.1 per cent of humanitarian entrants were in the labour force, of which 79.7 per cent were employed and 20.3 per cent were unemployed.54

3.63 Home Affairs has noted that it will use external reports such as a 2018 report on the settlement experiences of refugees who settled in Queensland after coming from Syria, Iraq and Afghanistan.55 Key findings were:

The policies and procedures that have been established to support humanitarian immigrant and refugee settlement in the first years of arrival — and the organisations that have successfully tendered to provide these services — are very successful…The evidence strongly attests that for refugees and humanitarian immigrants getting access to the Australian labour market is perhaps the greatest settlement challenge that they face.

3.64 The departments have not used the seven DEX score categories for employment outcomes to assess employment outcomes (these categories range from ‘significant disengagement in the labour market’ to ‘appropriate long term engagement in employment).’ However, KPI data extracted in October 2019 and backdated to commencement of the program shows 74% of clients aged 15 years and over have met the ‘application’ outcome level for the employment orientation. The ‘application’ outcome level indicates that these clients know there are a number of employment services to assist in finding work, and they can develop a job resume and complete a job application.

Health and well-being

3.65 The health status of HSP humanitarian migrants was covered in a longitudinal survey by DSS’ Australian Institute of Family Studies’ called Building a New Life in Australia.56 In the period when participants would have been in Australia for up to nine months and likely receiving settlement services (under the prior programs to the HSP), 12 per cent of males and 18 per cent of females indicated poor or very poor health over the previous four weeks.57 Thirty five per cent of males and 46 per cent of females had moderate or high levels of psychological distress.

3.66 The departments have not tracked the DEX score outcomes which support employment and quality of life, including mental and physical health. However, KPI data extracted in October 2019 and backdated to commencement of the program shows 85% of clients aged 15 years and over met the ‘application’ outcome level for the health orientation. The ‘application’ level outcome indicated clients can independently locate and make use of appropriate health services.

Future reviews of performance information

3.67 Given the transfer of the program to Home Affairs on 1 July 2019, there is an opportunity for Home Affairs to review the performance information it obtains to be better able to determine if the objective of the HSP is being met.

3.68 To date the departments have not used client outcome data (DEX scores) to assess HSP performance or to inform KPI design or program improvements.58 As noted at paragraphs 3.22–3.23, client outcome DEX scores are recorded in 13 outcome areas at points in the client’s journey. Home Affairs plans to consider additional KPIs for service providers in 2020. This review should be expanded to include a review of all program performance information.

3.69 The departments have also not collected client satisfaction data for the HSP at client exit interviews.5960 The draft Assurance and Compliance Strategy, planned to commence at the start of 2020, includes sampling through client interviews. The draft makes reference to client satisfaction with rental accommodation, basic household goods and English classes and has a ten point quality rating scale for the HSP services overall.

3.70 The departments have not established a data management plan, which could have helped plan data collection, analysis and application. Such a plan could make reference to DEX scores, KPIs, client satisfaction data or other settlement outcomes data in order to be better able to determine if the program is meeting its objective.

Recommendation no.3

3.71 Home Affairs should finalise the review and update of HSP performance information including by:

  1. refining the KPIs in the HSP contracts to reflect relevant elements of the HSP Program Outcomes Framework;
  2. providing additional guidance to service providers on how to measure attainment of the orientation competencies; and
  3. developing a Data Management Plan to help track if the HSP Program is meeting its objective

Department of Home Affairs response: Agreed.

3.72 The Department considers that it already has good performance information to assess program performance and client progress against outcomes. This includes data on client achievement against the orientation outcomes which directly links to the HSP Outcomes Framework, service provider performance reports, key performance indicators, research, internal data reports on service levels and client outcomes.

3.73 Notwithstanding this, the Department is implementing a new performance framework which will further strengthen performance reporting. This includes building on existing key performance indicators to better align to the HSP Outcomes Framework and our HSP data management plan to improve tracking of client progress towards outcomes and achievement of the program objective. The Department will continue to improve guidance to service providers on measuring attainment of orientation outcomes and work to better link HSP to the Adult Migrant English Program and other settlement programs to provide a more holistic approach to supporting clients to achieve settlement outcomes.

3.74 A new Assurance and Compliance Strategy has also been recently finalised, to provide assurance that settlement services are delivered appropriately, support client progression towards outcomes, and tests integrity of service provider claims. This includes gaining insights directly from clients on their overall satisfaction of settlement services and where improvements could be made. The Department is reorientating the HSP to move from an outputs measured model to an outcomes based model providing greater flexibility to respond to client needs and achieve outcomes.

Appendices

Appendix 1 Entity responses

Page 1 of the Department of Home Affairs' response

Page 2 of the Department of Home Affairs' response

Page 1 of the Department of Social Services' response

Page 2 of the Department of Social Services' response

Footnotes

1 The HSP factsheet refers to ‘clients’ of the program. [Internet], available from: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-factsheet.pdf [accessed 29 November 2019].

2 Australian Parliament, Budget Review 2019–20, ‘Key Trends: Humanitarian Program’, available from: https://www.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/pubs/rp/BudgetReview201920/Immigration, [accessed 7 June 2019].

3 Participation in the HSP is voluntary.

4 Home Affairs data.

5Evaluation of the HSS and CSP, prepared by Ernst and Young, June 2015.

6HSP Program Logic, Department of Social Services.

7 Prime Minister and Cabinet, Administrative Arrangements Order, Summary of Changes, 29 May 2019, available from: https://www.pmc.gov.au/sites/default/files/publications/aao-summary-changes-20190529.pdf, [accessed 2 August 2019].

8 It was expected that the majority of HSP clients will be classified as tier two. Based on HSP System data filtered by arrival date and whether the client transitioned from the previous program, Home Affairs advised the proportion was:

  • Not Yet Assigned: 0.1%
  • Tier 1: 0.3%
  • Tier 2: 94.2%
  • Tier 3: 5.3%

9 Humanitarian Settlement Program Fact Sheet, Department of Social Services, [Internet], available at: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-factsheet.pdf [accessed 29 November 2019].

10 The HSP Factsheet refers to ‘clients’ of the program. [Internet], available from: https://immi.homeaffairs.gov.au/settlement-services-subsite/files/hsp-factsheet.pdf, [accessed 23 October 2019].

11 Auditor-General Report No. 36 2008–09 Settlement Grants Program https://www.anao.gov.au/work/performance-audit/settlement-grants-program, [accessed 29 November 2019]

12 The Australian Government Contract Management Guide was released in July 2019, two years after the HSP contracts were signed. However, these common contract provisions are also included in the DSS Contract Management Guide, which was available at the time the HSP contracts were created and finalised.

13 The abatement regime was intended to be inactive for the first 12 months of the HSP, from October 2017 to October 2018. As at August 2019, the abatement regime has not yet been activated and KPIs have not yet been fully implemented.

14 In May 2017, during the tender evaluation process, DSS issued a clarification to provide a more detailed MOC definition based on it only capturing costs for maintaining a minimum level of capacity in the face of uncertainty in relation to client numbers. Labour costs were to only include staff costs not associated with case management or other aspects of service delivery which were to come under SSCs.

KPMG Pricing Reports which analysed the pricing in the tenders includes tables with the tendered price for the DSS budgeted components of the MOC which were: electricity, office lease, staffing, service coordination, and reporting. Some tenderers included extra components e.g. motor vehicles.

15 Some of the Settlement Services Charges have capped costs which were set by DSS during the tender process. For example, service providers can invoice DSS a total of $27.89 (2018–19 prices) for assisting each HSP client to lodge documents with the Free Translation Service (code: G04), and up to $152.10 (2018–19 prices) for purchasing a mobile phone with six weeks of credit for HSP clients (code: I06). The Contracts provide for annual adjustment of the payment amounts (MOC and SSCs) on 1 July each year in accordance with variations on the Wage Cost Index from the Department of Finance.

The value of an SSC varies extensively between service providers, regions and specific payment points (on average, the value of the same SSC varies by 800 per cent between different providers).

16 Refer to recommendation 10 in Table 3.5.

17 The HSP contract contains the disclaimer that the department provides no guarantee of: the volume or type of business the service provider will receive; the numbers of clients for any services under this contract; or the number of clients for any contract region in relation to any services under the contract.

18 Codes used for the providers for commercial-in-confidence reasons. These were randomly assigned to the five service providers with the letters F, J, K, R and Z used.

19 Performance management for the program is discussed in Chapter 3.

20 Department of Finance, Australian Government Contract Management Guide v1.1, July 2019, page 2, available from: https://www.finance.gov.au/procurement/contract-management-guide, [accessed 18 July 2019].

21 DSS drafted a contract management plan for one HSP service provider but this was not finalised.

22 For example, if a service provider accrues 40 or more failure points in a contract region over a six month period, the MOC payment may be reduced by 10 per cent.

23 The ANAO did not audit the performance of the service providers. Some service providers advised the ANAO that they had established or were establishing their own internal assurance activities.

24 The ABS sample size tool was applied [Internet], available from: https://www.abs.gov.au/websitedbs/D3310114.nsf/home/Sample+Size+Calculator For a 95% confidence level and a 0.05 confidence interval, the sample size for the 7579 arrivals would be 366.

25 One service provider commented to ANAO that for complex clients, case management plans can take longer than 21 days as a number of meetings may be required to build trust and fully identify client needs.

26 One service provider advised the ANAO there was a lack of clarity on what documents needed to be uploaded and practical difficulties, such as uploading copies of leases if the client sourced the accommodation themselves. This provider had been working with DSS on clarifications of required documentation.

27 Home Affairs also stated it is undertaking a review of the six-monthly report template with possible implementation of more real-time reporting prior to the next reporting period (January 2020).

28 A service cap increase can be requested when the cap on the number of times the service can be claimed is reached but the client needs more instances of the service.

29 Includes 630 users across 26 provider organisations, 175 Australia based departmental staff and 180 departmental staff across 12 other countries (as at September 2019).

30 These issues were reported through the Community of Practice (see Table 2.5).

31 DSS received a small number of responses to these surveys; 22 for the December 2018 release, and 11 for the February / March 2019 release.

32 Not all HSP system fields are reportable as there is an associated cost, but some were reportable since the HSP commenced

33 Individual service provider reporting which was examined in Chapter 2.

34 Non-corporate Commonwealth entities (including DSS and Home Affairs) must comply with the Department of Finance, Commonwealth Risk Management Policy 2014 (supports section 16 of the Public Governance, Performance and Accountability Act 2013) [Internet], available from: https://www.finance.gov.au/government/comcover/commonwealth-risk-management-policy, [accessed 8 July 2019].

35 This conclusion was based on assessment of documentation including an entity-level Risk Management Policy and Enterprise Risk Management Framework, and an assessment of HSP-specific risk documents.

36 The October 2017 report (just before the HSP commenced) recorded a high risk rating for the transition overall and for the IT system, noting insufficient test coverage. At June 2018 the overall transformation risks were still rated as high but the IT system risk had dropped to medium.

37 Supplier performance (Risks 1, 2, 6, 7 and 8); process and system issues including disclosure (Risks 3, 4 and 9); insufficient staff expertise (Risk 5); fraud, theft and corruption (not covered in April 2018 but included later); and shared risks with other entities (Risk 2).

38 There was no specific reference to risks due to HSP payment levels (see paragraph 2.58–2.64 on Service Provider K financial viability issues).

39 In September 2018 Ernst and Young (EY) produced an HSP — Business Improvement Future State Report. This identified six risks (aligned with the nine April 2018 risks) but added a fraud risk. EY recommended keeping 24 existing controls, making 18 control improvements and adding 18 new treatments to reduce the overall program risk rating from high to medium. Details on whole scope of this report are in paragraph 3.49–3.52.

40 This is also reflected in the DSS Enterprise Risk Management Framework. The risk appetite was also that low risks were generally acceptable and medium risks were acceptable or could require treatment.

41 A top-up payment was made to Service Provider K’s subcontractors in March 2019.

42 The non-critical incidents are only non-critical in that they are not directly related to the provision of HSP services.

43 It may also need to review KPI data quality to determine if compliance rates are accurate or still hampered by prior data issues (and noting that overall, the two orientation KPIs are not being met as at July 2019 data).

44 The ten core settlement topics are: settlement services; housing; health; money; feeling at home in Australia; transport; Australian law; education; employment; and family functioning and social support.

45 One provider explained to the ANAO its process to ascertain that a client can actually use public transport before a claim is made for achievement of the ‘application’ level for this skill. Another service provider noted it was developing an assessment framework for clients meeting the knowledge and application levels.

46 KPI 1 performance was measured for clients who reached their seven-month arrival anniversary in July 2019. KPI 2 performance was measured for clients who reached their 12-month arrival anniversary in July 2019.

47 The percentages are: number of services delivered in July 2019 which met the service standard divided by total number of services delivered in July 2019. Each standard is based on timeframes for service delivery set out in the contract.

48 Minister for Immigration, Citizenship, Migrant Services and Multicultural Affairs.

49 The 2018–19 Annual Report includes a percentage of assisted migrants and humanitarian entrants with improved engagement with support services but this relates to community settlement grants and not the HSP and the data is incomplete. Refer to pages 9, 55 and 67 [Internet], available from: https://www.dss.gov.au/sites/default/files/documents/10_2019/part-2-d19-1139120-dss-annual-report-2018-19.pdf, [accessed 16 October 2019].

50 Home Affairs stated that a move towards an outcomes focus was planned over time, following the collection of output data over the first few years of service delivery.

51 Home Affairs stated that it is working with the Department of Social Services to improve HSP client access to the NDIS.

52 There are six bands for the maximum value of the BHG package based on the household size. For 2017–18 (in the contracts) these varied from $4,300 for a single client to $10,500 for 9 or more clients (with the amounts updated annually given indexation). 

53 When establishing the HSP, it may have been possible to use the purchasing power of the HSP to establish preferred provider agreements with these retailers providing items at a discount compared to the retail price. The Department of Finance July 2019 Australian Government Contract Management Guide says that coordinated procurement arrangements (e.g. a whole of government panel) can be used to leverage the purchasing power of the Australian Government (a relevant principle) [Internet], available from: https://www.finance.gov.au/sites/default/files/Contract%20Management%20Guide%20July%202019.pdf, [accessed 14 August 2019].

54 ABS Australian Census and Migrants Integrated Dataset 2016 [Internet], access available from: https://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/3417.0.55.0012016?OpenDocument but this requires software from the ABS.

55Settlement experiences of recently arrived refugees from Syria, Iraq and Afghanistan in Queensland in 2018, University of Technology Sydney, Sydney University and Western Sydney University. This is the first of three place-based reports on settlement outcomes for Syrian Iraqi and Afghan refugees. It is a longitudinal study. [Internet], available from: https://www.uts.edu.au/research-and-teaching/our-research/centre-business-and-social-innovation/research/projects-0 [accessed 26 September 2019].

56 DSS, Australian Institute of Family Studies Building a new Life in Australia longitudinal study — following 2399 humanitarian migrants from 2013 as they settled [Internet], available from: http://www3.aifs.gov.au/bnla/index.html#about, [accessed 12 July 2019].

57 DSS, Australian Institute of Family Studies Building a new Life in Australia longitudinal study, Data Highlights Paper [Internet], available from: https://www.dss.gov.au/sites/default/files/documents/09_2015/data-highlight-no-2-2015-bnla_pdf.pdf, [accessed 12 July 2019].

58 It is recorded initially in a spreadsheet and then in the HSP system when this function is available.

59 There could be bias in the responses as the service provider administers the exit interview.

60 Two service providers outlined their exit interview client satisfaction survey approach to the ANAO. One looked at the client’s satisfaction with the relationship with the case manager, the services provided and the extent to which they felt safe and involved.