Audit snapshot

Why did we do this audit?

  • The internet is a fundamental utility for the majority of Australians.
  • $49 billion in public funds are being invested to establish the NBN.
  • The NBN rollout is due for completion in 2020 and migration will continue through to 2021. There is a public interest in service continuity.

Key facts

  • NBN Co was established in 2009 and the Australian Government has provided contributed equity of $29.5 billion and an additional $19.5 billion loan.
  • The audit examined NBN Co processes over the period March 2018 to March 2019. During this period, the average weekly volume of new connections to the NBN rose from 37,443 to 44,605.
  • As at March 2019, approximately 83% of surveyed users understood the need to switch to the NBN in the next 18 months to retain their telecommunications service.

What did we conclude?

  • NBN Co’s operations to support service continuity in the migration of telecommunication services to the NBN are largely effective. This is because:
    • communications effectively educate end users about migration;
    • processes are effectively designed to prevent service continuity faults, however operational risk management frameworks require improvement; and
    • complaints are appropriately handled, however are not being resolved within NBN Co targets or effectively analysed.

What did we recommend?

  • The Auditor-General made three recommendations to NBN Co. They relate to the risk register, complaints management policy and the use of data to analyse complaints.
  • NBN Co agreed to all recommendations.

$5.7 million

There are 5.7 million premises with active services on the NBN, and 10.5 million premises ready to be connected.

40%

Approximately 40 per cent of end user complaints during the audit period were related to new NBN connections

80%

Approximately 80 per cent of disruption related service faults were resolved within four working days.

Summary and recommendations

Background

1. The National Broadband Network (NBN) is a major upgrade to Australia’s fixed line telephone and internet infrastructure to provide Australian premises with access to high speed broadband and telephony services.

2. NBN Co Limited (NBN Co), a Commonwealth company, was established in 2009 to design, build and operate the NBN.1 As at 30 June 2019, NBN Co had 6,400 employees.

3. NBN Co projects that 8.1 million premises in Australia will have an active NBN service by 30 June 2021, with 95 per cent of these made using fixed line technology.

4. NBN Co’s responsibilities regarding migrating a premises onto the NBN include building the network to make premises serviceable, supplying wholesale products to retail service providers, working with stakeholders to activate and assure services, contracting technicians to carry out network repairs and maintenance, and undertaking public and industry awareness and education activities about the network rollout and migration processes.

5. The NBN Co Statement of Expectations, which sets out the Australian Government’s expectations of NBN Co as a Commonwealth company, states that together with retail service providers, NBN Co ‘…should work to ensure a high quality end user experience through the migration and ongoing service periods. This includes working closely with retail service providers to proactively manage any complaints’.

Rationale for undertaking the audit

6. The internet has become a fundamental utility for the majority of Australians. The NBN rollout represents a substantial investment of public funds, and the rollout is due for completion at the end of 2020.2 The process of migrating to the NBN will affect the majority of Australian residents and businesses, and there is a clear public interest in minimising interruptions to service continuity during this process. The migration process will continue beyond the completion of the rollout, and it is therefore timely to examine the effectiveness of NBN Co’s operations to support service continuity throughout the migration process.3

7. Under the Auditor-General Act 1997, as a Government Business Enterprise, performance audits of NBN Co can only be undertaken at the request of the Joint Committee of Public Accounts and Audit (JCPAA). A request from the JCPAA was received in October 2018 following an approach from the Auditor-General for such a request.

Audit objective and criteria

8. The objective of this audit was to assess the effectiveness of NBN Co’s service continuity operations relating to the migration of telecommunication services to the NBN.

9. To form a conclusion against the audit objective, the Australian National Audit Office (ANAO) adopted the following high-level criteria:

  • Has NBN Co implemented effective strategies to inform and educate telecommunication end-users about the connection process?
  • Does NBN Co have effective processes to manage the risks to service continuity between an end user lodging a request to migrate to the NBN, and the end user having a fully operational telecommunication service?
  • Does NBN Co effectively manage end user connection-related complaints and drive improvement?

10. The audit examined NBN Co’s operations from 16 March 2018 until 28 March 2019 in order to assess its effectiveness against contemporary industry settings.

Conclusion

11. NBN Co’s operations to support service continuity in the migration of telecommunication services to the NBN are largely effective.

12. NBN Co’s strategies to inform and educate end users about the connection process are effective. Communication structures, channels and products have been developed with the aim of maximising end user understanding and awareness of the migration process, and these products satisfy the requirements for public information contained in the Public Information on Migration Deed. Substantial qualitative and quantitative data is collected, analysed and reported to key stakeholders on a regular, ongoing basis to monitor and evaluate the effectiveness of communication activities.

13. NBN Co has established largely effective processes to manage service continuity between an end user lodging a request to migrate to the NBN, and the end user having a fully operational service. Structures and controls have been implemented to manage common service disruption issues during the connection and activation process, and frameworks have been established for ongoing operational engagement with retail service providers (RSPs). Clear processes have been established for the diagnosis, reporting, investigation and repair of faults, with ongoing work to increase the timeliness of fault resolution. NBN Co’s risk framework is partly effective in supporting the management of service continuity risks at the operational level.

14. NBN Co is partly effective in managing end user connection-related complaints. Effective internal policies, guidance and quality assurance processes have been established for the receipt and management of complaints by NBN Co, however, public guidance established to assist end users in the lodgement of complaints requires updating for completeness and to ensure appropriate accessibility. Connection-related complaints are not being resolved in line with NBN Co’s timeliness targets, and NBN Co has not established processes to conduct ongoing or systemic analysis of complaint causes to drive continuous improvement.

Supporting findings

Communications

15. NBN Co’s communication strategies for the connection process are effective. Strategic and process documentation reflect a high level of maturity in relation to the governance of communication activities. NBN Co has established a series of communication products and channels aimed at raising general awareness and understanding among end users of the NBN service, the process for the disconnection of legacy infrastructure, and the potential benefits of connecting to the NBN. Communications are appropriately targeted at specific demographics, and communication products are satisfying the Public Information on Migration Deed requirements.

16. NBN Co has established processes to monitor and review the effectiveness of its communications channels in educating and informing end users about the NBN connection process. Key measures in NBN Co’s annual operating plans are used to monitor the effectiveness of communications, and ongoing reporting and dialogue among key stakeholders provide the opportunity to validate these measures.

Management of service continuity

17. NBN Co has established processes that are effective in preventing risks to service interruption during the process of connecting and activating a service within its span of control. NBN Co has established clear, detailed procedures and operational structures to facilitate the process of migrating a premises on to the NBN. Governance arrangements support effective engagement with RSPs, including ongoing reporting and communication activities.

18. NBN Co’s resolution of disruption-related service faults on newly connected services is not fully effective, because NBN Co is not meeting targets for the resolution of service faults within agreed timeframes. NBN Co has established clear processes to diagnose, report, investigate and repair faults, and there is a downward trend in the volume of disruption-related service faults occurring within 30 days of a newly connected service, despite an overall increase in the volume of new connection activity over the same period. The average time taken to resolve service disruption faults on newly connected services is approximately 3.1 work days, and 80 per cent of faults are resolved within four days or less. However, NBN Co performance reporting reflects that 89 per cent of standard connection service faults are being resolved within agreed timeframes against a target of 90 per cent, and 95 per cent of priority assistance service faults are being resolved within agreed timeframes against a target of 100 per cent. A program of work is underway to improve service assurance and fault resolution, including a focus on reducing the number of faults and the instances where technicians need to be deployed.

19. NBN Co’s risk management framework is partly effective in supporting operations for the management of service continuity. NBN Co has an established enterprise risk management framework. However, NBN Co’s risk register does not cover all aspects of managing service continuity during the process of migrating a premises on to the NBN. It also has a high number of risks that are not ‘approved’, to indicate agreement between relevant staff responsible for the management of the risk, and that have no identified controls or treatment actions.

Complaints management

20. NBN Co has established a largely effective complaints management process. A complaints management policy has been established to provide guidance to end users in lodging complaints, however this does not fully align with the relevant industry standard (which is not mandatory in its entirety for NBN Co), and is not readily accessible to end users. Clear policies, guidance and quality assurance processes have been established for the receipt and management of complaints by NBN Co case management staff, noting that there are minor ambiguities in the accurate classification of complaints within the complaints management system.

21. Connection-related complaints are not being resolved in line with NBN Co’s timeliness targets. NBN Co has defined three complaint categories — Formal, Complex and Urgent — and established clear KPIs for each. However, 87.7 per cent of combined Formal and Complex complaints are being resolved within KPIs, which is below the NBN Co target of 93 per cent, and 45.5 per cent of Urgent complaints, of which there were only 11 in the audit timeframe, were resolved within the associated KPIs, which is also below the NBN Co target of 93 per cent. Complaints that are unresolved within the respective KPIs for all three categories are typically not resolved in a sufficiently timely manner after the KPI period has elapsed.

22. NBN Co undertakes regular internal reporting on the volume of complaints and associated resolution timeframes, but could not demonstrate that it conducts ongoing or systemic analysis of connection-related complaint causes either internally or with retail service providers.

Recommendations

Recommendation no.1

Paragraph 3.58

NBN Co formally reflect all risks, controls and ongoing monitoring processes for risks associated with the management of service continuity during migration of an end user’s premises within the NBN Co centralised risk register.

NBN Co response: Agreed.

Recommendation no.2

Paragraph 4.18

NBN Co revise its Complaint Management Policy to align with the Telecommunications (Consumer Complaints Handling) Industry Standard 2018, and ensure that the policy document and associated processes are approved and appropriately accessible on the NBN Co homepage.

NBN Co response: Agreed.

Recommendation no.3

Paragraph 4.51

NBN Co establishes a regular and ongoing process to use its complaint data to undertake root cause analysis of connection-related complaints, to inform continuous improvement internally and with complaints management stakeholders.

NBN Co response: Agreed.

Summary of entity response

23. The proposed report was provided to NBN Co. NBN Co’s summary response is below and its full response is at Appendix 1.

NBN Co appreciates the time and effort by the ANAO to review a wide range of key processes across strategies used to inform and educate telecommunications end users about the connection process, the management of risk to service continuity during the migration phase and how NBN Co manages end user connection related complaints and drive continuous improvement.

We welcome the overall positive conclusion that NBN Co’s operations in relation to support service continuity in the migration of telecommunications services to the nbn™ access network was assessed as largely effective.

We are particularly pleased that the following observations were recognised in your conclusion within the Report;

  • that NBN Co strategies to inform and educate end users about the connection process are effective.

    This includes communication structures, channels and products that maximise both end user understanding and awareness of the migration process.

    This is supported by processes that capture, analyse and report data to monitor the effectiveness of these activities.
     
  • that NBN Co has established largely effective processes to manage service continuity between an end user lodging a request to migrate to the nbn™ access network, and the end user having a fully operational service.

    This includes recognising that NBN Co has clear processes for the diagnosis, reporting, investigation and repair of faults, with on-going work to increase the timeliness of fault resolution.

    The ANAO has also acknowledged that governance arrangements that support the effective engagement with RSPs, including ongoing reporting are in place.

    Furthermore, it is noted that there is a downward trend in the volume of disruption-related service faults occurring within 30 days of a newly connected service, despite an overall increase in the volume of new connection activity over the same period.
     
  • that NBN Co has effective internal policies, guidance and quality assurance processes for the receipt and management of complaints with NBN Co having established a largely effective complaints management system.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages which have been identified in this audit that may be considered by other Australian Government entities, and Government Business Enterprises, when implementing government funded infrastructure initiatives.

Service delivery

Governance and risk management

Performance and impact measurement

1. Background

Introduction

The National Broadband Network and NBN Co Ltd

1.1 The National Broadband Network (NBN) is a major upgrade to Australia’s fixed line telephone and internet infrastructure to provide Australian premises with access to high speed broadband and telephony services.

1.2 The NBN is managed by NBN Co Ltd (NBN Co), a Commonwealth company under the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and, more specifically, a government business enterprise.4 To date, the Australian Government has provided NBN Co with contributed equity of $29.5 billion and has entered into a loan agreement for an additional $19.5 billion. NBN Co is accountable to the Australian Government through its shareholder Ministers and provides wholesale NBN telecommunications infrastructure and services to its customers (retail phone and internet service providers (RSPs)) that then sell access to the NBN to end users (the Australian households and businesses using the NBN).

Overview of the migration process

1.3 Maintaining service continuity throughout the migration process from a legacy telecommunication service to the NBN5 involves multiple parties delivering interdependent activities. A coordinated approach is required to manage:

  • communications regarding the migration process to an area that it is ready to be serviced via the NBN;
  • individual connections for premises to the NBN, enabling an active service; and
  • resolution of incidents, faults and complaints associated with the connections and activations process.6

1.4 NBN Co separates the process of migrating on to the NBN into four broad categories — Aware, Connect, Use, and Fix — that follow the linear chronological experience of migration from the perspective of the end user. NBN Co summarises its primary role within each of the migration phases as the following:

  • Aware: end user awareness or understanding of information needed to connect to the NBN and who to contact;
  • Connect: the end user experience during the process of being connected to an NBN service;
  • Use: the end user experience with regard to expectations on speed and reliability of the NBN connection; and
  • Fix: the end user experience where an issue has been reported about a service and parties are not satisfied with the resolution.

1.5 Defined in this way, the migration process does not change irrespective of the technology type being installed (see Appendix 2 for an explanation of the various technology types).7 The technology type does however influence the activities required to connect the service to a premises.8

Key stakeholders, legislation and policy governing the migration process

1.6 The key stakeholders involved in NBN migration activities and their associated roles are summarised at Table 1.1.

Table 1.1: NBN migration key stakeholders

Stakeholder

Role

Retail service providers (RSPs)

Third party providers of retail broadband services to end users using the NBN.

Delivery partners

Technicians contracted by NBN Co to complete NBN construction and connection assurance activities.

Department of Communications and the Arts (DoCA)

NBN policy lead on behalf of the Australian Government.

Australian Communications and Media Authority (ACMA)

The Australian Government regulator as it relates to broadcasting, radiocommunications, telecommunications and online content.

Telecommunications Industry Ombudsman (TIO)

Provides an independent external dispute resolution service for unresolved complaints about phone or internet services.

Australian Competition and Consumer Commission (ACCC)

Regulates access and non-discrimination obligations that apply to NBN Co on behalf of the Australian Government.

Telstra

Predominant owner of legacy telecommunications infrastructure being replaced or used to construct the NBN. Telstra is also the largest NBN delivery partner and RSP in terms of market share.

   

Source: ANAO summary.

1.7 A detailed explanation of the roles and responsibilities of each stakeholder in the migration process is at Appendix 3.

1.8 Numerous Australian Government and industry-led policies establish how NBN Co and its stakeholders should fulfil their respective roles in the NBN migration process. These are summarised in Table 1.2.

Table 1.2: Legislative and policy context for migrating a premises on to the NBN

Document

Purpose

Statement of Expectations

Sets out the Australian Government’s expectations of NBN Co as a Commonwealth company, including that NBN Co will work productively and collaboratively with stakeholders to provide a high quality end user experience through the migration, and it will work with retail service providers to proactively manage complaints.

Migration Assurance Framework

Developed by the DoCA in consultation with industry, the Migration Assurance Framework establishes principles that guide the migration process and sets out the agreed roles and responsibilities of various entities, including NBN Co, for a ‘seamless service transition to an NBN fixed line service for consumers and business’.

ACMA Industry Standards

Creates specific obligations for NBN Co in relation to service continuity during migration, and providing reasonable assistance to retail service providers in managing and resolving complaints. The Standards were developed following specific shortcomings with the migration process identified by the ACMA in 2017,9 and took effect from mid-2018.

Information Campaign on Migration Deed

Otherwise known as the Public Information on Migration or PIM. Sets out principles for NBN Co to abide by when communicating with end users. The Deed was signed between the Australian Government (through the DoCA) and Telstra that created the framework for Telstra’s participation in the NBN rollout.

Industry partner contracts

Contractual arrangements between NBN Co and contractors that engage technicians involved in the migration process (known as Operations and Maintenance Master Agreements), and between NBN Co and the RSPs for the supply of products and services (known as Wholesale Broadband Agreements).

Migration Plan

Sets out the steps that Telstra will take to progressively migrate voice and broadband services from its copper and Hybrid Fibre Coaxial networks to the NBN.

Communications Alliance Industry Guidelines

Developed and updated by telecommunications industry members, including NBN Co, in order to provide guidance for improving the end user experience when migrating on to the NBN, and to avoid issues such as double billing.

   

Source: ANAO summary.

NBN Co’s role in the migration process

1.9 According to the legislative and policy framework, NBN Co is responsible for specific aspects of the migration process and for the resolution of certain types of disruption to telecommunications service continuity that end users may experience. NBN Co does not have complete control of the end-to-end process for establishing an active NBN service at a premises — elements of the process must also be managed by end users themselves and RSPs, as illustrated in Figure 1.1.

Figure 1.1: Management of access to the NBN, and migration process summary

Note: This figure is a simplification of the actual arrangements for diagrammatic purposes only.

Source: ANAO adaptation of NBN Co documentation.

1.10 NBN Co’s central role is in facilitating the successful migration of phone and internet telecommunications services at a premises from legacy infrastructure to the NBN. NBN Co must do this in in a way that minimises disruption to the supply of telecommunications services to the end user.10 NBN Co is also responsible for undertaking public and industry awareness and education activities to support migration on to the NBN in order to maintain continuous access to a telecommunications service.

1.11 Once NBN Co declares that a location is ready for service, both household and business end users typically have 18 months to contact an RSP to indicate they are ready to connect their telecommunications services to the NBN, or to choose an alternate telecommunications infrastructure. The end of this 18 month window is known as the ‘disconnection date’. If the end user chooses to use the NBN, the end user’s RSP will contact NBN Co to request a connection and NBN Co then completes any necessary activities required at the premises to connect and activate a service.

1.12 Once a premises is connected with an active service on the network, NBN Co starts the process of wholesale billing the relevant RSP. An end user may require equipment to be installed (such as a modem or in-house wiring), however these activities are the responsibility of the RSP and the end user. Once a successful NBN service has been established, or the disconnection date has passed, NBN Co engages the owner of the legacy telecommunications infrastructure to arrange for its disconnection.

1.13 Within NBN Co’s span of control, a successful migration from legacy telecommunications infrastructure to the NBN is one where:

  • the end user is adequately informed to begin the connection process once the NBN is available at their location;
  • NBN Co works with its industry stakeholders to complete a connection within accepted timeframes;11 and
  • the connection remains active without any loss to continuous service12 at the premises for 30 days following a service being activated.13

Status of the upgrade and migration process

1.14 As at April 2019, NBN Co reporting suggests that 10,023,604 premises were classified as being ready for service. Of the premises that NBN Co has listed as being ready for service, NBN Co reporting indicates that 88.1 per cent14 have initiated the process to connect to the NBN. The remaining premises comprise end users who have:

  • not yet contacted an RSP to initiate a process of migrating to the NBN;
  • contacted an RSP to commence the migration process, but the RSP is yet to contact the NBN Co; or
  • elected not to migrate to the NBN at the current point in time.

Audit approach

Audit objective, criteria and scope

1.15 Under the Auditor-General Act 1997, as a Government Business Enterprise, performance audits of NBN Co can only be undertaken at the request of the Joint Committee of Public Accounts and Audit (JCPAA). A request from the JCPAA was received in October 2018 following an approach from the Auditor-General for such a request. The objective of this audit was to assess the effectiveness of NBN Co’s service continuity operations relating to the migration of telecommunication services on to the NBN.

1.16 To form a conclusion against the audit objective, the following high-level criteria were adopted:

  • Has NBN Co implemented effective strategies to inform and educate telecommunication end-users about the connection process?
  • Does NBN Co have effective processes to manage the risks to service continuity between an end user lodging a request to migrate to the NBN, and the end user having a fully operational telecommunication service?
  • Does NBN Co effectively manage end user connection-related complaints and drive improvement?

1.17 The audit scope focused on fixed-line technologies that will support voice and broadband internet telecommunications services for around 93 per cent of Australian premises.

1.18 The audit examined NBN Co’s operations from 16 March 2018 until 28 March 2019 in order to assess its effectiveness against contemporary industry settings. This timeframe is henceforth referred to as ‘the audit timeframe’.

Audit methodology

1.19 The audit methodology included:

  • examination of relevant NBN Co records;
  • analysis of available and relevant data;
  • discussions with relevant NBN Co employees; and
  • discussions with NBN Co stakeholders, including NBN delivery partners, retail service providers, the TIO, the ACMA, the ACCC and the DoCA.

1.20 The audit was conducted in accordance with ANAO auditing standards at a cost to the ANAO of $731,492.

1.21 The team members for this audit were Glen Ewers, Renee Hall, Shane Armstrong, Song Khor and Paul Bryant.

2. Strategies to inform and educate end users about connecting to the NBN

Areas examined

This chapter examines the effectiveness of NBN Co Ltd’s (NBN Co) strategies to inform and educate end users about the process of connecting to the National Broadband Network (NBN).

Conclusion

NBN Co strategies to inform and educate end users about the connection process are effective. Communication structures, channels and products have been developed with the aim of maximising end user understanding and awareness of the migration process, and these products satisfy the requirements for public information contained in the Public Information on Migration Deed. Substantial qualitative and quantitative data is collected, analysed and reported to key stakeholders on a regular, ongoing basis to monitor and evaluate the effectiveness of communication activities.

Areas for improvement

This chapter includes two areas for improvement for NBN Co in relation to the provision of an explanation for any delays in an area deemed ready for service, and the articulation of key roles and responsibilities in the migration process to end users.

2.1 In order to come to a conclusion against this criteria, the ANAO investigated:

  • the effectiveness of communication strategies—because NBN Co’s governance structures for communications, the methods and channels utilised, and the alignment of communication practices with the requirements for public information contained in the Public Information on Migration Deed are relevant to evaluating the extent to which end users are informed and educated about the process of connecting to the NBN; and
  • processes to monitor the effectiveness of communications—because NBN Co’s measures used to review the effectiveness of communications, and ongoing processes for reporting to key stakeholders, are important in ensuring that communication strategies effectively target the information needs of a diverse group of end users.

Has NBN Co developed effective communications strategies for the connection process?

NBN Co’s communication strategies for the connection process are effective. Strategic and process documentation reflect a high level of maturity in relation to the governance of communication activities. NBN Co has established a series of communication products and channels aimed at raising general awareness and understanding among end users of the NBN service, the process for the disconnection of legacy infrastructure, and the potential benefits of connecting to the NBN. Communications are appropriately targeted at specific demographics, and communication products are satisfying the Public Information on Migration Deed requirements.

Communications strategy and governance

2.2 Under the Information on Migration Campaign Deed (Public Information on Migration Deed, or PIM), the Australian Government required NBN Co to undertake a public information campaign in relation to the NBN which would satisfy a set of key principles (the PIM principles). The objective of the campaign is to:

…ensure to the maximum extent practicable that every end-user who is on a service to be migrated to the NBN receives advance notice of the planned migration, is familiar with the action required to migrate and, unless they make a decision that they do not wish to take an NBN service, initiates the necessary migration process prior to the disconnection of the relevant copper or hybrid fibre coaxial (HFC) service to their premises. The campaign will focus on the needs of the end-user to support an early migration and positive experience.

2.3 The PIM principles can be broadly separated into two categories that focus on end user awareness, and NBN Co engagement with the telecommunications industry (as summarised in Table 2.1).15

Table 2.1: Summary of PIM principles

Principles that relate to informing end users about migration on to the NBN

That NBN Co communicate to end users:

  • in advance of planned migration and the actions required to migrate;
  • NBN rollout commencement or delays in a particular area;
  • the consequences of deciding to migrate from legacy infrastructure on to the NBN, and deciding not to migrate;
  • the effect on telecommunication services currently available once NBN infrastructure has been made available in an area; and
  • the respective responsibilities of different parties in the migration process.

Principles that relate to industry engagement, including communications effectiveness review

That NBN Co collaborate and act as an industry leader by:

  • providing consistent messaging to retail service providers (RSPs) and other industry participants to inform their end user communications;
  • collaborating with industry to develop and continuously improve communications to end users;
  • agreeing annual targets to inform NBN Co’s communications strategy; and
  • neutrality in relation to influencing end users regarding their choice of RSP.

Source: ANAO summary.

2.4 The NBN Co Corporate Plan 2019-22 outlines NBN Co’s objectives in relation to end user communications as part of the ‘Aware’ stage of the migration process, which is to:

…help Australians better understand what they need to do before signing up to retail services, clarify the role of NBN Co and RSPs, and support end users to get the best out of their broadband service, including education in relation to factors such as speed.16

2.5 The NBN Co Marketing, Channels and Partnerships (Marketing) business unit has primary accountability for NBN Co’s communications objectives. The Marketing business unit is led by an Executive General Manager reporting to the Chief Customer Officer Residential17 who reports to the Chief Executive Officer (refer to Appendix 4 for a summary NBN Co organisational chart). Clear responsibilities and performance measures have been established within the Marketing business unit to achieve its communications objectives.

2.6 A communications and marketing strategy has been articulated outlining the key processes and channels NBN Co utilises to undertake communications activities. This strategy is informed by data collected on measures of end user awareness of the NBN, the reach of NBN Co’s communications, as well as NBN connection uptake levels to determine the locations of end users, and those that are yet to make the decision to migrate on to the NBN.

2.7 Paragraph 2.22 outlines NBN Co’s process for the ongoing review of the effectiveness of communications strategies.

Communication methods

2.8 NBN Co communicates with end users directly and through RSPs.

Direct communications with end users

2.9 NBN Co has established direct mail (letters and postcards) and the NBN Co website (including the rollout map18) as the foundation communication channels for end users seeking information about migrating on to the NBN, with the objective that end users could review the information at a time suitable to them. This base level information is supplemented by broad-scale media activity such as television, radio, social media, and community engagement. NBN Co also communicates with end users via delivery partners and ‘experiential’19 communications. NBN Co’s multi-channel media approach is designed to ensure that coverage and education processes are nationwide.

2.10 NBN Co communications are pitched generically and apply to as many migration scenarios as possible. NBN Co communications frequently refer end users to their RSP as the entity with which the end user has a direct contractual relationship for service provision, and who are therefore best placed to discuss the end user’s specific circumstances.

2.11 NBN Co uses an external media and advertising agency20 to inform its communications aimed at specific demographics (such as regional areas, culturally diverse communities, and high density apartments). The agency uses demographic analysis and other proprietary tools to determine which geographic areas receive targeted communications, and what channels are most suitable. NBN Co’s direct mail, website, television and radio communications also provide options for readers from a non-English speaking background or who have a disability or impairment to access relevant information. NBN Co reporting indicates that its communications are increasing awareness of the connection process among culturally and linguistically diverse populations. Details on the processes used to measure this awareness are provided in paragraphs 2.22 to 2.30.

2.12 NBN Co forecasts the timing of premises becoming ready for service using a database of information that also determines when to commence sending a series of direct mail communications (or emails for those who register online) about the NBN.21 NBN Co sends mail directly to premises from 20 months prior to its disconnection date, through to the time that a connection is activated,22 or two months after the disconnection date has lapsed.23 Direct mail is sent at specific times in the lead up to the disconnection date for a premises, and varies in messaging depending on how far away this disconnection date is. Up to six individual pieces of direct mail correspondence may be sent over the 20 month period depending on when an end user chooses to initiate migration on to the NBN. Detail on the direct mailing sequence is shown in Appendix 5.

2.13 Since October 2018, letters and postcards distributed by NBN Co have either focussed information on the upcoming disconnection of legacy infrastructure, or on the benefit of connecting to the NBN. Irrespective of which headline messaging is used, all mail reviewed as part of the audit consistently conveyed the need for end users to migrate from legacy infrastructure to avoid being left without telecommunications services, and to contact an RSP to obtain further information.

Communications with end users through RSPs

2.14 The Telecommunications (Consumer Information) Industry Standard 2018 requires that RSPs meet minimum requirements to consistently communicate with end users about the NBN. NBN Co assists RSPs to meet these requirements by providing them with:

  • samples of NBN Co communications;
  • templates for RSPs and other industry participants to facilitate clear and consistent messaging when communicating with end users;
  • data that NBN Co uses to inform when its communications will commence; and
  • analysis that monitors how end users are responding to rollout uptake and sentiment.

2.15 NBN Co provides RSPs with visibility of roll-out and marketing activity in target areas on a weekly basis. NBN Co conducts quarterly briefings with RSPs and the Department of Communications and the Arts (DoCA) to discuss NBN Co’s marketing and the impact of this marketing on end user sentiment and behaviour.

2.16 NBN Co has been transparent in its engagement to assist RSPs to meet communication requirements under the Telecommunications Industry Standard.

Alignment of NBN Co communications with PIM requirements

2.17 The PIM outlines a set of principles that NBN Co must communicate to end users regarding the migration process via its public information campaign. Table 2.2 provides a summary of these principles, lists the various NBN Co marketing channels and how each is aligned to address the principles, and shows the ANAO assessment NBN Co’s overall coverage of each PIM principle.

Table 2.2: ANAO assessment of NBN Co communication practices and coverage of the PIM principles

Summary of PIM principles

Delivery partners

Direct mail (and email)

Online (inc. nbn.com.au)

ATL (advertising)

Community engagement

Experientiala

Social media

ANAO assessment

That NBN is being rolled out in their area

That the NBN rollout will involve the physical presence of construction crews, and the nature of activities they will be involved in

 

 

That NBN will require the migration of traditional landline phone and internet services

The timeframe for migration

 

 

 

 

 

 

The effect on telecommunication services currently available, including that:

  • end users will be informed that a separate disconnection regime will apply to special services;
  • end users will be informed that they need to take additional steps to enable their legacy services to continue to function when their premises are migrated to NBN

 

 

 

The consequences for existing end users if they do not take action before the mandatory disconnection of existing fixed line phone and broadband services

 

 

The respective responsibilities of different parties in the migration process, including the role of RSPs in providing services over NBN

 

 

 

 

Where there has been no action taken by an end user in relation to an existing service address after the physical infrastructure is serviceable or approaching disconnection in their direct locality, to reiterate end user responsibilities in connecting to NBN

 

 

 

 

Where premises within the NBN fixed line footprint are not currently NBN serviceable, to inform end users that the NBN is not currently available

 

 

 

 

NBN will require residential and business customers to migrate any traditional fixed line phone and broadband services

The rollout will be taking place progressively in multiple areas across Australia

 

 

 

Arrangements for premises located in the NBN fixed line footprint will not apply to non-premises

 

 

 

 

 

Where there are unreasonable delays in the availability or delivery of services within an area that has been declared by NBN Co to be Ready For Service

 

 

 

 

 

         

Key:  = the communications channels that NBN Co have used to implement the related PIM principle summary. = where ANAO has reviewed implementation and found the related PIM requirements have been satisfied.

Note a: NBN Co’s communications matrix indicates that community engagement and experiential communication channels (footnote 19) meets all of the PIM requirements. To achieve these results end users would be required to know what to ask in order to be informed in accordance with the intention of the PIM principles.

Source: ANAO analysis of NBN Co documentation.

2.18 NBN Co’s communications practices have satisfied the PIM principles. The following opportunities for improvement are noted:

  • While NBN Co aligns its direct mail distribution and rollout map information with expected rollout dates for a particular location, it does not provide an explanation of any delays in an area deemed ready for service.24 Analysis of complaints lodged with NBN Co indicated that the second largest category of complaints was ‘Rollout Maps/Timeframe’25 at six per cent. While some complaints may be attributed to end users being unsatisfied with a rollout date when it has not changed, they may also be associated with unreasonable delays. Management of complaints is further discussed in Chapter 4.
  • Opportunities exist for NBN Co to further improve the articulation of key roles and responsibilities in the migration process to end users. NBN Co’s direct mail26 and webpage provide high level advice on its role as distinct from that of RSPs, however key industry responsibilities for NBN migration are commonly misunderstood. Direct mail does not explain why NBN Co was sending communications, as opposed to an RSP.

Does NBN Co have a process to review the effectiveness of communications strategies on an ongoing basis?

NBN Co has established processes to monitor and review the effectiveness of its communications channels in educating and informing end users about the NBN connection process. Key measures in NBN Co’s annual operating plans are used to monitor the effectiveness of communications, and ongoing reporting and dialogue among key stakeholders provide the opportunity to validate these measures.

Policy requirements

2.19 The PIM principles include requirements to establish a process of ongoing monitoring and review of NBN Co communications strategies and activities, as illustrated in Table 2.1. These principles are also reflected in the Migration Assurance Framework, which states that:

NBN Co is responsible for working with community and peak industry body groups to seek input as part of a continuous improvement process focused on maximising service continuity and facilitating a seamless migration experience for customers’.27

2.20 The Migration Assurance Framework further refers to the following:

…as part of a continuous improvement process, the Public Information on Migration Governance Framework allows community, industry and government groups the opportunity to provide feedback on Public Information on Migration activities.28

2.21 The PIM also includes a requirement that:

In developing the [public interest] campaign strategy NBN Co will agree measurable targets with the Commonwealth and Telstra. A baseline awareness level will be identified at the start of the migration process and changes monitored.

Monitoring, review and evaluation of NBN Co communication strategies

2.22 NBN Co’s primary methods for monitoring, reviewing and evaluating the effectiveness of its communication processes are through post-communication take up analysis, customer experience survey activity, and qualitative message testing.

2.23 NBN Co undertakes analysis of the effectiveness of direct mail in supporting end user decisions to migrate to the NBN. This involves reviewing actual uptake against rollout forecasts within 30 days of direct mail being sent to a particular area. NBN Co conducts this analysis on a monthly basis.

2.24 NBN Co conducts customer experience surveys of residential and business end users on a quarterly basis. An external provider is utilised to conduct 20 minute telephone surveys each quarter of: 1,800 residential end users in ready for service areas that are connected, not connected and outside of rollout; 500 business end users that are connected and have not connected; and 400 fixed wireless29 end users that are connected and have not connected. As part of these surveys, NBN Co tests the following:

  • understanding: customers knowing the need to switch in order to retain service;
  • readiness: customer readiness to connect30;
  • intent: customer urgency to connect; and
  • sentiment: customer feelings towards the NBN.

2.25 Surveys results, key learnings and proposed actions are shared with RSPs through quarterly briefings. NBN Co survey results during the audit period are illustrated in Figure 2.1.

Figure 2.1: Percentage of NBN Co survey results on end user awareness (a) business and (b) residential

(a) business

 

(b) residential

 

Source: ANAO analysis of NBN Co data.

2.26 Figure 2.1 indicates that for both business and residential end users surveyed, a high proportion understood the need to switch in order to retain their telecommunications service. This indicates that NBN Co has satisfied its PIM requirements to educate and inform end users to know the need to switch. The survey results also indicate that end user readiness and urgency to connect remains low. While the readiness and urgency to connect are not requirements under the PIM, NBN Co works with RSPs to monitor and nuance communications in response to negative or inaccurate end user sentiments, with the intention of increasing end user sentiment towards the NBN. NBN Co analyses the possible causes of end user survey results to inform modifications to marketing activity.

2.27 NBN Co also uses market research to measure and calibrate its communications to maximise its appeal and accessibility to end users.

2.28 In response to results from both the customer experience surveys and qualitative message testing, NBN Co identifies and executes strategies, which are aimed at addressing deficiencies in end user understanding, and modifies existing communication processes to better satisfy end user knowledge requirements.

Measuring and reporting on the performance of NBN Co communication strategies

2.29 NBN Co has a series of measurable targets in its annual operating plans to monitor the impact of its communications. Specifically:

  • market metrics, with targets for residential and business end users in relation to the understanding, readiness, intent and sentiment data generated from the customer experience surveys, the results of which as shown in Figure 2.1; and
  • operational metrics, with targets of less than one per cent for the number of direct mailings that are sent to the incorrect address (‘erroneous mailings’) and less than five per cent for the number of mailings which are not sent out in a timely manner (‘delayed mailings’). NBN Co’s most recent reporting on erroneous delivery of direct mail reflects rates of less than one per cent. NBN Co did not meet its delayed mailing KPI over the same timeframe, and attributed this to a cease in mailing over the Christmas period that resulted in a significant increase in mailing activity in January.

2.30 While NBN Co could not provide documentation to substantiate that it formally agreed these targets with the Australian Government and Telstra, as required by the PIM, performance against these targets is reported quarterly to RSPs (including Telstra) and the DoCA allowing for any issues with the targets to be raised.

3. Management of disruption to service continuity

Areas examined

The audit examined NBN Co Ltd’s (NBN Co) management of service continuity for end user telecommunications service during the process of migrating on to the National Broadband Network (NBN), responsiveness of assurance services to fix faults when they occur within 30 days of a service being activated, and how these are reflected in NBN Co’s risk management.

Conclusion

NBN Co has established largely effective processes to manage service continuity between an end user lodging a request to migrate to the NBN, and the end user having a fully operational service. Structures and controls have been implemented to manage common service disruption issues during the connection and activation process, and frameworks have been established for ongoing operational engagement with RSPs. Clear processes have been established for the diagnosis, reporting, investigation and repair of faults, with ongoing work to increase the timeliness of fault resolution. NBN Co’s risk framework is partly effective in supporting the management of service continuity risks at the operational level.

Area for improvement

This chapter includes one recommendation for NBN Co to update its risk register to cover all aspects of managing service continuity, and an opportunity to improve its compliance monitoring with the Service Continuity Industry Standards.

3.1 To form a conclusion against this criteria, the ANAO investigated:

  • NBN Co processes to prevent risks to service continuity when connecting and activating services to the NBN—because the establishment of clear roles and responsibilities and effective processes and controls for managing NBN Co activities during the connect phase are important elements in maintaining service continuity;
  • NBN Co operational performance against targets for resolving faults that occur within 30 days of a new connection—because the timely resolution of any faults arising during the migration process is critical in minimising interruptions to service continuity; and
  • how service continuity is integrated into NBN Co’s risk management framework—because the significant volume and scale of NBN Co operations requires that service continuity risks are effectively identified, managed and monitored at an enterprise level.

When connecting and activating services to the NBN, has NBN Co effectively prevented risks to service continuity over which it has influence or control?

NBN Co has established processes that are effective in preventing risks to service interruption during the process of connecting and activating a service within its span of control. NBN Co has established clear, detailed procedures and operational structures to facilitate the process of migrating a premises on to the NBN. Governance arrangements support effective engagement with RSPs, including ongoing reporting and communication activities.

Connect phase roles and responsibilities

3.2 Once an end user is aware that their premises is ready for service and contacts a retail service provider (RSP) to access the NBN, NBN Co is responsible for working with its industry stakeholders to complete a connection within accepted timeframes.31 For NBN Co, the ‘Connect’ phase of migration is the process of installing NBN Co equipment to connect an end user’s premises to the NBN infrastructure and activating the service. The newly connected and activated service is then linked to the end user’s RSP.

3.3 Within NBN Co’s span of control, a successful migration has been defined for the purposes of this audit to be one where the service remains active without any loss to continuous supply of telecommunications32 at the premises for 30 days after it has been activated. Disruption or loss of service during this timeframe may include:

  • when a connection to the NBN is attempted and is unsuccessful;
  • during a network outage; or
  • when a service fault occurs.

3.4 Once a premises has an active NBN service, the ‘Use’ phase of migration commences when NBN Co has notified the RSP of an operational NBN service and the service is available to the end user. Faults that NBN Co manage during the ‘Use’ phase are discussed in paragraphs 3.17 to 3.47.

Key industry standard requirements

3.5 The Telecommunications (NBN Service Continuity) Industry Standard 2018 (Service Continuity Industry Standard) sets out a number of responsibilities for carriers, such as NBN Co, and service providers to prevent an end user experiencing a disruption of their telecommunications service during the ‘Connect’ phase. Under section 8 of the Service Continuity Industry Standard, NBN Co ‘must take all reasonable steps to manage the migration at a premises in a way that minimises disruption to the continuous supply of carriage services to the consumer’.

3.6 Under subsections 8(2)(a) and (b) of the Service Continuity Industry Standard, NBN Co must also:

  • not advise an RSP or owner of the legacy telecommunications infrastructure that the migration at the end users premises is complete until it has taken all reasonable steps to ensure that successful migration at the premises has occurred.
  • where the migration at the end users premises has been unsuccessful, take all reasonable steps to notify, within one working day, the end users’ RSP that migration to the NBN service has not been successful.

NBN Co processes and controls for managing service continuity

General processes and controls

3.7 Once serviceability and specifications of an order at a particular premises is known, the NBN Co Customer Connections Business Unit coordinates with its delivery partners to install required equipment,33 and establish a connection at the location.

3.8 A series of service classes are used by NBN Co to categorise individual premises based on the serviceability and the equipment requirements for connection. NBN Co defines 34 service classes, of which 28 are related to the fixed line technologies within the scope of this audit (see Appendix 2). For a newly connected premises, NBN Co is also responsible for activation of the service, and when NBN Co confirms an active service for a location it is linked to an RSP for billing purposes.34

3.9 NBN Co has established clear, detailed procedures and operational structures to facilitate the process of connecting end users to the NBN. Specifically to:

  • validate connections orders that it receives from the RSP;
  • complete NBN connections, including the resolution of any issues by the technician required before a connection can be activated; and
  • run tests while on site, with controls in place to ensure that work tickets are not closed without passing relevant checks.
Notifications to RSPs of an unsuccessful NBN connection

3.10 The Service Continuity Industry Standard took effect from 21 September 2018. Subsection 8(2)(b) of the Service Continuity Industry Standard (discussed in paragraph 3.6) establishes a requirement that where an NBN migration at a premises has been unsuccessful, NBN Co must notify the end user’s RSP within one working day. This notification process aims to manage the risk of service disruption at an end user’s premises by enabling NBN Co and the RSP to coordinate with the end user in a timely way, as well as determine the most appropriate action when an unsuccessful migration appointment occurs.35

3.11 From January to April 2019, NBN Co developed a formal framework for managing compliance with the Service Continuity Industry Standard. Procedures were also updated to reflect implementation of a new notification for when an unsuccessful migration attempt warrants NBN Co action. A dedicated compliance monitoring team was established in May 2019 with responsibility for conducting ongoing testing of compliance with the Service Continuity Industry Standard. However, this team only measures compliance in relation to RSP notifications where NBN Co is directly responsible for any subsequent action following an unsuccessful appointment. This means that the following notification types are not assessed by NBN Co as part of its compliance review process:

  • notifications that inform an RSP that RSP action is required after an unsuccessful appointment; and
  • notifications that clarify responsibilities following an outstanding appointment notification.36

3.12 While NBN Co’s procedures reflect the need for these notifications to be issued, NBN Co should revise the testing methodology used by the Service Continuity Industry Standard assurance team to ensure that there is appropriate monitoring of compliance with timeliness requirements for these notifications.

Controls to manage network events and outages during the connection process

3.13 Scheduled or unscheduled network outages may cause an interruption to an NBN service, and this may occur during the time that an end user is being migrated from legacy infrastructure on to the NBN. Under the Wholesale Broadband Agreement (WBA)37, NBN Co is required to communicate scheduled outages for maintenance and assurance activity to RSPs in advance, and this is undertaken through outage activity notifications. During the audit timeframe, the volume of service faults recorded that were a result of an outage within 30 days of an activated service were low (approximately 1.7 per cent of all service faults).

3.14 NBN Co has stated that it plans to further increase its capacity to understand the impact of network events on particular premises during migration via post connection reports and through conducting live health checks38 on a service.

General governance, reporting and communication processes with RSPs

3.15 Under the WBA, NBN Co is required to conduct governance meetings and performance reporting39 with RSPs on a monthly basis. NBN Co engagement with 20 RSPs reviewed during the audit timeframe indicated inconsistency in the extent to which records were maintained of governance meetings and outcomes.40 NBN Co stated that it offers the same governance approach to all RSPs, however the structure and timing of meetings and associated processes is then mutually determined with each individual RSP. Examination of RSP reporting indicates that NBN Co provides sufficient information to support monitoring and discussion of any performance issues and proposed improvement measures as they are necessary. Meeting records that were available indicate that representatives from the NBN Co connections and assurance areas attend all meetings. Specific issues relating to the management of performance for particular technologies, connection and activation timeliness, appointment scheduling and delivery partner capacity were regularly discussed, including:

  • faults closed per 100 active premises;
  • ‘no fault found’ volumes and trends;
  • standard connections timeliness, causes of lower performance, and opportunities to raise performance levels; and
  • priority assistance connections within agreed timeframes.

3.16 NBN Co has governance arrangements in place with RSPs in accordance with the WBA, and had more frequent and detailed performance discussions with those RSPs that have the most NBN market share. NBN Co reporting suggests that its performance discussions with RSPs include statistics on fault volumes and action taken to address them, and for some RSPs, analytics of what is contributing to missed or achieved service level agreements.

Does NBN Co resolve service faults associated with a newly connected NBN service in an effective and timely manner?

NBN Co’s resolution of disruption-related service faults on newly connected services is not fully effective, because NBN Co is not meeting targets for the resolution of service faults within agreed timeframes. NBN Co has established clear processes to diagnose, report, investigate and repair faults, and there is a downward trend in the volume of disruption-related service faults occurring within 30 days of a newly connected service, despite an overall increase in the volume of new connection activity over the same period. The average time taken to resolve service disruption faults on newly connected services is approximately 3.1 work days, and 80 per cent of faults are resolved within four days or less. However, NBN Co performance reporting reflects that 89 per cent of standard connection service faults are being resolved within agreed timeframes against a target of 90 per cent, and 95 per cent of priority assistance service faults are being resolved within agreed timeframes against a target of 100 per cent. A program of work is underway to improve service assurance and fault resolution, including a focus on reducing the number of faults and the instances where technicians need to be deployed.

Service fault volumes and trends

3.17 The ‘Use’ phase begins when an NBN service has been connected and activated by NBN Co. When an end user experiences a loss of service during this phase, the circumstances are raised by the RSP and logged with NBN Co as a type of incident,41 otherwise known as a service fault. A total of 1.22 million service faults were recorded by NBN Co in the audit timeframe.42

3.18 Service faults that occur within 30 days of a new connection to the NBN were the focus of analysis in this chapter,43 as they can indicate how well the connection has been installed and whether the ‘Connect’ phase of the migration was successful.44 Of the 1.22 million service faults in the audit timeframe, 127,464 (approximately 10 per cent)45 occurred within 30 days of a newly connected service.

3.19 Service fault types can be divided into:

  • disruption faults—those faults where there is no functionality in the NBN service itself for a period of time, specifically: new service never worked,46 no data, intermittent service/dropouts, and no sync faults;47 and
  • non-disruption faults—those faults where the NBN service is functional, but the quality of the service is impacted, specifically: faults related to speed, speed degradation and performance latency.

3.20 The average weekly volume of new connections over the audit timeframe (16 March 2018 to 28 March 2019) was 40,353. Figure 3.1 illustrates the volume of faults occurring within 30 days of a newly connected service over the same period, divided into disruption and non-disruption categories.

Figure 3.1: Volume of service faults within 30 days of newly connected services

Notes: The records for service faults (recorded as customer incidents in NBN Co data) included those for priority assistance services. Priority assistance services are explained in Table 3.3.

The disruption faults are for all fixed line technology types.

Trend line is for visual representation of general trend only.

Source: ANAO analysis of NBN Co data.

3.21 Figure 3.1 reflects the following:

  • There is a general downward trend in the volume of disruption faults within 30 days of newly connected services. This is a positive result given there was a general increase in the volume of weekly new connection activity over the same period (from 37,443 new connections in the week ending 22 March 2018 to 44,605 new connections in the week ending 28 March 2019 (refer to Figure 4.3).
  • Disruption faults form the vast majority of service faults that occur within 30 days of a new connection.

Fault management process

3.22 NBN Co has established the WBA operations manual, which describes the procedures that apply to the operational aspects of the supply of products by NBN Co to each RSP under the WBA. Responsibilities and interactions between NBN Co and RSPs for resolving faults48 that occur on an NBN service once it has been connected are clearly outlined in this document.49

3.23 A service portal and a ‘Business-to-Business’ interface have been established, and are the two key avenues for NBN Co to share information and facilitate assurance activities over the connection process with RSPs.50 In the first instance, the RSP must perform test and diagnostic tasks to identify whether a fault is a service fault. If an RSP determines after diagnostic testing that the issue is likely to be a service fault, it raises a ‘trouble ticket’51 with NBN Co for resolution. In response, NBN Co will diagnose, report, investigate and repair the fault through rectification or remediation, (see Figure 3.2).

Figure 3.2: End user service fault management process

Source: ANAO analysis of NBN Co documentation.

Fault diagnosis

3.24 Table 3.1 shows NBN Co diagnosis of disruption fault types within 30 days of a newly connected service over the audit timeframe.

Table 3.1: Disruption fault types within 30 days of a newly connected service

Fault type

Volume of faults

Proportion of total faults (%)

No data

21,892

18.3

Intermittent service/dropouts

9,828

8.2

No synca

66,967

56

New service never worked

20,851

17.4

Total

119,538

100

     

Note a: ‘No sync’ refers to circumstances where there is a problem in the premises which impact on connectivity with the NBN network (for example, a fault with the modem, or the wiring within the premises itself).

Notes: There were 378 service faults in the NBN Co dataset with a status of ‘cancelled’ that contained no categorisation for fault type.

Table 3.1 represents service disruption faults only, which is a proportion of the 127,464 total service faults within 30 days of a new connection shown in Figure 3.1.

Percentage totals may not add to 100 per cent due to rounding.

Source: ANAO analysis of NBN Co data.

3.25 Disruption fault types ‘no data’, ‘intermittent service/dropouts’ and ‘no sync’ relate to situations where the NBN Co connection process has occurred, however functionality of the service was then lost for a period of time. When an end user reports that their service has had no functionality since the connection and activation process occurred, it is classified as a ‘new service never worked’ fault type, and potentially reflects that the NBN connection process has not been effectively undertaken.52

3.26 Within the audit timeframe, NBN Co internal reporting for all technology types53 stated that approximately three per cent of new connections did not work in the first 30 days, against a KPI of 2.7 per cent as illustrated in Figure 3.3.

Figure 3.3: Trend in ‘new service never worked’ faults within 30 days of a new connection

Source: ANAO analysis of NBN Co data.

3.27 Figure 3.3 shows that, while the rate of new service never worked faults as a proportion of new connection activity is above NBN Co’s long term KPIs, this rate shows improvement in the five months to March 2019.

3.28 A total of 43,187 new service never worked fault types were recorded by NBN Co within the audit period. Figure 3.4 illustrates this fault type by technology type (service category).

Figure 3.4: Volumes by technology type of (a) NBN connections activity; and (b) new service never worked faults

(a) Volumes of new connections by technology type

 

(b) Volumes of new service never worked faults

 

Source: ANAO analysis of NBN Co data.

3.29 Figure 3.4(a) shows that the largest volume of new connections are in the ‘Fibre to the Basement’ and ‘Fibre to the Node’ (FTTPB/N) technology types, and Figure 3.4(b) shows that these generate the majority of new service never worked faults. Figure 3.4(b) shows that, following an initial increase, the volume of new service never worked faults for the FTTB/N technology types decreased over the audit period, although it is unclear whether this is driven by a more general decrease in the volume of new connections in these technology types. Figure 3.4 does show that, despite a general increase in the volume of connection activity for Hybrid Fibre Coaxial (HFC) connections, the volume of new service never worked faults remained low.

3.30 Although new service never worked faults are initially received by NBN Co’s assurance area, once the fault type is confirmed, it is managed by the connections area to complete the connection again. The metrics on this scenario are reported internally as ‘right second time’. NBN Co operational reporting in May 2019 indicates that it had achieved a financial year to date outcome of 98 per cent against a target of 9754 per cent for its ‘right second time’ connection metric.

Fault resolution

3.31 As illustrated in Figure 3.2, outcomes of NBN Co’s fault resolution process can result in it being classified as being ‘no fault found’, a requirement for a technician work order (otherwise known as a ‘truck roll’ to the premises), or a remediation. For all service faults within 30 days of a new connection during the audit timeframe, 30.5 per cent resulted in ‘no fault found’, as shown in Table 3.2.

Table 3.2: Outcomes of all service faults within 30 days of a new connection

Outcome

Volume of service faults

Proportion of total service faults

No fault founda

36,463

30.5

NBN Co resolved remotely

260

0.2

Fault found – technician work order required (truck roll)

62,027

51.9

RSP related issuesb

8,559

7.2

Network related issuec

2,051

1.7

Otherd

10,178

8.5

Total

119,538

100

     

Note a: No fault found can be as a result of line testing, where NBN Co determines the service is working to specification, or after a technician has been to a premises and confirmed a fully operational service. In these instances the reported fault may be due to the end user.

Note b: These faults outcomes indicate action by the RSP or are related to RSP coordination with the end user. These resolution categories are labelled as ‘AS’ indicating access seeker, or ‘3rd party’. Examples of these resolution categories are: ‘AS – No Response Received’; ‘AS – End User Unavailable’; ‘3rd Party – Remote Configure’ and ‘3rd Party – Fault Fix’.

Note c: These service faults have been determined by NBN Co to be related to a planned outage or resolved as a network fault.

Note d: Examples of other service fault resolution categories in NBN Co data include those labelled with ‘restoration’, ‘blank’, ‘potential fault cleared’ or ‘other fix’.

Note: Percentage totals may not add to 100 per cent due to rounding.

Source: ANAO analysis of NBN Co data.

3.32 Table 3.2 shows that although the majority of faults received relate to service disruption, a large combined proportion (37.7 per cent) are related to RSPs or associated issues with end users, and are therefore not within NBN Co’s direct influence. Once NBN Co has resolved a fault, the outcome is communicated to RSPs to manage with the end user.

Timeliness to resolve

3.33 Timeliness for resolving faults is agreed in the WBA and determined by:

  • premises location (specifically remoteness);
  • technology type;55 and
  • the category of end user connection.

3.34 The agreed timeliness for service fault rectification in the WBA is based on these factors, and range from one to three days.

3.35 The two categories of end user connections defined in the WBA are ‘Standard’ and ‘Priority Assistance’ services, as explained in Table 3.3.

Table 3.3: End user connections categories

Category

Description

Standard connection

A connection and activation at a premises which is not a service transfera or an accelerated connectionb or a priority assistance service function.

Priority assistance connection

‘Priority assist’ means assistance supplied by an RSP to an end user who suffers, or has a member of their household who suffers, from a diagnosed life threatening medical condition and is eligible for priority assistance in accordance with the relevant Industry Code or any carrier licence condition. The connection and activation of components on the basis that they will be used as an input to supply a priority assistance service for an end user, such as medical assistance alarms, fire alarms and elevator emergency phones.

   

Note a: Service transfers occur at premises already connected to the network where an end user changes their preferred RSP. Service transfers are not migrating an end user off a legacy service to connect to the NBN.

Note b: Accelerated connections are for premises that are already connected to the network but inactive for the end user and the RSP has proposed to provide a standard telephone service. If the premises has previously been migrated to the network, RSPs should not order accelerated connections unless the order meets the required conditions.

Source: NBN Co documentation.

Resolving standard service faults

3.36 The WBA establishes that the KPI for faults on standard connections for fixed line technologies is 90 per cent or more will be resolved within agreed service level timeframes (i.e. between one and three days). Operational metrics on faults information are reported weekly to the NBN Co executive.56 For the proportion of all service faults that were ‘resolved within WBA SLA due date’, NBN Co reported an average result of 89 per cent for the audit timeframe.

3.37 The method NBN Co uses to report on fault resolution timeliness excludes time associated with matters outside of its control (for example, where a technician attends an end user premises, yet the end user misses the appointment). The end user experience of timeliness to resolve a fault may therefore be longer than what is calculated and reported by NBN Co.

3.38 In relation to the end user’s experience, the average elapsed work days taken to resolve faults relating to service disruption within 30 days of a newly connected standard service57 is shown in Table 3.4.

Table 3.4: Average elapsed work days for resolving service disruption faults for a standard connection from an end user perspective

Region type

FTTN/B

FTTC

FTTP

HFC

Remote

3.3

4.1

3.5

N/A

Minor rural

4.0

4.2

3.8

N/A

Major rural

3.8

4.0

3.6

N/A

Urban

2.9

3.3

3.2

3.7

         

Note: HFC technology is not available outside urban geographies.

Source: ANAO analysis of NBN Co data.

3.39 Table 3.4 indicates that there is no substantial variation in the average elapsed work days taken to resolve service disruption faults based on technology type or the geographic location of a premises. The average time experienced by the end user for having a service disruption fault resolved (shown in Table 3.4) is 3.1 working days, and 80 per cent of these faults were resolved in four days or less. A breakdown of fault resolution volumes over time by geography and technology is at Appendix 6.

Resolving priority assistance service faults

3.40 The WBA KPI for faults on priority assistance services for fixed line technologies is that 100 per cent will be resolved within agreed service level timeframes, which range from 24 to 48 hours.58

3.41 NBN Co internal reporting covering the audit timeframe indicated that the reported average result for the resolution of priority assistance service faults within agreed service level timeframes was 95 per cent against a target of 100 per cent.

3.42 For the audit period, only 1.8 per cent of priority assistance connections resulted in a service fault.59 On average, this small number of faults were resolved within the next business day.

3.43 NBN Co introduced an initiative in November 2018 for priority assistance connections to improve its performance.60 The initiative involves assurance measures such as line testing the service each day post-activation for 30 days. The introduction of the initiative has improved NBN Co’s performance towards its 100 per cent KPI for resolving faults relating to priority assistance services.61 Individual case management and high level handling of priority assistance connections orders have also contributed to the improvement of NBN Co’s performance against this measure.62

Continuous improvement

3.44 NBN Co is implementing an assurance improvement and maturity program that has included the introduction of a dedicated area for resolving problematic incidents, and has a current focus on reducing the number of ‘truck rolls’63. The components and progress of this program is summarised in Table 3.5.

Table 3.5: Assurance improvement program components

Components

Status

Create a case management function for managing aged, escalated, rejected and repeat incidents

Complete

Implement a program for reducing the number of truck rolls and pattern managementa

Underway

Reduce incidents/faults within 30 days by 50 per cent

Planned

Introduce quality evaluations on 10 per cent of all new services never worked

Planned

   

Note a: Pattern management is an advanced operations area that analyses the network and faults for patterns where similar faults may be occurring. By conducting root cause analysis NBN Co intend to facilitate the reduction of unnecessary truck rolls. NBN Co advised that pattern management is intended to become a formal program of work for NBN Co in 2020.

Source: NBN Co documentation.

3.45 The case management function was created in December 2018 to address problematic or repeat incidents, including ‘aged incidents’ which are classified as faults that have not been resolved after seven days. NBN Co operational reporting on aged incidents does not distinguish if the incident relates to service faults, network faults or were particularly related to speed or disruption to service. NBN Co reported that for the 12 weeks from 14 February to 25 April 2019, it was managing approximately 1,100 aged incidents per week.

3.46 Reducing the number of truck rolls required for resolving faults is important for efficiency, expenditure and impact on the end user. NBN Co has a current focus on reducing the number of truck rolls, particularly those that result in ‘no fault found’. It is noted however that NBN Co executive reporting on 6 June 2019 indicated that reducing truck rolls relating to faults within 10 days of activation is a priority for NBN Co to improve.

3.47 Metrics for repeat incidents within 30 days of a new connection and repeat incidents within 30 days of a new connection that also required a truck roll are included in operational monitoring. From February 2019 to April 2019, NBN Co operational reporting of 12 week averages indicates that repeat incidents in 30 days (two or more faults) occurs for 18 per cent of locations reporting a fault in the first instance; a KPI has not been set for this metric. For the same quarterly time period, NBN Co also reports that repeat incidents that also required a truck roll was six per cent of locations reporting a fault in 30 days, which is in line with NBN Co’s current target for this metric.

Does NBN Co’s risk management framework support NBN Co operations for management of service continuity?

NBN Co’s risk management framework is partly effective in supporting operations for the management of service continuity. NBN Co has an established enterprise risk management framework. However, NBN Co’s risk register does not cover all aspects of managing service continuity during the process of migrating a premises on to the NBN, and has a high number of risks that are not ‘approved’ to indicate agreement between relevant staff responsible for the management of the risk, and that have no identified controls or treatment actions.

3.48 The Public Governance, Performance and Accountability Act 2013 (PGPA Act) is a principles-based Act64 supported by the Commonwealth Risk Management Policy (CRMP), which aims to strengthen the risk management practices of Australian Government entities. Entities subject to the CRMP must ensure that systematic management of risk is embedded in key business processes.

3.49 It is not mandatory for Commonwealth companies such as NBN Co to comply with the CRMP. However ‘all GBEs are encouraged to consider applying the CRMP in relation to risk management frameworks and systems’65.

3.50 In accordance with the CRMP, NBN Co reviews and maintains:

  • a current risk management framework and policy which has been approved by the NBN Co Board;
  • risk appetite and tolerance statements; and
  • risk registers identifying operational controls, procedures and residual risk.

3.51 The Group Risk team is responsible for maintaining this framework across NBN Co, and reports to the Audit and Risk Committee every six months on the risk profile and status of risk management activities.

3.52 In August 2018, NBN Co began transitioning business function risks from spreadsheet-based risk register to a centralised enterprise level risk management system as part of a risk management and capability improvement plan.66 In June 2019, 226 of the 493 risks included in NBN Co’s central risk register were ‘approved’, indicating that the risk was discussed and agreed by the risk manager, risk owner and risk coordinator in a business unit. However, the 226 ‘approved’ risks in the risk register included:

  • 38 with no controls identified;
  • 118 with no associated treatment; and
  • 18 with no controls, or treatments.67

3.53 The ANAO identified 75 risks in NBN Co’s central risk register which could broadly be considered as relating to the maintenance of service continuity throughout the migration process (as described in paragraph 1.3).68 These are summarised in Table 3.6.

Table 3.6: Central risk register and disruption to service continuity

 

Number of risks

Number of approved risksa

Total risks in central register

493

226

Risks relating to service continuity

75

37

  • with no controls listed

31

13

  • with no treatments listed

45

15

     

Note: The status of other risks in the register were: ‘awaiting review’, ‘in progress’ or’ blank’.

Source: ANAO analysis of NBN Co documentation.

3.54 The identified risks did not cover all aspects of disruption to service continuity that have been discussed in this chapter, for example there is no reference to:

  • a delayed, or missing notification by NBN Co to an RSP of an unsuccessful attempt to connect an end user’s premises to the NBN;
  • service fault volumes, and their impact on end user experience; or
  • how service fault resolution timeliness impacts on end user experience.

3.55 NBN Co’s central risk management system does not provide historical reports and therefore did not enable review of historical risk registration or status changes in its system. 69

3.56 Operational plans and risk registers for two key business units relating to service continuity—Customer Connections and Customer Service and Assurance (see NBN Co organisational chart in Appendix 4)—were reviewed for July 2018 and February 2019 respectively. This review identified that:

  • the majority of the risks in the central register relating to the Customer Service and Assurance business area had no controls identified;70 and
  • risks are not systematically linked to operational risk and plans.71

3.57 As outlined in this chapter, NBN Co has embedded a number of key risk management practices into operational practices associated with service continuity, in particular through:

  • the implementation of contractual arrangements with delivery partners for service standards;72
  • engagement with RSPs in accordance with the WBA; and
  • NBN Co initiatives in response to monitoring and reporting of its business functions activities.

There is a need, however, for these practices to be formally reflected and monitored in NBN Co’s enterprise risk management framework.

Recommendation no.1

3.58 NBN Co formally reflect all risks, controls and ongoing monitoring processes for risks associated with the management of service continuity during migration of an end user’s premises within the NBN Co centralised risk register.

NBN Co response: Agreed.

3.59 We can confirm that NBN Co has a Board endorsed risk management policy and framework which is based on ISO31000-2018. Over the last several years NBN Co has been executing against its risk management strategy, resulting in substantial improvements in risk maturity across the Company. These risks are regularly stress tested by business unit leadership teams, Executive Committee and the Audit and Risk Committee. NBN Co acknowledges that notwithstanding the increased maturity in risk management practices, this process will and should continue to evolve. Consequently, NBN Co has strategies in place to:

  • Implement a regular review process to ensure all material risks associated with the management of service continuity during migration of an end user’s premises are documented within the NBN Co centralised risk register;
  • Implement data quality and exception reporting for the NBN Co centralised risk register (e.g. approved risks, controls and treatments as required); and
  • Provide enhanced guidance and training to management to enhance how business units monitor their material risks, including the evaluation of control effectiveness and mitigation strategies.

4. NBN Co complaints management

Areas examined

This chapter examines the effectiveness of NBN Co Ltd’s (NBN Co) complaints management process, whether NBN Co resolve complaints in a timely manner, and the extent to which NBN Co undertake ongoing analysis to inform transparency and continuous improvement.

Conclusion

NBN Co is partly effective in managing end user connection-related complaints. Effective internal policies, guidance and quality assurance processes have been established for the receipt and management of complaints by NBN Co, however, public guidance established to assist end users in the lodgement of complaints requires updating for completeness and to ensure appropriate accessibility. Connection-related complaints are not being resolved in line with NBN Co’s timeliness targets, and NBN Co has not established processes to conduct ongoing or systemic analysis of complaint causes to drive continuous improvement.

Areas for improvement

The ANAO made two recommendations aimed at ensuring NBN Co complaints management information is complete and appropriately accessible, and that NBN Co establishes processes for the regular and ongoing use of complaints data to undertake root cause analysis and inform continuous improvement, and an opportunity to improve the quality of data entered into the complaints management system.

4.1 In order to come to a conclusion against this criteria, the ANAO investigated:

  • NBN Co complaints handling processes—as the establishment of clear complaints management roles and responsibilities, the provision of public guidance on the lodgement of complaints, and the maintenance of appropriate frameworks for the receipt and handling of complaints are critical elements in the effective management of end user connection-related complaints by NBN Co;
  • the timeliness of connection-related complaints resolution—as the categorisation of complaints, and their resolution within NBN Co’s target timeframes, are important in ensuring that complaints management processes are supporting effective performance; and
  • analysis and reporting of complaints data—as NBN Co processes to analyse the causes of complaints, to monitor performance in the resolution of complaints, and to use this information to drive continuous improvement are critical to ensuring the ongoing effectiveness of complaints management processes.

4.2 The audit assessed NBN Co alignment with the Telecommunications (Consumer Complaints Handling) Industry Standard 2018 (Complaints Industry Standard), given its responsibilities to provide reasonable assistance to RSPs, and to receive complaints as part of its public-facing role. However, it is recognised that NBN Co is not required to handle complaints in accordance with all parts of the Complaints Industry Standard.

Has NBN Co established an effective complaints management process?

NBN Co has established a largely effective complaints management process. A complaints management policy has been established to provide guidance to end users in lodging complaints, however this does not fully align with the relevant industry standard (which is not mandatory in its entirety for NBN Co), and is not readily accessible to end users. Clear policies, guidance and quality assurance processes have been established for the receipt and management of complaints by NBN Co case management staff, noting that there are minor ambiguities in the accurate classification of complaints within the complaints management system.

Complaints management roles and responsibilities

4.3 The Migration Assurance Framework states:

[NBN Co] is a wholesale infrastructure provider that sells its products to retail service providers, who then add additional functionality and provide retail services to end users.

If a customer has a concern regarding the installation or operation of their NBN services, they should contact their retail service provider first, as they have the main relationship with the customer as well as with [NBN Co] and are best placed to provide assistance or resolve a query or complaint.73

4.4 This statement is reflective of the Complaints Industry Standard, which also places responsibility for NBN complaints management predominantly with RSPs. However, Part 6 of the Complaints Industry Standard requires NBN Co to assist RSPs in the resolution of complaints through the provision of ‘reasonable assistance’ when the complaint relates to matters within NBN Co’s span of responsibility. While it is the intention of both the Migration Assurance Framework and the Industry Standard that RSPs act as the first point of contact for complaints, it is nonetheless appropriate that NBN Co establish policies and procedures for managing complaints given its responsibilities in the provision of reasonable assistance to RSPs, and that it receives complaints as part of its public-facing role.

4.5 Processes within NBN Co for the provision of information to end users on the lodgement of complaints, and for the receipt and management of complaints, are the responsibility of the Enquiries and Complaints team within the Customer Service and Assurance business unit (refer to Appendix 4 for a high level NBN Co organisational chart).

4.6 NBN Co’s complaints management and handling processes74 are articulated in three key documents — the Complaint Management Policy, the Case Management Handbook, and the Customer Resolutions Team Agent Handbook. The Complaint Management Policy is a public-facing, end user focused document, and the Case Management Handbook and Customer Resolutions Team Agent Handbook are internal guidance documents for NBN Co complaint handling and case management staff.

Public guidance for the lodgement of complaints by end users

NBN Co Complaint Management Policy

4.7 As noted above, the intent of both the Migration Assurance Framework and the Complaints Industry Standard is to make RSPs the focal point for end user complaints. While NBN Co is therefore not bound by Parts 2 to 5 of the Complaints Industry Standard, the standard lists a useful set of information elements that should be provided to the general public about the complaints process. The Complaint Management Policy is NBN Co’s primary source of information to the public about making a complaint. The policy includes contact information to make a complaint via telephone, email, NBN Co’s website and fax. It also states that ‘[As NBN Co] is a wholesale only provider, complainants are encouraged to seek to resolve issues directly with their access seeker or telecommunications retail service provider’.75

4.8 The current version of the Complaint Management Policy came into effect in July 2015, and was initially drafted in 2012 in consultation with the Telecommunications Industry Ombudsman (TIO) and the Australian Communications and Media Authority (ACMA). The requirements of the Telecommunications Consumer Protections Code (TCP Code) were considered as part of the policy development process, having regard to the fact that the TCP Code applied to RSPs at that time.

4.9 In November 2017, NBN Co completed a review and drafted a revised Complaint Management Policy for noting by the Board. The ANAO was advised that after the review had been completed and the new Complaint Management Policy had been drafted it ‘was not submitted for final approval’, and ‘the appropriate level of approval…was not obtained’ for the new version of the policy to be signed off by the appropriate officer, noted by the Board, and published. As a result, the 2017 draft version of the Complaint Management Policy was not put into effect and the July 2015 version remains the current version available to the public on the NBN Co website.

4.10 In September 2018, the TCP Code was replaced by the Complaints Industry Standard in response to problems with complaint handing by RSPs identified in a 2018 report by the ACMA.76 NBN Co’s complaints management policy is therefore inconsistent with current telecommunications industry regulations. While NBN Co has provided updated guidance to RSPs on how reasonable assistance requests should be submitted, Table 4.1 lists the Complaints Industry Standard elements that are not addressed by the current (July 2015) NBN Co Complaints Management Policy.

Table 4.1: Complaints Industry Standard elements not addressed by NBN Co Complaints Management Policy77

Complaints Industry Standard Element

Comment

Providing a full breakdown of individual timeframes for every element of the complaint receipt, assessment, investigation and resolution process78

The Complaints Management Policy does not cover the full breakdown of individual timeframes for every element of the complaint management process (refer Figure 4.1 for details on these elements). Rather, it advises that NBN Co will ‘seek to acknowledge written complaints within two working days’, that ‘[NBN Co] endeavours to resolve formal complaints within five working days and preferably at the first point of contact’ and ‘In the case of complex complaints… [NBN Co] will seek a resolution within 20 working days of receipt.’

No further advice is publicly provided.

Advising the timeframe that applies for a complaint that a carrier has determined to be frivolous or vexatious79

There is no advice regarding timeframes of this nature provided to the public.

 

Advising what the carrier will do if it cannot contact the complainant80

Information on what to do if a complainant is uncontactable is contained within internal NBN Co guidance documents only.

Describing how escalated complaints will be managed81

The Complaint Management Policy notes that the complainant can make a request for escalation, but does not describe how escalated complaints will be managed.

   

Source: ANAO Analysis.

4.11 NBN Co reporting to the Board indicates that the next review of the Complaints Management Policy is planned for November 2019.

General information to support end users in making a complaint

4.12 The Complaints Industry Standard states that:

A carriage service provider must ensure that information about how a consumer can contact the carriage service provider to make a complaint or enquiry can be accessed via a link on the homepage for its website.82

4.13 As previously stated, while NBN Co is not compelled to comply with this requirement, it is important that clear information is made appropriately available to consumers on the lodgement of complaints.

4.14 The Complaint Management Policy document is not readily available to the public. There is no link to the Complaint Management Policy document from the NBN Co homepage, and the utilisation of the NBN Co website’s search function using the keywords ‘complaints’ and ‘complaints management policy’ respectively does not have any result. The policy document is located within a sub-folder of the NBN Co website,83 however this can only be found via a specific keyword search in an external search engine.

4.15 The NBN Co website has a ‘Contact Us’ page, and the first result when utilising the website’s search function with the keywords / phrases ‘complaints’ and ‘how to complain’ is a link to the Contact Us page.84

4.16 The Contact Us page has a list of common topics of enquiry, many of which may be standard topics of complaint (e.g. ‘Your address is missing or incorrect’). The Contact Us page also has a clear listing of matters / issues which an end user should contact their RSP for (e.g. ‘Concerns about speed’) versus those which they should contact NBN Co for (e.g. ‘Damages to property or [NBN Co] equipment’). The Contact Us page is not clear, however, on how and where to lodge a complaint. Two references to ‘complaints’ on the Contact Us webpage are not about the general process for making a complaint, but are instead specifically about NBN Co’s Privacy Policy and how an end user can complain if they believe the Australian Privacy Principles have been breached. The webpage links to a general inquiries web form titled ‘Didn’t find your answer? Complete the form below’, which is a way an end user can make a complaint online, though the end user must select from a drop down menu to determine that the form can be used to make a complaint. Further, the Contact Us webpage does not include the complaint-related contact details contained in the Complaint Management Policy, previously identified in paragraph 4.7.

4.17 NBN Co’s direct mail to end users is another channel which could be used to clarify how to lodge a complaint. NBN Co use only one letter in its direct mail series to end users to define its role and that of the RSP (‘Letter sent 16 months before disconnection date’ — refer Appendix 5). Under the list of responsibilities for the RSP identified in the letter, NBN Co include the statement ‘resolves any issues with your connection’, which is intended to direct end users to the RSP as their first point of contact for complaints.

Recommendation no.2

4.18 NBN Co revise its Complaint Management Policy to align with the Telecommunications (Consumer Complaints Handling) Industry Standard 2018, and ensure that the policy document and associated processes are approved and appropriately accessible on the NBN Co homepage.

NBN Co response: Agreed.

4.19 As acknowledged by the ANAO, NBN Co is not subject to all parts of the Telecommunications (Consumer Complaints Handling) Industry Standard 2018.

4.20 Specifically, this standard outlines responsibilities for complaints management by Retail Service Providers, however NBN Co is required to provide ‘reasonable assistance’ when the complaint relates to matters within NBN Co’s span of responsibility.

4.21 As part of this audit, NBN Co provided evidence of compliance in providing ‘reasonable assistance’ as part of the complaints management system. The ANAO has recognised that the NBN Co complaints management system was largely effective. Notwithstanding this, NBN Co agrees it is good governance to update the policy to reflect the most recent standard, which is updated on a two-year basis.

Framework for the receipt and management of complaints by NBN Co

Complaint management guidance and procedures

4.22 The NBN Co Complaints and Enquiries team is responsible for the receipt and management of all customer complaints. The primary complaint handling guidance document supporting the team is the Case Management Handbook. This document:

…outlines the fundamental steps taken to manage [NBN Co] complaints. It describes the overarching requirements of the case management process and the stages of a complaint as it progresses through [NBN Co’s] complaint management system.85

4.23 An additional document, the Customer Resolutions Team — Agent Handbook, provides guidance for case management staff where a complaint cannot be resolved at the first point of contact. Staff are also provided with:

  • a complaints matrix to enable the appropriate triage of complaints86;
  • e-mail templates and scripts for phone conversations;
  • procedures and associated guidance documentation; and
  • quick reference guides.

4.24 These documents provide references to further related documents and templates, and information on document control.

4.25 NBN Co’s complaint handling process includes activities for recording and allocating complaints for resolution by subject matter expert teams, as illustrated in Figure 4.1.

Figure 4.1: NBN Co’s complaint handling process

A flow diagram that shows the process used by NBN Co to handle complaints. The process is summarised further in paragraphs 4.25 and 4.26.

Source: ANAO analysis.

4.26 A Customer Enquiries Quality Framework and a Case Management Compliance to Process procedure have been established, and case management teams are subject to regular compliance reviews to ensure they are managing complaints to the appropriate internal standards.87 The results from these reviews are reported to case management teams on a quarterly basis. NBN Co stated that these reports are used to identify performance development opportunities, but was unable to substantiate how review results were reported at the business unit management level or how they were used to drive continuous improvement.

Complaint management system

4.27 Complaints made to NBN Co are logged and monitored in Mosaic—NBN Co’s complaint management system (CMS). NBN Co staff categorise the complaint types as ‘Formal’, ‘Complex’ or ‘Urgent’ using the Complaint Management Policy and Complaints Matrix (refer Table 4.2 for detailed explanation of these categories). Complaints are then further categorised according to:

  • Phase: where in the ‘customer journey’ the complaint event occurred, such as Information, Planning, Build, Connection, Post-Connection, General, or Uncategorised,
  • Category: the technology type, such as Fibre to the Node (FTTN), or Hybrid Fibre Coaxial (HFC); and
  • Subcategory descriptor: to best describe the complaint subject, such as Equipment Not Working or Connection Quality/Speed.

Staff also log the subject of the complaint and a description using free text fields.

4.28 Accuracy and consistency in the logging and categorisation of complaint data in the CMS is important in ensuring the validity of NBN Co’s internal reporting on complaint data, which in turn supports root cause analysis and continuous improvement (refer to paragraph 4.47 to 4.50 for detailed analysis of continuous improvement processes).

4.29 Review of complaints across the audit period identified that structures are largely in place to ensure that data has been accurately captured by staff in a manner which is in line with NBN Co policy. However, the following ambiguities in structures to ensure the accurate classification of complaints were noted:

  • Data does not clearly isolate connection-related complaints by Phase. Examples of connection-related complaints could be found in the Connection, Post-Connection, General, or Uncategorised phases.
  • Data shows multiple, overlapping entries for technology types in the Category fields, which can create errors when analysing complaints according to technology. For example, a complaint about a service on Fibre to the Node could be classified in Fibre — Brownfields FTTN — Brownfields, FTTN — Greenfields, or FTTN.88
  • Utilisation of free text fields for complaint record data entry:
    • While there is a need for free text to capture the unique circumstances of each complaint, and free text fields are a common feature in a CMS, these fields are used by NBN Co inconsistently — some records include detailed information, others very little and on very few occasions the field is left blank.
    • The ability to continually edit a free text field provides flexibility allowing a staff member to work on a complaint they may be unfamiliar with (something NBN Co identifies as important), however this flexibility also makes it possible for relevant data to be deleted without a corresponding audit trail.

4.30 The potential impact of these ambiguities is that NBN Co analysis and reporting on complaints data may not accurately capture all complaints associated with a particular connection phase, or technology type; and individual records may not contain sufficient detail to understand the nature of the complaint, and therefore develop appropriate responses to address systemic issues.

4.31 NBN Co should revise its complaints management guidance to address these issues, and ensure that the regular compliance review process includes appropriate coverage of these items.

Are connection-related complaints resolved in a timely manner?

Connection related complaints are not being resolved in line with NBN Co’s timeliness targets. NBN Co has defined three complaint categories — Formal, Complex and Urgent — and established clear KPIs for each. However, 87.7 per cent of combined Formal and Complex complaints are being resolved within KPIs, which is below the NBN Co target of 93 per cent, and 45.5 per cent of Urgent complaints, of which there were only 11 in the audit timeframe, were resolved within the associated KPIs, which is also below the NBN Co target of 93 per cent. Complaints that are unresolved within the respective KPIs for all three categories are typically not resolved in a sufficiently timely manner after the KPI period has elapsed.

General complaint types and volumes

4.32 Both the Complaint Management Policy and Case Management Handbook define complaint types and resolution timeframe Key Performance Indicators (KPIs) as shown in Table 4.2.

Table 4.2: Complaint types and resolution timeframe KPIs

Complaint type

Description

Resolution KPI

Formal

A relatively simple, straight forward expression of dissatisfaction.

Five working days (though preferably at first point of contact)

Complex

A complaint that cannot be resolved/managed at the first point of contact and are generally of a more serious or complex nature …They typically require cross functional collaboration to resolve, a number of interactions or discussions with the complainant, and may involve commitment of resources. Complex complaints are referred to a dedicated case management team.

20 working days

Urgent

A complaint involving a complainant who is a priority assist customer, a complaint where disconnection of service is imminent, or where a complainant is in financial hardship.

Two working days

     

Source: NBN Co, Complaint Management Policy and Case Management Handbook.

4.33 The Operating Plan for the Customer Service and Assurance Business Unit, which is responsible for complaints management, has an 2019 target that 93 per cent of combined formal and complex complaints will be resolved within their respective KPIs (i.e. five or 20 working days) for these complaint types.

4.34 Within the audit timeframe, NBN Co logged 173,601 end user complaints. Figure 4.2 shows this volume of complaints according to complaint type.

Figure 4.2: Total complaint volumes during the audit time frame

 

Source: ANAO analysis of NBN Co data.

Connection-related complaints — types and volumes

4.35 Figure 4.3 shows the trend in the weekly volume of new NBN connections during the audit timeframe.

Figure 4.3: Weekly volume of new NBN connections

 

Note: Service class 4 (wireless) and 9 (satellite) were not represented in the NBN Co data.

Source: ANAO analysis of NBN Co data.

4.36 Of the 173,601 total complaints recorded by NBN Co within the audit timeframe, 34,196 (approximately 20 per cent) were attributed by NBN Co as being related to new connections.89 Figure 4.4 shows the trend in connection-related complaints during the audit timeframe.

Figure 4.4: Connection-related complaint volumes during the audit timeframe

 

Source: ANAO analysis of NBN Co data.

4.37 Figure 4.4 reflects:

  • a decreasing trend in the volume of Formal connection-related complaints, from a peak in the January 2019 (this peak correlates with a rise in the volume of weekly connections during the same period—refer Figure 4.3); and
  • an increasing trend in the volume of Complex connection-related complaints toward the end of 2019—it is unclear what is specifically driving this, however NBN Co has stated that it expects to see an increase in complaint volume and complexity as the rollout, build and connect phases continue to grow, reflected in Figure 4.3.

Connection-related complaints — resolution within KPIs

4.38 For connection-related complaints, the proportion that were resolved within their respective KPI timelines were analysed, with the result shown in Figure 4.5.

Figure 4.5: Percentage of complaints resolved within respective KPIs during the audit timeframe

 

Note: Urgent complaints were not included because of the fluctuations caused by being in such small numbers over the timeframe compared with the other two complaint types. See paragraph 4.40 below for urgent complaint volumes and percentage resolved within NBN Co’s KPI.

Source: ANAO analysis of NBN Co data.

4.39 During the audit timeframe, the majority (87.7 per cent as an average of total connection-related complaints) of Formal and Complex complaints were resolved within their respective KPI timeframes. This is below the 93 per cent target set by NBN Co for the combined resolution of these complaint types in FY2019.

4.40 Urgent complaints also lag behind the target set by NBN Co that 93 per cent of complaints be resolved within timeframes, with only 45.5 per cent of Urgent complaints resolved within the two day target. While Urgent complaints are a very small proportion of the total connection related complaints received by NBN Co during the audit period (11 complaints, or 0.000063 per cent), these complaints potentially involve priority assistance end users.

Connection-related complaints — resolution outside of KPIs

4.41 For those connection-related complaints which were not resolved within their respective KPIs, analysis was undertaken to determine the timeliness in which these complaints were ultimately closed.

4.42 Figure 4.6, Figure 4.7 and Figure 4.8 show trends in the number of days that elapse before complaints that were opened and are not resolved within the respective category KPIs before they are ultimately closed during the audit timeframe.

Figure 4.6: Percentage of Formal connection-related complaints closed past five working day KPI

 

Note: For reference, the 22.1 per cent of complaints that were closed beyond 10 working days equates to 493 total individual complaints.

Source: ANAO analysis of NBN Co data.

Figure 4.7: Percentage of Complex connection-related complaints closed past 20 working day KPI

Note: For reference, the 4.6 per cent of complaints that were closed beyond 40 working days equates to 94 total individual complaints.

Source: ANAO analysis of NBN Co data.

Figure 4.8: Percentage of Urgent connection-related complaints closed past two working day KPI

 

Note: For reference, the 67.8 per cent of complaints that were closed beyond four working days equates to four total individual complaints.

Source: ANAO analysis of NBN Co data.

4.43 Figure 4.6, Figure 4.7, and Figure 4.8 reflect that, of those complaints that were not resolved within their category KPIs:

  • the majority of Formal complaints (51.51 per cent) took up to four working days to resolve post the expiry of the associated KPI (five working days);
  • the majority of Complex complaints (50.54 per cent) took up to nine working days to resolve post the expiry of the associated KPI (20 working days); and
  • the majority of Urgent complaints (50 per cent) took up to five working days to resolve post the expiry of the associated KPI (two working days).90

4.44 This data reflects that complaints that are unresolved within the respective KPIs for all three categories are typically not being resolved in a sufficiently timely manner once the KPI has elapsed. The cause of this issue is unclear, as discussed in paragraphs 4.48 and 4.49. A recommendation to address this issue has been raised at paragraph 4.51.

Does NBN Co undertake ongoing connection-related complaints analysis and reporting to provide transparency and inform continuous improvement?

NBN Co undertakes regular internal reporting on the volume of complaints and associated resolution timeframes, but could not demonstrate that it conducts ongoing or systemic analysis of connection-related complaint causes either internally or with retail service providers.

Complaint policy and internal reporting

4.45 NBN Co’s Complaint Management Policy states that:

In order to seek to identify and reduce the recurrence of complaints arising from systemic issues, [NBN Co]:

  • monitors complaints to seek to identify emerging issues requiring specific attention and address those as soon as practicable, and
  • analyses and classifies complaint data at least every 3 months to seek to identify recurring problems and issues and action opportunities for improvement.

Processes also seek to ensure rapid and effective management of, and notification to appropriately senior management of, significant complaints or issues.91

4.46 NBN Co uses complaint data in a variety of internal reports for review by relevant business areas and at the executive level. Operational levels of the organisation produce between 13 and 17 complaint-related metrics, of which the NBN Co executive is provided with two — Formal and Complex complaint turnaround times. Reporting to the Executive is focused on complaint volumes, aged complaints,92 and resolution turnaround times. It does not contain information or analysis on the potential causative factors associated with volumes and trends in respective complaint categories and types. For the audit timeframe, there was some reporting on complaint categorisation during the months of March to August 2018, but this was not consistently maintained.

Continuous improvement

4.47 In January 2019, NBN Co introduced an initiative to reduce complaints volumes in response to ‘trends identified during the governance of [NBN Co’s] enquiries and complaints performance’. In particular, the most significant trend was the higher incoming contact volumes than forecast, which had corresponding impacts to the average speed of answer and aged complaint volumes.’ While NBN Co’s strategy documents outline an intention to transform the end-to-end process for receiving, resolving and managing complaints, it could not substantiate that the initiative has commenced.

4.48 Root cause analysis of individual cases within a subcategory of complaints for continuous improvement is not easily supported by the process in which complaints data is entered into NBN Co’s CMS. While NBN Co captures and reports complaint volumes and closure rates, NBN Co’s CMS does not record the specific actions taken to support changing the status of a complaint record to resolved or closed, aside from inconsistent use of a free text notes field. NBN Co record the complaint resolution date and status in the CMS, however for those complaints that require action by NBN Co or its contractors, the associated action taken to resolve an end user’s complaint cannot be substantiated, even if this was a judgement made by a case manager that no further action was required.

4.49 NBN Co has no established structure to regularly consider systemic causes of complaints with RSPs. Additionally, NBN Co does not analyse the relationship between complaint and incident records. For the audit timeframe, there were 80,788 premises that had logged both a complaint and a service fault. The majority of these complaints (73.3 per cent) were logged after an associated fault was raised with NBN Co via the RSP.

4.50 In June 2019, NBN Co commenced the following initiatives to address the issues identified in paragraph 4.49:

  • complaints trends and analysis with a major RSP broken down by each phase in the migration process;
  • planned processes to deal with issues arising out of complaints analysis; and
  • internally report RSP escalations.

Recommendation no.3

4.51 NBN Co establishes a regular and ongoing process to use its complaint data to undertake root cause analysis of connection-related complaints, to inform continuous improvement internally and with complaints management stakeholders.

NBN Co response: Agreed.

4.52 NBN Co takes the management of complaints data seriously and takes steps to analyse data and report outcomes.

4.53 NBN Co believes it has always had an approach to continuous improvement, albeit that we can make this more regular and on-going. This is acknowledged by the ANAO in the report where it is noted that NBN Co uses complaints data in a variety of internal reports for review by relevant business areas and at executive level.

4.54 Overtime this has led to improvements in the complaints handling process. Recently implemented initiatives include, the analysis of complaints trends, the analysis by phase in the migration process, and programs targeting complaints analysis working with Retail Service Providers. NBN Co will continue to evolve these processes in a pragmatic manner.

Appendices

Appendix 1 NBN Co Limited response

Appendix 2 Technology types and service classes

NBN technology type descriptions

Technology type

Brief description

Fibre to the Premises (FTTP)

Deploys fibre optic cable all the way to premises.

Available in a variety of existing locations as well as most large new developments.

Fibre to the Basement (FTTB) / Node (FTTN) / Curb (FTTC)

Deploys fibre into neighbourhoods and then makes use of the existing copper into the premises.

Hybrid Fibre Coaxial (HFC)

Leverages existing networks of fibre and coaxial cable to deliver broadband services into the premises.

Fixed Wireless

Largely targeted at regional communities and provides the means for broadband to extend to Australians outside the reach of the fixed-line network.

Satellite

NBN access via a communication satellite.

Source: NBN documentation.

Service class descriptions

Technology type

Service class

Requirement for connecting to the network

Fixed line (Yes/No)

Fibre to the premises (FTTP)

0

Premises allocated in the network footprint but not serviceable

Yes

1

In Fibre Network footprint - work required

Yes

2

Serviceable by fibre. Drop in place, no NTD

Yes

3

In Fibre Network footprint - NBN Co can remotely provision service

Yes

Wireless

4

The address is planned to be serviced by wireless, but is not yet serviceable

No

5

Serviceable by wireless. NTD not installed

No

6

Serviceable by wireless. NTD installed

No

Unallocated

7

NBN Co has not yet determined the network footprint which the premises will belong

No

Satellite

8

Serviceable by Satellite, VSAT Terminal and NTD not installed.

No

9

Serviceable by Satellite, VSAT Terminal and NTD installed.

No

Fibre to the basement (FTTB) or fibre to the node (FTTN)

10

Premises allocated in the network footprint but not serviceable

Yes

11

Serviceable by copper, active node present

Yes

12

Serviceable by copper, jumpering is required

Yes

13

Serviceable by copper, infrastructure in place

Yes

Hybrid fibre coaxial (HFC)

20

Premises allocated in the network footprint but not serviceable

Yes

21

Serviceable by HFC. No drop, wall plate, or NTD

Yes

22

Serviceable by HFC. Drop in place, no wall plate or NTD

Yes

23

Serviceable by HFC. Drop and wall plate in place, no NTD

Yes

24

Serviceable by HFC. Drop, wall plate, and NTD in place

Yes

Fibre to the curb (FTTC)

30

Premises allocated in the network footprint but not serviceable

Yes

31

Serviceable by FTTC, no line available (NCD required)

Yes

32

Serviceable by FTTC, cut-in required (NCD required)

Yes

33

Serviceable by FTTC, cut-in complete (NCD required)

Yes

34

Serviceable by FTTC, infrastructure in place (Check NCD status)

Yes

Source: NBN Co documentation.

Appendix 3 Key responsibilities for the process of migrating on to the NBN

Stakeholdera

Responsibilityb

NBN Coc

  • Forecast and notify RSPs areas that are ‘ready for service’
  • Notify an end user in advance that their premises will be ready for service
  • Notify end users in ready for service areas what they need to do to initiate a connection and how a connection will be made
  • Receive a work order that a premises is ready to connect
  • Validate that a premises is ready to connect
  • Install NBN technology, including attending necessary appointments at the premises to connect and enable an active service
  • Fix problems encountered during the connection process including attending necessary appointments at the premises
  • Notify the RSP of an unsuccessful NBN connection
  • Notify the legacy network carrier when a premise has successfully migrated to the NBN
  • Fix problems caused by NBN wholesale infrastructure after an NBN connection has been completed
  • Notify ready for service premises that have not yet initiated migration to the NBN of the consequences of not doing so before the Disconnection Date
  • Receive complaints from end users, refer relevant complaints to RSPs and manage those that relate to matters that are the responsibility of NBN Co

Retail service providers (RSPs)

  • Market to end users in ready for service areas that they can provide a service over the NBN at their location
  • Facilitate requests from end users that are ready to connect and activate a service on the NBN
  • Charge the end user for accessing the NBN service once it is installed
  • Notify NBN Co of circumstances where an appointment is required or cancelled by an end user, and communicate to the end user when NBN Co technicians can be expected at their premises
  • Only disconnect (or request the disconnection of) a legacy infrastructure connection when post-migration testing indicates that a successful migration has occurred at the premises
  • Notify NBN Co of issues reported by the end user after they have been connected to the NBN
  • Refer problems associated with NBN wholesale infrastructure to NBN Co
  • Act as the first point of contact for customer queries and resolution of complaints once an order is placed

End-users

  • Inform themselves of the need to migrate their telecommunications services
  • Request connection to the NBN through their preferred RSP
  • Be aware of the NBN services that will meet their needs, including retention of existing phone numbers
  • Following any instructions given by their service provider(s)
  • Be available and attend a scheduled appointments associated with migration on to the NBN
  • Raise with RSPs or NBN Co when service has been disrupted or lost

The Australian Communications and Media Authority

  • To develop and maintain rules obliging carriage service providers (CSPs) to ensure consumers receive appropriate pre-sale information and advice from CSPs to help them prepare for their migration to the NBN
  • To develop and maintain rules to minimise the likelihood of consumers being left without a working telecommunications service for extended periods when migrating on to the NBN
  • To develop and maintain rules to give consumers confidence that their complaints will be effectively managed by RSPs if problems arise with an NBN service

The Telecommunications Industry Ombudsman

  • To handle complaints about telephone and internet services, including by collecting any documents or information relevant to the complaint, and to decide the resolution of a complaint the RSP is legally obliged to implement

Note a: While there are other industry stakeholders that have responsibilities related to the NBN migration process, for the purposes of this audit the key responsibilities and relationships are among NBN Co, RSPs and end-users. See the legal and policy documents listed at Table 1.2 for more detail on NBN Co responsibilities.

Note b: Timeframes to complete many of these responsibilities are agreed between NBN Co and RSPs via the Wholesale Broadband Agreement, discussed in Chapter 3.

Note c: For the purposes of this table, ‘NBN Co’ refers to both its permanent and contracted employees, and so therefore covers the role of delivery partners.

Source: ANAO analysis.

Appendix 4 NBN Co high level organisation structure (as at 1 August 2019)

Note: Business units included are only those that have been referred to in the report.

Appendix 5 Direct mail content and timing leading up to disconnection date (DD)

Sequence of direct mail

Direct mail information

1. Postcard sent two months before ready for service and 20 months before DD

Provides a link to NBN Co’s website to register and track when an area is ready for service.

2. Letter sent 16 months before DD

Four page document to notify that an area is ready for service and the steps that an end user can take to migrate on to the NBN. Provides reference to RSPs being the entity that ‘resolves any issues with your connection’.

3. Letter sent 14-12 months before DD

Two page letter reiterating information that was in letter sent at 16 months before DD about how to connect.

4. Postcard or Letter sent 11-9 months before DD

Almost identical letter to the letter sent at 14-12 months, or postcard directing to an NBN Co list of RSPs.

5. Letter and Brochure sent 8-6 months before DD

Almost identical letter to that sent at 14-12 months. Includes a seven page brochure that expands and reiterates letter messages.

6. Postcard sent 2-1m before DD

‘Urgent’ or ‘Final reminder’ messages accompanied by the DD. Provides link NBN Co’s website to choose a RSP.

Source: ANAO analysis.

Appendix 6 Fault resolution timeliness volumes over time by technology type and geography

Remote

 

Minor rural

 

Major rural

 

Urban

 

Notes: The ‘20+’ figure on each x axis has been used to allow for earlier fault volumes to be visible on each figure. By geographic region, the number of faults taking longer than 20 days to resolve were: urban = 213; minor rural = 22; major rural = 34; remote = 3.
According to NBN Co data within the audit timeframe, HFC service faults were only recorded at urban premises.

Source: ANAO analysis of NBN Co data.

Footnotes

1 NBN Co is a Commonwealth company under the Public Governance, Performance and Accountability Act 2013, and more specifically is classified as a Government Business Enterprise. NBN Co’s Government Shareholder Ministers are the Minister for Communications and the Minister for Finance.

2 NBN Co, Corporate Plan 2020–23, p. 4.

3 The migration process only applies to existing premises that already have a connection to legacy telecommunications infrastructure and therefore would ‘migrate’ to the NBN (commonly referred to as brownfields) if an end user chooses to do so.

4 A government business enterprise is a Commonwealth entity or company as defined by section 8 of the PGPA Act and prescribed by section 5 of the PGPA Rules.

5 The migration process only applies to existing premises that already have a connection to legacy telecommunications infrastructure and therefore would ‘migrate’ to the NBN (commonly referred to as brownfields) if an end user chooses to do so. New developments (or greenfields) have no need to migrate from legacy infrastructure.

6 Once a premises is confirmed to be connected with an active service on the NBN, the legacy system service is disconnected.

7 Fixed-line technologies include five different methods of connecting to the NBN: fibre to the node (FTTN); fibre to the basement (FTTB); fibre to the curb (FTTC); fibre to the premises (FTTP) and hybrid fibre coaxial (HFC).

8 For example, FTTC technology requires connecting legacy copper infrastructure to the fibre network before the service can be activated. FTTP technology only requires installation of NBN connecting equipment as the premises already has fibre built in.

9 Australian Communications and Media Authority, Migrating to the National Broadband Network—the consumer experience—key findings from analysis of industry information, 2017.

10 Page 22 of the Migration Assurance Framework – telecommunications industry guide (2019) states that NBN Co must ‘…[ensure] that NBN services are installed in a timeframe that enables responsible parties to migrate their services with minimal service disruption’.

11 Timeframes are set for responding to connection requests, managing work orders for installation of equipment for connection and activation, and disconnecting legacy infrastructure are defined in contractual arrangements and agreements.

12 Services include internet, telephone or Hybrid Fibre Coaxial (subscription television) services.

13 The use of the 30 day window has been applied for the purposes of the audit. NBN Co internal reporting includes a number of metrics to measure success of new connections within 30 days of activation, including service faults occurring in that period.

14 88.1 per cent represents 8,834,649 premises.

15 These principles have also been largely reflected in the Migration Assurance Framework since it was agreed in 2014.

16 NBN Co, Corporate Plan 2019–22, p. 46.

17 The business unit reporting to this role includes both residential and business end user communications and marketing.

18 NBN Co’s rollout map is a comprehensive geospatial tool for end users to find out about their location on almost any scale and can be found at https://www.nbnco.com.au.

19 ‘Experiential’ refers to face to face contact between NBN Co staff and end users via community events, shopping malls, etc and provides end users with the opportunity to ask questions and receive an immediate response.

20 In the last two financial years, NBN Co paid approximately $49 million to this agency for communications (including fees and production costs), the majority of which was for passed on to media service providers for advertising.

21 These lists are referred to as the Historical Footprint List and the Monthly Rollout Plan and are updated and shared with RSPs on a weekly basis.

22 This is prompted by a premises’ status being changed to ‘Ready to Connect’, which means that the end user has been sufficiently aware that they can access their NBN service.

23 NBN Co may send correspondence to end users outside of the timeframe included in paragraph 2.12 by exception.

24 NBN Co stated that it ‘wouldn’t want to feature historical dates (if they have changed) because we believe to do so, could cause widespread confusion about which date is correct, and therefore confuse the end user. In order to prevent this occurrence only current dates are disclosed’.

25 NBN Co complaints are logged using categories and includes subcategories for the NBN rollout and mapping of the rollout.

26 NBN Co use only one letter in its direct mail series to end users to define its role and that of the RSP (‘Letter sent 16 months before disconnection date’ — refer Appendix 5).

27 Department of Communications and the Arts (2018) Migration Assurance Framework – telecommunications industry guide, p. 19.

28 Department of Communications and the Arts (2018) Migration Assurance Framework – telecommunications industry guide, p. 66.

29 An NBN fixed wireless connection utilises data transmitted over radio signals to connect a premises to the NBN.

30 Readiness is measured by surveying end users who are in ready for service areas and are not yet connected, and those who have connected. The data collected is weighted by age, state, rollout stage and NBN technology to ensure it is representative.

31 Timeframes for responding to connection requests, managing work orders for installation of equipment for connection and activation, and resolving problems are defined in contractual arrangements and agreements.

32 Services include internet, telephone or HFC (television subscription) services.

33 Examples of connecting equipment include a: jumper cable; connecting fibre; copper connecting cable; and indoor/outdoor network access point.

34 NBN Co starts charging an RSP as soon as it determines a connection is active for all technology types, except for FTTC. NBN Co provides completion advice to RSPs on completed end user connections for FTTB and FTTN. If a location has been connected to the network by a previous end user then a premises can be operationally connected to the network but considered inactive by NBN Co until a new end user activates the service through their selected RSP.

35 An unsuccessful migration appointment may not result in a disruption to service as the end user’s legacy service may still be operational. However, if service disruption has occurred, the RSP and end user can choose among a range of options for an alternate service, including offering a mobile service or reconnecting the premises to legacy infrastructure.

36 An outstanding appointment notification is sent when NBN Co becomes aware that the outcome of the appointment is unknown and it is not clear who is responsible. The notification states that an update will be provided within 24 hours. NBN Co stated that it does not consider the outstanding appointment notification as impacting compliance with the Service Continuity Industry Standard.

37 The current WBA is a suite of arrangements executed in 2017 to manage commercial expectations and NBN Co performance with RSPs. The documentation includes: technical term and specifications; NBN Ethernet product descriptions and prices; service level agreements and performance targets. The fourth WBA is currently in negotiation between NBN Co and RSPs and due to be finalised in November 2019.

38 Since January 2019, NBN Co has trialled a service health card on FTTN/B technology which allows internal operators to view live data on a service, such as its operational connectivity status, current speed and average speeds over the last seven days, and number of drop outs in the last 48 hours.

39 Performance reporting is data related and considered a subset of governance reporting according to the WBA.

40 NBN Co groups RSPs into two tiers to facilitate engagement according to size and complexity.

41 Anything that results in an interruption or failure of service can be an incident and may or may not be in NBN Co’s control (for example, an end user premises shortfall, or alternatively a weather event that destroys existing infrastructure). Additional to a standard service fault, incidents can include network faults (where multiple premises are affected), and enhanced faults (where the end user has ordered a product that requires NBN Co to rectify faults as a high priority) and external faults (defined as ‘not a service fault’). The audit did not include a review of enhanced or external faults.

42 As discussed in paragraph 3.32, a large proportion are related to either end users or RSPs, and are therefore not within NBN Co’s control.

43 NBN Co connections data did not always have a consistent time stamp. As a result, ANAO could not determine whether a connection was activated before or after a fault occurred, and therefore excluded records where a connection was completed and an incident logged on the same date in approximately three per cent of instances.

44 NBN Co considers that an end user is not fully migrated until the option to return to a service on legacy infrastructure has been removed, known as disconnection. For technologies that use a combination of legacy infrastructure and NBN equipment, the option to reconnect to legacy infrastructure remains until the end user has been disconnected and can no longer have services provisioned in that way.

45 To analyse service faults within 30 days of a new connection, three datasets were obtained from NBN Co: connection work order records; incident records; and incident work order records. NBN Co work processes allow for multiple connection work orders at the same location, and multiple incidents per connection. As data was extracted from a dynamic system, the analysis reported represents the status for the audit timeframe as at May 2019. ANAO used the combined datasets to complete the service fault analysis contained in Chapter 3.

46 When an end user reports that their service has never worked and NBN Co confirm this is the case it is classified as a ‘new service never worked’ fault type. Where NBN Co have used a combination of legacy infrastructure and NBN equipment to establish the end user’s service (for connections made on FTTB/N and FTTC technology), a ‘new service never worked’ fault is considered a disruption of service to the end user.

47 Some of these faults may be reported by an end user due to a network fault or outage. NBN Co resolves these issues through its network assurance area, and as previously mentioned in paragraphs 3.13 and 3.14, while these fault types relate to a disrupted service they were not considered as directly related to migrating an individual’s telecommunication service off legacy infrastructure and onto the network. The number of service faults that were found to be related to a network outage or fault is illustrated in Table 3.2.

48 RSPs raise an incident trouble ticket once they have used NBN Co’s test and diagnostic tool (available through the service portal or Business-to-Business interface) to identify if they reasonably consider a fault to be a service fault requiring NBN Co’s assistance. If neither the service portal nor Business-to-Business interface is available, then RSPs may raise a trouble ticket by email or telephoning the relevant RSP support centre contact.

49 The operations manual includes information and guidance on: RSP on-boarding; physical access responsibilities and requirements; activations; assurance; management of major events; billing and payments; and operational governance. To assist RSPs understand how NBN Co implements these operational components, there is also an operations manual user guide.

50 For example, the status of a work order is shown in the service portal for RSPs to monitor as required. RSPs can also submit an order (connect/modify/disconnect); amend/cancel an inflight order; receive order notifications and retrieve information such as checking for serviceability of locations. Some of the larger RSPs have implemented the Business-to-Business, which is a direct software interface for communication and access to NBN Co information.

51 A trouble ticket is a term used by NBN Co to describe a notification and record of a fault.

52 In some instances the technology used to establish the end user’s service will mean that if a new service never worked fault occurs it does not interrupt the end user’s service as they will continue having access to their legacy connectivity—this is for FTTP only. Where NBN Co has used a combination of legacy infrastructure and NBN equipment to establish the end user’s service (for connections made on FTTB/N and FTTC technology), a new service never worked fault is considered a disruption of service to the end user.

53 These include faults on fixed line, satellite and wireless technologies.

54 NBN Co revised its target down from 98 per cent in April 2019.

55 There are different timeliness targets for satellite and wireless technologies. Performance targets for fixed line technologies were only influenced by remoteness and not the technology type.

56 NBN Co produces weekly service assurance scorecards that include 66 metrics on: triage processes for managing incidents; assurance operations; case management and aged incidents. Metrics reported to the Executive relate to timeliness and effectiveness of fault resolution, such as ‘mean time to resolve’, fixing a service fault ‘right first time’, and monitoring of the number of technician appointments required at a location to fix a fault (‘repeat truck rolls’).

57 NBN Co considers that an end user is not fully migrated until the option to return to a service on legacy infrastructure has been removed, which is known as disconnection. For technologies that use a combination of legacy infrastructure and NBN equipment, the option to reconnect to legacy infrastructure remains until the end user has been disconnected and can no longer have services provisioned in that way.

58 The service level agreement for resolving priority assistance service faults is 24 hours for urban, minor or major rural areas and remote areas where NBN Co does not require external or internal plant work or attendance at the premises. For remote areas where the fault requires external or internal plant work or NBN Co attendance at the premises the service level target is 48 hours.

59 1.8 per cent represents 35 service faults within 30 days of a new connection recorded from 1,910 priority assistance connections within the audit timeframe. This is low when compared to the 127,464 service faults within 30 days of a new connection as a proportion of 1.86 million standard connection work orders that occurred in the audit timeframe, which is approximately seven per cent.

60 This risk was clearly identified in NBN Co’s central risk register.

61 NBN Co reporting indicates that its performance has improved from 88 to 99 per cent between November 2018 and January 2019.

62 The service level targets for resolving faults for priority assistance services are the same timeframes as the connection targets. For urban and rural areas priority assistance fault rectifications are expected to be completed within 24 hours. For remote areas the timeframe is 48 hours. There is no service level target for isolated areas or areas with limited access.

63 A ‘truck roll’ is a term used to describe when a technician is dispatched to undertake work at a premises.

64 The PGPA Act seeks to improve the high level accountability of all Commonwealth entities through focusing on their duties, internal controls and the way they engage with, and manage, risk. In particular, Chapter 3 of the PGPA Act applies to Commonwealth companies.

65 Commonwealth of Australia, Commonwealth Government Business Enterprises – Governance and Oversight Guidelines, Resource Management Guide No. 126, 2018, p. 28.

66 According to a December 2018 Integrated Assurance Forum paper, NBN Co risk reporting was ‘inconsistent’ and ‘decentralised’ due to the entity ‘growing organically and risk reporting developing on an as needs basis’.

67 These terms are separately used in NBN Co’s risk register, but not explicitly defined in its Risk Management Standard or policy.

68 Maintaining service continuity throughout the migration process requires effective communications, successful installation of infrastructure and timely resolution of incidents, faults and complaints.

69 NBN Co stated that ‘[t]his is the case as risks are captured in a live system that is being constantly updated, with the first priority being the capture of risks to ensure that these are visible to management.’

70 For the business functions recorded in the central register relating to the assurance area there were 11 of 15 risks with no controls.

71 NBN Co planning guidance states that there is no direct alignment between team operating plans and risks, mitigations and critical topics in the integrated operation plans (executive level strategic plans).

72 NBN Co contracts delivery partners for the network rollout, installation of end user connection orders and assurance services under the operations and maintenance master agreement (OMMA). This framework provides financial incentives for delivery partners to achieve performance targets and enables NBN Co to monitor and manage risks to quality and timeliness. For example, delivery partners receive payment for keeping the number of repeat service assurance work orders for a site below five per cent per month. Clause 1.3.4 “Improvement Plan” of Section 9 of the OMMA indicates that delivery partners must provide improvement performance plans if requested by NBN Co in the event of repeated failure of meeting key performance indicators.

73 Department of Communications and the Arts, Migration Assurance Framework – telecommunications industry guide, 2018, p. 40.

74 In this context, ‘complaints management’ refers to all NBN Co operations and personnel required to deal with complaints, whereas the ‘complaints handling’ process is the procedure for responding to and seeking resolution and closure of complaints as they are received.

75 NBN Co, Complaint Management Policy, Version 3, July 2015, p. 1.

76 Australian Media and Communications Authority, Migrating to the National Broadband Network — the consumer experience Review of information retail service providers make available to consumers — key findings, 2018.

77 It should be noted that NBN Co is not legally bound to comply with parts 2–5 of the Telecommunications (Consumer Complaint Handling) Industry Standard 2018.

78Telecommunications (Consumer Complaint Handling) Industry Standard 2018, section 9.

79 ibid., section 16.

80 ibid., section 17.

81 ibid., section 10(d).

82 ibid., section 8(3).

83 NBN Co, Complaint Management Policy, https://www.nbnco.com.au/content/dam/nbnco/documents/

complaint-management-policy.pdf [accessed: 25 June 2019].

84 NBN Co stated that the site was designed to take the user to a channel for lodging a complaint, as opposed to presenting them with the Complaint Management Policy in the first instance which might cause confusion.

85 NBN Co, Case Management Handbook, Version 1.6, p. 3.

86 NBN Co has established a senior escalations team to manage complaints that it determines as being high risk.

87 Compliance reviews include assessment against criteria based on internal policies and the case management handbook.

88 NBN Co have advised that like categories are consolidated for the purposes of internal reporting.

89 The proxy provided by NBN Co for connection-related complaints was upon request by ANAO, and is not an established proxy that NBN Co staff use to categorise complaints when entering a record (as discussed in paragraph 4.29) and to inform analysis and continuous improvement (as discussed in paragraph 4.49).

90 A total of 86.7 per cent of urgent complaints were resolved in less than 10 working days, at which time NBN Co is required to advise the complainant about their option to take the case to the TIO.

91 NBN Co further advised that:

it is committed to improving the end user experience by reviewing complaint data and trends to improve instances where [NBN Co] may contribute to the underlying cause of complaints. In this regard, [NBN Co] continues to work with our delivery partners, Access Seekers and RSPs to help reduce complaint drivers, improve service levels, complaint resolution timeframes, and the customer experience.

92 NBN Co defines an aged complaint as any complaint that has exceeded its resolution KPI twice over.