Audit snapshot

Why did we do this audit?

  • The Parliamentary Expenses Management System (PEMS) is the IT system for parliamentarians, their staff and administering agencies to claim and process office and travel expenses and administer payroll services.
  • The aim of PEMS was to be the key enabling IT system to support better administration, accountability, and transparency of work expenses for parliamentarians and their staff.
  • The audit assessed if the Department of Finance (Finance) successfully implemented the PEMS project to deliver required capabilities and agreed project outcomes.

Key facts

  • PEMS was intended to allow parliamentarians, their staff and administering agencies to claim office and travel expenses, process payroll, request leave and access budget reports.

What did we find?

  • Finance was partly effective in implementing the PEMS project.
  • Project governance, planning and risk management arrangements were in place, although they were not impactful in driving successful project outcomes.
  • PEMS supports the payment of office and travel claims and delivery of payroll services. However, these processes are supported by manual workarounds and end users were not appropriately consulted on the design of PEMS, impacting user views on the system.

What did we recommend?

  • There were two recommendations related to ensuring future projects have agreed scope and user requirements and that Finance measures the delivered capability of PEMS against expected benefits.
  • Finance agreed to the recommendations.

151,221

office and travel claims processed through PEMS in 2022–23.

$74.3m

total cost of PEMS project to June 2023 against original budget of $38.1 million.

2117

end users of PEMS as at July 2023.

Summary and recommendations

Background

1. Parliamentarians are provided with a range of support services and allowances to assist them to carry out their duties including office accommodation and facilities, travel costs and staff. The Parliamentary Expenses Management System (PEMS) is the IT system used by parliamentarians and their staff to claim office and travel expenses and that processes payroll services for parliamentary staff.

2. PEMS was implemented as a result of a recommendation from a 2016 review into the parliamentary entitlement system that noted the need for a fit-for-purpose online expenses system to replace manual processing and support high quality customer service. The Department of Finance (Finance) was responsible for the delivery of the IT system, with input from the Independent Parliamentary Expenses Authority (IPEA).

3. The project commenced in January 2018 and was delivered through a staged approached with original completion due by June 2020.

Rationale for undertaking the audit

4. The Auditor-General agreed to undertake an audit into PEMS on 4 April 2023.1 The audit was conducted to provide assurance to Parliament on how effective Finance was in successfully delivering the PEMS project. A previous ANAO audit of IPEA, tabled in March 2021, identified delays and budgetary overruns in the implementation of the project.

Audit objective and criteria

5. The audit objective was to assess the effectiveness of Finance’s administration of the PEMS project.

6. The high-level criteria were:

  • Did Finance effectively manage the PEMS project to achieve the agreed deliverables?
  • Does the system meet the agreed deliverables, and achieve outcomes including alignment with relevant legislative requirements?

Conclusion

7. Finance’s administration of the PEMS project was partly effective.

8. Finance established appropriate project governance arrangements, project documentation and risks plans. However, initial planning did not clearly define the scope and user requirements. This led to a significant increase to costs and multiple delays. The project was supported by internal project reporting to governance bodies and external reviews, however these reports did not support sufficient actions or decisions to keep the project on track. Finance has not assessed if the benefits of PEMS have been achieved.

9. The current PEMS capability does not meet all deliverables as agreed in the business case. The functionality delivered meets basic requirements to process office and travel expenses and payroll for parliamentary staff. There is a reliance on manual workarounds particularly for payroll services. Reporting functionality to meet legislative requirements was delayed. Parliamentarians and their staff were not sufficiently consulted throughout the project implementation to ensure the system was simple and easy to use.

Supporting findings

Management of PEMS

10. Finance established appropriate project governance arrangements, including a project board and steering committee which comprised representatives from technical, business and stakeholder areas. Project governance forums operated throughout the project and met regularly, supported by papers and minutes. (See paragraphs 2.2 to 2.8)

11. The PEMS project team developed a project brief and project management plan to support project delivery. However, the initial scope was not clear, there was an inconsistent approach to gathering and agreeing user requirements, and the change management process was not always followed. Original milestones were not met and the project schedule was re-baselined a number of times following delays. This impacted the budget, with Finance reprioritising internal funding towards the project. IPEA provided administered appropriation funding to Finance which was used for the departmental purposes of the project. Risk management processes and documentation were in place to identify and track risks and issues, with risks realised throughout the project. Finance does not have a process in place to track if the benefits of PEMS, as presented in the business case, have been achieved. (See paragraphs 2.9 to 2.47)

12. There was reporting to stakeholders on project status and delivery. This reporting was not effective in supporting appropriate escalation and decision making to resolve project issues. The findings of the Gateway Reviews of PEMS were inconsistent with Finance’s internal reporting on project status and were based on the re-baselined delivery. The performance measure reported in Finance’s annual performance statements does not provide a complete view of PEMS performance. (See paragraphs 2.48 to 2.70)

System and deliverables and capabilities

13. For the functionality that has been delivered, PEMS correctly processes office expenses, travel claims and payroll, supported by manual processing. Controls around the claim process are largely appropriate, however there are a number of manual workarounds in relation to payroll services. There is a backlog of items that need to be implemented in order for the system to meet its scope as set out in the business case and user requirements. (See paragraphs 3.3 to 3.18)

14. The transparent reporting of expenses was delayed. PEMS expenditure reporting functionality was originally due in January 2020, with the expenditure reporting component released to IPEA in November 2023 and IPEA publishing its first reports in December 2023. Based on the design of the reporting component of PEMS the system will support the reporting of expenses. (See paragraphs 3.19 to 3.23)

15. User experience was planned to be considered in project establishment however it was not a focus throughout project implementation. Users of the system have reported that they are not satisfied with the usability of the system. (See paragraphs 3.24 to 3.41)

Recommendations

Recommendation no. 1

Paragraph 2.32

Department of Finance ensures future projects have:

  1. clearly defined and agreed scope and deliverables;
  2. a planned approach to gathering and agreeing user requirements that is followed; and
  3. a process to implement and monitor changes to scope and requirements, including budgetary impacts.

Department of Finance response: Agreed.

Recommendation no. 2

Paragraph 2.46

Department of Finance completes a benefits realisation review by December 2024 to:

  1. track whether PEMS does or will have the future capability to deliver its intended benefits as originally agreed by government; and
  2. establish a process to track and report on the ongoing benefits of PEMS.

Department of Finance response: Agreed.

Summary of entity response

16. The proposed final report was provided to Finance. Extracts of the proposed report were provided to IPEA and the Digital Transformation Agency (DTA). Finance and DTA summary responses to the report are provided below and the full responses from all three entities are at Appendix 1.

Department of Finance

The Department of Finance (Finance) welcomes the report and agrees to the ANAO’s two recommendations in the areas of defining project scope and user requirements more clearly, and conducting a benefits realisation assessment.

Governance arrangements have been significantly strengthened over the past 12 months and have led to the successful delivery of Milestone 8. These will carry through to the shift from project delivery to a new phase of overseeing the future strategic direction of PEMS, including:

  • Clear accountabilities have been introduced for project delivery
  • Transparent and accountable processes are in place for prioritisation of system changes and future enhancements of PEMS
  • User engagement has been reinvigorated with workshops held with Parliamentarians and their staff to understand their needs, and the User Reference Group has been re-established, with regular meetings in place.

Finance would like to acknowledge the commitment and dedication of employees in the department and IPEA who work every day to support Parliamentarians and their staff, including those working to support PEMS.

Finance will continue to work collaboratively with key stakeholders to deliver ongoing system enhancements to improve functionality and the experience of users. This includes continuing the work with the User Reference Group to ensure the client experience is appropriately prioritised.

Independent Parliamentary Expenses Authority

No summary response provided.

Digital Transformation Agency

The Digital Transformation Agency (DTA) acknowledges the findings contained in the extract of the audit report provided for comment, and acknowledges no recommendations have been directed to it. The PEMS program was commissioned prior to the introduction of new assurance requirements administered by the DTA in late 2021. These requirements were introduced as part of the Commonwealth Digital and ICT Investment Oversight Framework (IOF), which supports the Australian Government to manage its digital and ICT-enabled investments — from early planning through to project delivery and realisation of planned benefits. The DTA will carefully consider the findings and recommendations in the full report, and apply them, where applicable, in the administration of the IOF.

Key messages from this audit for all Australian Government entities

Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Group title

Policy/program implementation

Key learning reference
  • When designing and implementing systems entities should have a clear and robust understanding of user requirements and agreed scope to inform budget, timeframes, delivery approach and risk.

1. Background

Introduction

1.1 Parliamentarians are provided with a range of supports and allowances to assist them to carry out their duties including the following.

  • Office expenses for example printing and communication, office stationery and supplies, and property operating expenses (such as cleaning, security and fit out for electoral offices).2
  • Travel expenses for both parliamentarians and staff. This includes for example commercial transport (such as commercial aircraft, trains, taxis), travel allowance to cover accommodation, meals and incidentals for overnight stays and other allowances such as a budget for travel in large electorates.3
  • Staff personal employees’ (such as Chief of Staff, advisor, media advisor or office manager) and ‘electoral employees’ (based in parliamentarians’ electoral offices).4

1.2 The use of office and travel expenses by parliamentarians is covered by:

  • the Parliamentary Business Resources Act 2017 — provides for parliamentarian remuneration, expenses, allowances, and other public resources, it also imposes compliance and enforcement arrangements;
  • Parliamentary Business Resources Regulations 2017 — provides details on travel expenses, travel allowances, office accommodation and equipment, insurance, and legal assistance available; and
  • determinations made under the Parliamentary Business Resources Act 2017 that provide further clarifications on the arrangements.

1.3 Parliamentarians hire staff under the Member of Parliament (Staff) Act 1984 (MOPS Act) to help them fulfill their duties. The Commonwealth Members of Parliament Staff Enterprise Agreement sets the staff employment terms and conditions. Domestic travel expenses for parliamentary staff are generally not included in the Parliamentary Business Resources Act 2017. A determination of the MOPS Act and the Enterprise Agreement guide the eligibility and rules for parliamentary staff domestic travel, for example staff may only travel if directed by their employing parliamentarians and may only travel by the most efficient direct route available.

1.4 The Department of Finance (Finance) and the Independent Parliamentary Expenses Authority (IPEA) support the above arrangements as outlined in Table 1.1.

Table 1.1: Parliamentary expenses by administering agency

 

Agency

Role

Office expenses

 

Finance

Processing and providing advice on claims

IPEA

Conducting reporting and assurance on claims

Travel expenses

IPEA

Processing and providing advice on claims and conducting reporting and assurance on claims

Parliamentary staff payroll services

Finance

Processing on-boarding of staff, termination and paying salaries and supporting work, health and safety

115

75

219

Source: Department of Finance, About MaPS [Internet], Finance, available from https://maps.finance.gov.au/about-maps-and-other-services/about-maps [accessed 29 August 2023].

Independent Parliamentary Expenses Authority (IPEA), Who we are [Internet], IPEA available from https://www.ipea.gov.au/about-ipea/who-we-are [accessed 29 August 2023].

Parliamentary Expenses Management System (PEMS)

1.5 PEMS is the IT system used by parliamentarians and their staff to claim office and travel expenses and perform human resources tasks. Administering agencies also use PEMS to process and pay claims and administer payroll services for parliamentary staff.

1.6 PEMS refers to the combination of an online portal, with a front-end interface, and a back-end SAP enterprise resource planning (ERP) solution.5 Parliamentarians and their staff interact with PEMS via the front-end interface for claims, employee self-services and reporting. IPEA and Finance access both the front-end and the back-end ERP to process and pay claims and administer payroll services for parliamentary staff.

1.7 PEMS was developed in response to the 2016 Independent Parliamentary Entitlements System Review that recommended Finance develop a business case for government consideration for a fit-for-purpose work expenses system.6

1.8 The Australian Government agreed to the second pass business case (business case) for the PEMS project as part of the 2017–18 Mid-Year Economic and Fiscal Outlook (MYEFO) process, including agreement for Finance to deliver the project in-house by June 2020 and provided funding of $38.1 million.

1.9 The project commenced in January 2018 with the aim to deliver a ‘fit-for-purpose online, integrated and agile ICT system to provide self-service capabilities, reduce manual processing and facilitate improved reporting to assist parliamentarians to better manage and monitor expenses.’

1.10 The project was delivered through eight milestones, with changes to scope and timeframes throughout the project (see paragraph 2.24). Milestones 1–5 delivered a front-end portal with online forms connected to the previous IT system (Entitlements Management System). Milestone 6 and Milestone 7 replaced this with the new ERP system for financial management (Milestone 6), payroll services (Milestone 7 phase 1) and office and travel expenses (Milestone 7 phase 2). Milestone 8 covers functionality to support public expenditure reporting. Finance advised the ANAO on 18 December 2023, that the expenditure reporting functionality was released in PEMS in November 2023, with IPEA publishing its first report in December 2023.

Table 1.2: PEMS project milestones

Milestone

Deliverables

Actual release dates

1–5

Release of client portal, eight online forms for travel and office expenses and supporting functionality such as ‘office manager’ (view claims for an office), ability to print forms, notifications and re-assignment of claims

Front end functionality only with connection to the previous Entitlements Management System

Progressive releases from 1 July 2018 to 14 December 2018

6

PEMS back-end financial related functions and additional development to continue online forms until the release of Milestone 7 phase 1 and 2

1 July 2019

7 phase 1

Payroll services functionality, including entry and processing of pay and applying for certain leave types within PEMS

1 July 2021

7 phase 2

Office and travel expenses claim functionality and self-service budget reports for parliamentarians and their staff

1 July 2022

8

Public expenditure reporting for IPEA

December 2023

91

258

150

Source: ANAO analysis of PEMS project documentation.

1.11 There is a backlog of items from the project intended to be delivered over the next two to three years (see paragraph 3.14 to 3.16).7

1.12 Finance built the system, customising a SAP solution, with input from IPEA. The roles and responsibilities are outlined in Table 1.3.

Table 1.3: Finance and IPEA roles and responsibilities in the PEMS project

Business area

Role in PEMS implementation

Service Delivery Office (SDO)

  • Build and technical development of the system (between 1 January 2018 and February 2021 and again from 1 July 2022 to current8)
  • Ongoing management and maintenance of system
  • Represented on the project board and steering committee

ICT Division

  • Lead planning, management, governance and assurance for the project from 1 January 2018 to March 2023
  • Development of the system (between February 2021 and 1 July 2022)
  • Represented on the project board (chair) and steering committee

Ministerial and Parliamentary Services (MaPS) Division

  • Business owners for office expenses and parliamentary staff payroll services functions
  • From March 2023 planning, management, governance and assurance moved from ICT Division to MaPS Division
  • Represented on the project board and steering committee

IPEA

  • Business owner for travel expenses, public expenditure reporting and assurance
  • Represented on the project board and steering committee

130

325

Source: Department of Finance PEMS project documentation.

1.13 In 2022–23 PEMS processed 35,833 office claims and 115,388 travel expense claims.

1.14 In 2022–23 PEMS paid out approximately $153 million in office expenses, $61 million in travel expenses and processed $270 million in payroll for parliamentary staff payroll.9

Rationale for undertaking the audit

1.15 The Auditor-General agreed to undertake an audit into PEMS on 4 April 2023. The audit was conducted to provide assurance to Parliament on how effective Finance was in successfully delivering the PEMS project. A previous ANAO audit of IPEA, tabled in March 2021, identified delays and budgetary overruns in the implementation of the project.

Audit approach

Audit objective, criteria and scope

1.16 The audit objective was to assess the effectiveness of Finance’s administration of the PEMS project.

1.17 The high-level criteria were:

  • Did Finance effectively manage the PEMS project to achieve the agreed deliverables?
  • Does the system meet the agreed deliverables, and achieve outcomes including alignment with relevant legislative requirements?

1.18 The audit scope included assessing Finance’s project management and implementation — project planning, governance, delivery, finances, and reporting — and whether the project delivered the required IT capability.

Audit methodology

1.19 The audit involved examining:

  • governance arrangements throughout the project lifecycle;
  • project documentation such as project plans, schedules, workforce planning and project reviews;
  • evidence that project implementation plans were followed and evidence of change management processes and controls;
  • the project’s financial management and reporting;
  • the risk management approach and evidence of active management of project risks;
  • internal and external reporting on the project including to project stakeholders;
  • a survey of parliamentarians and their staff to assess the user experience;
  • PEMS (through IT demonstrations) to test processes; and
  • meetings with key staff in Finance, IPEA and Digital Transformation Agency.

1.20 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $589,626.

1.21 The team members for this audit were Anne Rainger, Ben Thomson, Renina Boyd, Dr Vivian Turner, Thea Ingold, Kelvin Le, Caitlin Williams, Aaron Ramshaw and Michelle Page.

2. Management of the Parliamentary Expenses Management System project

Areas examined

This chapter examines whether the Department of Finance (Finance) effectively managed the Parliamentary Expenses Management System (PEMS) project to achieve agreed deliverables.

Conclusion

Finance established appropriate project governance arrangements, project documentation and risks plans. However, initial planning did not clearly define the scope and user requirements. This led to a significant increase to costs and multiple delays. The project was supported by internal project reporting to governance bodies and external reviews, however these reports did not support sufficient actions or decisions to keep the project on track. Finance has not assessed if the benefits of PEMS have been achieved.

Areas for improvement

The ANAO made two recommendations: ensuring projects had appropriate approaches to gathering and agreeing business requirements; and Finance undertake a benefits realisation process.

The ANAO identified one opportunity for improvement for Finance to review its performance measure and targets to better track the performance of PEMS and how it relates to Finance’s key activities.

2.1 When managing complex projects, it is important to have fit-for-purpose project planning documentation that clearly identify project scope, requirements, timeframes, and budget. Good governance and risk management enables monitoring of project requirements, sound decision making and ensures project outcomes are achieved. Effective project reporting supports transparency and accountability of project progress.

Was there effective project governance oversight in place?

Finance established appropriate project governance arrangements, including a project board and steering committee which comprised representatives from technical, business and stakeholder areas. Project governance forums operated throughout the project and met regularly, supported by papers and minutes.

2.2 Finance’s project management guidance states that:

  • governance should provide an identifiable, single point of accountability for project delivery;
  • governance structures should reflect the size and complexity of the project;
  • projects require two levels of governance — operational and strategic — that for complex projects should include a project control group (operational) and a project steering committee (strategic); and
  • projects may establish working groups where appropriate to support the project.

PEMS project governance

2.3 The PEMS project governance forums comprised a project board between January 2018 and August 202210 and steering committee throughout the project. A design authority also met from June 2020 to November 2021 to provide guidance, advice, and assurance on scope and business requirements for Milestone 7. As set out in the PEMS project management plan, the governance forums were intended to report to the Project’s Senior Responsible Office (Deputy Secretary, Business Enabling Services)11 who reported into Finance’s enterprise governance structures as outlined in Figure 2.1.

Figure 2.1: PEMS planned project governance roles and responsibilities

A diagram depicting the planned components of the PEMS project governance across responsibilities to govern, manage and deliver. The govern section includes the Secretary, SRO, Executive Board and Steering Committee. The manage section includes the project board. The deliver section includes the ICTD Project Team, SDO and key stakeholders.

Note: While the project governance structures were intended to report to Finance’s Executive Board it was not involved in project governance outside of funding decisions (see paragraph 2.17) and a project status update in September 2023.

Source: ANAO analysis of the PEMS Project Governance Roles and Responsibilities figure from the PEMS project management plan.

2.4 The terms of reference for the project board and steering committee contained information on key areas such as objectives, escalation and delegation of issues, frequency and recording of meetings, and quorum. The terms of reference were not dated nor were changes clearly identified through version control. The remit of the project board and steering committee as per their terms of reference is outlined in Table 2.1.

Table 2.1: Project board and steering committee terms of reference

 

Project board

Steering committee

Objective

  • Provide a unified direction to the project
  • Delegate effectively using the organisational structure and controls designed for this purpose, as set out in the project plan
  • Provide effective decision making
  • Provide visible and sustained support for the project
  • Support effective communication within the project team and with external stakeholders

The steering committee will provide advice and support to the project and to the Senior Responsible Officer as decision maker and signatory authority for the project. The steering committee will consider and endorse recommendations for approval by the Senior Responsible Officer, which will include consideration of options, engagement and communication and strategic direction.

Role

  • Provide support and guidance to the Senior Responsible Officer across the Project
  • Provide direction and oversight on the progress of the project
  • Approve project deliverables
  • Monitor project costs within agreed tolerances and escalate as required
  • Monitor progress, quality, risks and issues within agreed tolerances and escalate as required
  • Communicate with stakeholders as defined in the to be developed stakeholder engagement plan
  • Co-operate with the Gateway Review team on audit and quality assurance aspects of the project, as required
  • Responsible for ensuring the project is delivered in accordance with the Government’s direction, as reflected in the approved business case.
  • Provide strategic direction and advice to the project and project board.
  • Consider and endorse recommendations escalated by the project board to address:
    • quality and supplier management issues;
    • costs that have exceeded agreed tolerances;
    • mitigation actions for risks and issues; and
    • maintain oversight of the delivery of key project outcomes, including organisational change management.

Frequencya

Fortnightly

Monthly

Chair

Project Executive

Senior Responsible Officer

Membership

Service Delivery Office (SDO) representatives, and senior users from the Independent Parliamentary Expenses Authority (IPEA) and Ministerial and Parliamentary Services (MaPS) (SES Band 1 level). Finance’s Chief Financial Officer (CFO) was an observer.

Project Executives, senior users (MaPS SES Band 2 and IPEA CEO), senior supplier (SDO SES Band 2) and representatives from the Digital Transformation Agency, Australian Tax Officeb, and the Department of the Prime Minister and Cabinet.c

104

194

221

Note a: Between January 2018 and August 2018 project board meetings were monthly and steering committee meetings were every second month. The meeting frequency was changed to fortnightly and monthly respectively after recommendations from the second Gateway Review (July 2018).

Note b: The Australian Tax Office representative was on the steering committee until June 2021 when the representative changed job roles and was not replaced.

Note c: From September 2022 the Special Minister of State’s Chief of Staff became an observer on the steering committee. From March 2023 an independent external advisor (who was a former Gateway Reviewer of PEMS) also joined the steering committee.

Source: Project board and steering committee terms of reference.

2.5 In line with Finance’s internal project management guidance, the project board was set up to monitor progress of the project on an operational level. The steering committee was set up to provide oversight and strategic advice to the project board and Senior Responsible Officer as the decision maker.12

2.6 The project board and steering committee met as per the terms of reference. Meetings were supported by meeting papers, action items were tracked and meeting minutes recorded. The ANAO reviewed project board papers and minutes from January 2019 to August 2022 and steering committee papers and minutes for the period January 2019 to April 2023. Project updates, schedule, risks, issues, budget and change management were consistently discussed.

2.7 In March 2023 the steering committee agreed to revised governance arrangements to support ‘business as usual’ management of PEMS and the project moving from ICT division (as an IT-led project) to MaPS division (business-led project). The new arrangements included continuing the steering committee13, a PEMS management board (previously the project board), and PEMS working group (with a key focus being the prioritisation of the backlog as outlined in paragraph 3.14 to 3.16). The steering committee and management board terms of reference were comparable to their predecessors in terms of roles and delineation between the forums. The steering committee terms of reference were amended to include the committee actively incorporating the experience of users in decision making.

2.8 Other project governance roles are outlined below.

  • Senior Responsible Officer/Project Sponsor — chair of the steering committee, oversight and approval, ultimate responsibility for achieving the objectives and benefits. This was the Deputy Secretary, Business Enabling Services.
  • Project Executive — chair of the project board, oversight and approval, authorise project status and transitions. This was the First Assistant Secretary of the Information, Communication and Technology (ICT) Division who led the project between 1 January 2018 and March 2023. This was the First Assistant Secretary of MaPS from March 2023.
  • Senior Supplier — responsible for producing agreed deliverables, technical integrity and quality of project and represents those designing, developing and procuring the project products. This was the First Assistant Secretary of SDO between 1 January 2018 and February 2021 and from 1 July 2022. The First Assistant Secretary ICT Division (also the project board chair and Project Executive) was the Senior Supplier between February 2021 and 1 July 2022 (covering the period when responsibility for the technical build moved to the ICT Division).

Were there fit-for-purpose project implementation plans in place which were followed by Finance?

The PEMS project team developed a project brief and project management plan to support project delivery. However, the initial scope was not clear, there was inconsistent approach to gathering and agreeing user requirements, and the change management process was not always followed. Original milestones were not met and the project schedule was re-baselined a number of times following delays. This impacted the budget, with Finance reprioritising internal funding towards the project. IPEA provided administered appropriation funding to Finance which was used for the departmental purposes off the project. Risk management processes and documentation were in place to identify and track risks and issues, with risks realised throughout the project. Finance does not have a process in place to track if the benefits of PEMS, as presented in the business case, have been achieved.

Project planning

2.9 The initial project documentation for the PEMS project included a project brief, project management plan and an initial risk and issues register.

2.10 Finance’s guidance outlines that initial project documentation should be provided to the Project Sponsor and governance bodies for review, comment and final approval. The approval process for the initial project documentation did not follow the internal guidance as outlined in Table 2.2. Finance was also unable to provide final signed copies of initial project documentation.

Table 2.2: Initial PEMS project documentation and approval

Document

Governance body

Approval

Project brief

 

Project board — discussed at 30 January 2018 meeting

Approved 28 February 2018

Steering committee — provided to 14 February 2018 meeting

No discussion or approval recorded in meeting minutes

Project management plan

 

Project board — provided to 29 March 2018 meeting

No approval recorded in meeting minutesa

Steering committee — not provided to steering committee

N/A

Initial risk and issue register

 

Project board — completed risk register initially provided to 29 March 2018 meeting

Discussion but no approvalb

Steering committee — updated risk register provided to 6 April 2018 meeting

Discussion but no approval

113

194

168

Note a: A Gateway Review (see paragraph 2.53) in March 2018 included a critical recommendation that a project management plan for PEMS is completed. A subsequent Gateway Review in July 2018 outlined that the action was partially implemented and no date of approval for the project plan was provided.

Note b: The risk register was an operational/living document, with updates discussed at risk workshops and provided to governance bodies with meeting papers.

Source: Initial PEMS project documentation and governance papers.

2.11 The project brief and project management plan included appropriate information specific to the PEMS project, except for sufficient details on scope (see paragraph 2.14) and budget. The 13 capabilities detailed in the project brief did not specify clear or sufficiently detailed deliverables, and budget was provided as a total project cost with no breakdown of how it was to be spent.

2.12 The project management plan was updated in response to the significant changes to scope and timeline — creation of Milestone 7 (plan updated July 2019), delay of Milestone 7 from April 2020 to July 2021 (plan updated January 2021) and splitting Milestone 7 into phase 1 and 2 (plan updated February 2021). A project management plan was not in place to support the delivery of Milestone 8.

2.13 The revised project management plan was approved by the project board each time the project was re-baselined. The approved project plan was provided to the steering committee for visibility, not endorsement as per Finance’s internal project management guidance. The updated plans were also not finalised following the review by governance forums.14

Project scope

2.14 During the project Finance did not have a clear scope of the PEMS project agreed with MaPS and IPEA as business owners. Scope was inconsistently represented throughout project documentation, presented as 13 core capabilities (see paragraph 3.2) and as high-level milestone deliverables.

2.15 Unclear and inconsistent scope definition created risks and issues throughout the project as evidenced by:

  • Gateway Review recommendations — outlined the critical need to finalise business requirements, including governance sign-off;
  • discussion surrounding scope at project board and steering committee meetings;
  • risks and issues included in the risk register (i.e. scope is referenced in 14 risks and six issues from a sample risk register dated 12 April 2022);
  • scope being reported in the project status report as amber or trending down for eight months between September 2019 and May 2020;
  • recommendations from a review of the PEMS build (as at 20 November 2020) conducted by SAP that found there needed to be alignment between business, SDO and ICT Division to ensure functional scope was delivered; and
  • scope items being moved to the backlog of outstanding deliverables (see paragraph 3.14).

Budget

2.16 Through the 2017–18 Mid-Year Economic and Fiscal Outlook (MYEFO) process the Australian Government approved $38.1 million (to 2020–21) for the development of PEMS.15 The initial funding included project funding, comprising a mix of capital and operating funding, and ongoing funding to support the operating and maintenance costs of PEMS. Throughout the project, Finance made internal reallocations of funding towards the project and IPEA also made contributions. The PEMS funding and expenditure breakdown is outlined in Table 2.3.

Table 2.3: PEMS funding and expenditure 2017–18 to 2022–23

 

Initial funding 2017–18 MYEFO for PEMS project

$m

Reallocated funding from Finance and IPEA

$m

Total funding

$m

Actual spend

$m

PEMS project budget

30.2a

29.1b

59.3

60.7

Operating and maintenance budget

14.9c

0

14.9

13.6

Total PEMS

45.1

29.1

74.2

74.3

117

144

148

75

66

Note a: The initial MYEFO decision provided for $38.1 million, including $30.2 million in capital funding for PEMS.

Note b: Finance provided a total of $23.4 million and IPEA provided $5.7 million, including $5.0 million in June 2020 and $0.7 million in 2021–22 from its administered funding (see paragraph 2.18).

Note c: The initial MYEFO decision included $2.7 million for operating and maintenance costs for 2019–20, $4.0 million for 2020–21 and then $4.1 million per financial year ongoing funding. This figure includes the initial $6.7 million to 2020–21 from the initial MYEFO announcement and $4.1 million per annum for 2021–22 and 2022–23.

Source: PEMS budget breakdown provided by Finance.

2.17 Finance’s main internal funding reallocations were provided in June 2019, December 2020, and March 2021. These budget injections were approved by Finance’s Executive Board. Specific papers were provided to the Executive Board outlining the need for the additional funding for the March 2021 decision. The June 2019 and December 2020 decisions were part of Finance’s broader process for requesting internal budget adjustments. The PEMS project was included in the overall request for bids.

2.18 IPEA contributed $5.7 million to the project from its administered appropriation related to Outcome 1, consisting of $5.0 million in June 2020 and a second transfer of $0.7 million in 2021–22.16 This was used by Finance for departmental expenditure on the PEMS project.17 There was no documented evidence of Finance’s decision making and how Finance considered the relevant frameworks to apply to the transactions, noting that money initially appropriated for administered purposes was used for departmental purposes. This increases potential risks to the controls put in place by the Parliament, in particular to ensure that appropriations are used for the intended purpose and with the Parliament’s permission.18

2.19 Finance advised the ANAO on 4 October 2023 that total expenditure to 30 June 2023 was $74.3 million, including $60.7 million from the PEMS project budget. This represents a 96 per cent increase from the original PEMS project budget. In Finance’s financial statements as at 30 June 2023, the total expenditure is reflected as $45.9 million of software assets. The remaining $28.5 million spent has been recognised as operating expense through 2017–18 to 2022–23, including $3.9 million of impairment.

2.20 In addition, Finance was provided $4.0 million in the 2023–24 Federal Budget for further development of PEMS during 2023–24 (see paragraphs 3.14 to 3.16).

Workforce

2.21 Finance advised the ANAO that the majority of expenditure for the PEMS project was on contractors, consultants, outsourcing and employees.19 For Finance to build PEMS in-house the business case stated that a fully contracted workforce would be required. Finance advised that the project workforce planning was covered under an initial procurement plan. This document is the approval to approach the market for resources for PEMS and another project. It was not a discrete document that maps the resources or capability required over the project and it did not specify what resources were specifically for PEMS. The initial PEMS project management plan listed the areas resources would be required (for example, organisational change management and technical architecture and configuration) and included a high-level structure of the SDO delivery teams. A resourcing plan was implemented for the updated project management plan for Milestone 7. The plan covered July 2019 to June 2020, and outlined the roles required, expected effort (represented as estimated hours/days of work required and full-time equivalent resourcing) according to build component (for example, payroll services, travel, office and budget).

2.22 During the project additional resources were required. Finance advised the ANAO that the decision to procure additional project resources was managed through changes to the integrated project plan and schedule. Updates were made to the integrated schedule throughout the project. However, Finance was unable to demonstrate how changes to resource requirements were operationalised. For example, when the schedule was delayed there was no evidence that the resource plan was updated in response. Conversely, when there was budget overspend there was no separate documentation or process that captured a new resourcing profile to reduce expenditure.

Project schedule and timing

2.23 The business case, project brief and project management plan included the proposed timeline for project delivery, outlined through a series of milestones. Each milestone had a brief description of the expected functionality to be delivered and the proposed delivery date. The milestone descriptions were presented differently to the way project scope was detailed, which was through the 13 core capabilities (see Table 3.1).

2.24 The project experienced delays in particular for the delivery of the Enterprise Resource Planning (ERP) solution for human resources, office and travel expenses and reporting. In addition, milestones were re-named, functionality was moved between milestones and scope not delivered as further discussed in paragraphs 3.8 to 3.16. The changes to the milestones are set out in Appendix 3. Key points in the delivery timeline are outlined below.

  • The original Milestones 1 to 5 were largely delivered on time with minor changes to timeframes and milestones re-named.20
  • Milestone 6 was originally the delivery of the ERP solution, including implementing financial backend related functionality, human resources and integrated expenses management for office and travel claims. The original due date for Milestone 6 was 1 July 2019. In February 2019 the steering committee agreed to deliver the financial backend functionality on 1 July 2019 and deliver the remaining functionality for the project in Milestone 7 with a new delivery date of April 2020.
  • In September 2019 the steering committee agreed to delay the delivery of Milestone 7 to 1 July 2020 and in May 2020 it agreed to further delay the delivery to 1 July 2021.
  • In December 2020 the steering committee agreed to split Milestone 7 into two phases — phase 1 for human resources (due 1 July 2021) and phase 2 for expense management of office and travel claims and expenditure reporting (with delivery date to be set later).
  • In July 2021 the steering committee agreed to set the delivery date of Milestone 7 phase 2 as April 2022. In December 2021 the steering committee agreed to move this to July 2022.
  • Further delays occurred for the expenditure reporting component of PEMS, discussed further in paragraph 3.20.

2.25 Decisions to delay milestone releases were approved by governance forums (both project board and steering committee), with the reasons for the main delays outlined in Table 2.4.

Table 2.4: Reasons provided to project governance on key delays in the project schedule

Steering committee decision date

Re-baseline decision

Reason provided

February 2019

Milestone 7 created with due date of April 2020

Upcoming federal election increased activity for MaPS and IPEA which created risk of diverting MaPS and IPEA project resources, slippage within the project schedule and there were in-scope items that could not be delivered if the release date was not delayed.

September 2019

Milestone 7 delayed from April to July 2020

Reported resource shortage affecting development of work. Design requirements not finalised, issues with testing and delays to key components of the build.

May 2020

Milestone 7 delay from July 2020 to July 2021

Lack of specialist resources to support reporting, user acceptance testing and data validation and migration. Testing behind schedule. Data designs do not support business needs and require re-design.

December 2020

Split Milestone 7 into phase 1 (July 2021) and phase 2 (delivery date to be set latera)

Lack of SDO technical resources, six-month pause in build work.b In addition, issues with parts of the build (reporting and expense management) would affect delivery times.

December 2021

Milestone 7 phase 2 delayed from April 2022 to July 2022

Testing was behind schedule. Potential election to occur at time of release was considered high risk due to disruption with end users and increased workload of MaPS and IPEA. Reporting functionality would not be completed to a robust level before April 2022.

133

108

170

Note a: The steering committee decided in July 2021 to set phase 2 as April 2022.

Note b: The build component of the project was put on hold due to a fraud case which resulted in termination of IT resources on the project and the need to hire replacement IT resources.

Source: ANAO analysis of PEMS project and governance documentation.

2.26 The project delays increased the project length from the proposed two and a half years to more than five years duration. This three-year delay has impacted the project resources required.

User requirements

2.27 Finance did not have a consistent approach to gathering business requirements, in particular for the full ERP solution for human resources and office and travel expenses. Various approaches were used to gather and confirm requirements throughout the project.

  • High-level requirements developed by Finance in 2017 that informed the IT solution (as an ‘commercial off-the-shelf’ product) and the intended core capabilities of PEMS.
  • Business process models — documents developed by the SDO between August 2018 and November 2020 that articulate system business rules.
  • User stories — as a parallel process to the business process models. Between August and October 2018 Finance worked with IPEA to develop user stories to cover a high-level statement of what the system should do.
  • Knowledge transfer workshops — demonstration workshops were held in March 2020 to show and explain the solution to business areas to assist with user acceptance testing. Subsequent reporting to the project board showed these sessions highlighted design gaps that required further IT build work.
  • Solution playback sessions — these sessions were held between May 2020 and June 2020 to identify system operability issues and gaps between what was required functionality and what functionality was available in the system at the time.
  • List of ‘essential items’for Milestone 7 phase 2 (July 2022) a list of ‘essential items21’ was developed (informed by the solution playback sessions and the work of the design authority). This would form the minimum viable product (MVP) for delivery of phase 2.

2.28 Finance did not articulate the connection between the above approaches and whether these approaches captured a complete view of user requirements to inform the PEMS build.

2.29 Throughout the project there were delays in agreeing business requirements. The project status report (see paragraph 2.48) consistently reported requirements definition (under ‘design’) as red and amber throughout the project. On 26 November 2019 the steering committee agreed to ‘baseline22’ the requirements in the business process model documents without final approval from IPEA as business owners of the travel expenses component. The steering committee minutes noted that IPEA did not agree to this approach. There is no evidence that the travel expenses requirements in the baselined documents were subsequently formally agreed by Finance and IPEA.

2.30 The delays in agreeing business requirements resulted in Finance building and customising the system, as early as February 2019, without requirements being agreed by business owners. Finance called this ‘building at risk’ and the approach was noted in discussions at governance forums and in risk registers. The risk of this approach was realised when the system build had to be re-worked when it was identified that the solution would not meet business requirements.

2.31 Unclear requirements and poorly defined scope at the start of the project also led to incorrect assumptions on the amount of customisation required from a ‘commercial off-the-shelf’ IT system. The initial project plan had an assumption that no more than 10 per cent of the system would need to be configured. Finance advised that it did not track the percentage of configuration required, and that office and travel expense management was a largely bespoke system and Milestone 8 is an entirely bespoke system. This meant that more IT build work was required than planned, leading to schedule delays and continued increases in project cost.

Recommendation no.1

2.32 Department of Finance ensures future projects have:

  1. clearly defined and agreed scope and deliverables;
  2. a planned approach to gathering and agreeing user requirements that is followed; and
  3. a process to implement and monitor changes to scope and requirements, including budgetary impacts.

Department of Finance response: Agreed.

2.33 Finance has implemented significant improvement initiatives in PEMS, revising the approach to governance, processes, practices, and project outcomes. Finance will incorporate the ANAO recommendations as part of the design and delivery of future PEMS improvements. Finance has also recently undertaken an audit into its approach to delivering complex projects. Outcomes and lessons learnt from these audits will be incorporated into future PEMS project delivery work and shared for broader application across the department and the APS.

Project change management

2.34 The project management plan outlined the approach to managing changes to the ‘approved baseline for the project’.23 This included a change request template to capture the details and impact of the change and a change register to track requests over the project.

2.35 There is no evidence that the governance bodies used this process to effectively manage change in the project. Separate scope decisions were being made by business areas without undergoing the change request process. For example, MaPS approved changes in 2019 to the scope of human resources capability to remove on-boarding, personnel administration, time administration and organisational planning. Further, in addition to the change request process, system requirements that were not delivered were placed on a backlog (see paragraphs 3.14 to 3.16). Finance’s processes were unclear on what items should have gone through the change request process and what decisions could be made outside the process.24

Risk management

2.36 The PEMS project was initially rated as a medium risk project as outlined in the business case. During the project, significant changes to schedule, budget and scope occurred. There is no evidence that Finance reevaluated the overall project risk rating during the project.

2.37 The PEMS project had risk documentation that aligned to Finance’s enterprise risk management guidance from 2017. Project risk documentation and risk management activities for the project included:

  • risk register;
  • risk workshops;
  • PEMS project risk and issue management plan; and
  • risk reporting.

2.38 The risk register was established at project onset. The risk register was updated for individual risks regularly, and included risk description, source/cause, date raised, current controls, proposed treatments and risk owners. Project issues were also included in the risk register.25

2.39 Finance advised that risk workshops were held approximately monthly, were not formal meetings, minutes were not kept and that the workshops were responsible for maintaining and updating the risk register. Finance also advised meetings included Project Executive, Project Manager/Director and representatives from IPEA, MaPS and the SDO.

2.40 The PEMS project risk and issue management plan was established 18 months into the project, in July 2019. It provides high-level information on how risks and issues are to be managed throughout the project.

2.41 Risks and issues were reported at every project board and steering committee meeting and were included in the project status reports provided to governance forums (see paragraph 2.48).

2.42 Although risk documentation and risk management procedures were in place, it did not necessarily prevent risks being realised. Table 2.5 provides an example of a key risk, including risk description, source/cause of the risk, current controls and proposed treatments.

Table 2.5: How budget risks were managed through the risk register

 

Detail provided in risk register

Risk description

Risk #42 — Cost to deliver the project will exceed budget tolerance

Date raised — 29 October 2019

Source/cause

  • The effort and cost to deliver is higher than estimated
  • Delivery of the project extends beyond July 2020

Current controls

  • Review SDO and ICTD effort and costs
  • Re-forecast effort and cost to deliver in July 2020
  • Closely monitor resource allocations and monthly spend
  • Review spend to date and journal non project related costs to correct cost centre
  • Escalate issues in accordance with the project management plan
  • Consider scope priorities for July 2021 with a view to delay less important functions until post go-live

Proposed treatments

  • Executive Board consideration of business case for additional funding to deliver PEMS

93

233

Note: The above table provides a snapshot of how budget risks were addressed throughout the project. Risk #42 was the first risk to address budget issues in October 2019. Additional risks were raised in May 2020 and June 2022 to address separate budget issues. This included potentially ceasing the project unless additional funding is sourced, and the cost of enhancements was not affordable without further funding. A project issue was raised in response to the realisation of several risks in October 2022, with the cost to deliver the remainder of the project exceeding the available funds. This issue has not been closed.

Source: Risk registers provided by Finance.

2.43 The ANAO assessed the effectiveness of the treatment in managing the budget risk. The risk controls in the risk registers are focused on reducing the risk through monitoring or remediation once the risk is realised, i.e. seeking additional funding and re-directing existing Finance employees to the project instead of using contractors. However, the project had budgetary risk throughout the project and both the monitoring and reactionary remedial actions did not effectively manage the risk of budget overspend.

Benefits realisation

2.44 The benefits of PEMS to parliamentarians, administering agencies and government were outlined in the business case. These included: client centric service delivery; improved administration efficiency and effectiveness; increased transparency of parliamentarian expenses; providing agility in enabling ICT to cater for future changes; and value for money by minimising ICT capital investment. Benefits management was included in the project brief, the initial project management plan and the updated plans.

2.45 At times during the project Finance attempted to develop processes to track benefits, for example a benefits realisation plan was developed in 2019 and updated in 2021. Given the delays to the project, reduced scope and user views of the system, coupled with Finance not having a current process in place to track benefits, there is no evidence that the expected benefits of PEMS have been realised. Finance advised it intends to undertake work on benefits realisation as part of the two to three year strategy for PEMS improvements, with the approved strategy due early 2024.

Recommendation no.2

2.46 Department of Finance completes a benefits realisation review by December 2024 to:

  1. track whether PEMS does or will have the future capability to deliver its intended benefits as originally agreed by government; and
  2. establish a process to track and report on the ongoing benefits of PEMS.

Department of Finance response: Agreed.

2.47 A benefits realisation review is in progress. Feedback from this audit will be incorporated into the approach, which will be finalised in the first half of 2024.

Was there effective reporting to relevant stakeholders?

There was reporting to stakeholders on project status and delivery. This reporting was not effective in supporting appropriate escalation and decision making to resolve project issues. The findings of the Gateway Reviews of PEMS were inconsistent with Finance’s internal reporting on project status and were based on the re-baselined delivery. The performance measure reported in Finance’s annual performance statements does not provide a complete view of PEMS performance.

Project reporting

Project status reports

2.48 Project status reports were provided to the project board from September 2018 until August 2022 and to the steering committee until March 2023. The reports contained an overview of the project status and a range of subcategories aligned with project components, such as schedule, resources and testing. The report used a ‘traffic light’ system to rate the overall project status and against the separate subcategories, with accompanying points to provide context to the ratings. Figure 2.2 shows the overall status results reported to the project board and steering committee.

Figure 2.2: Overall status of the project from 2018–2023 as reported in the project status reports

 

Jan

Feb

March

April

May

June

July

Aug

Sept

Oct

Nov

Dec

2018

 

 

 

 

 

 

 

 

2019

2020

2021

2022

2023

 

 

 

 

 

 

 

 

 

 

53

53

53

64

53

53

55

49

49

53

53

53

53

Key:  = project status reported as red (one or more of the subcategories were rated red).

= project status reported as amber (more than one subcategory was rated amber but no red in the subcategories).

= project status reported as green (all subcategories rated as green or only one amber subcategory, with no red rated subcategory).

Note A steering committee meeting was not held in January 2023.

Source: ANAO analysis of project status reports.

2.49 The project status reports provided sufficient detail on project implementation. However, throughout the reporting of the project ‘risks’, ‘issues’, ‘governance’ and ‘project management’ continued to be rated as green. The description of a green rating in the report was ‘identified issues being managed and updated regularly’. The updates in the report under these subcategories were more around routine matters rather than providing a clear view of project issues. For example, under ‘governance’ the update was the next meeting date and under ‘risk’ the update was how many risks were in the risk register (regardless of their severity). This approach to reporting masked the overall project health.

Other operational reporting

2.50 Other operational reports were produced to support the project, for example:

  • PEMS weekly status report — these reports were originally a two–four page summary by respective areas of the project (for example SDO, ICT Division and data migration). The format changed from May 2020 to support increased reporting in response to delaying Milestone 7. Key components of the updated report were a traffic light assessment across project components (for example, schedule, testing), key activities in the previous and upcoming week, escalation of issues and key milestones. Finance advised these were provided to the Project Executive and business areas (MaPS and IPEA) involved in the project teams.
  • Adoption tracker — the project board received statistics on the usage of PEMS (smart form solution) from October 2019 to September 2021. The statistics included the number of users and transactions performed. Meeting minutes from the project board did not demonstrate discussion of the contents of the tracker.
  • Routine updates were also provided to the Major Projects Committee between July 2021 and September 2023.26 This committee did not have governance responsibility for PEMS and the project was presented for information only.
Ministerial briefings

2.51 Finance provided briefings to the Minister for Finance and the Special Minister of State as outlined in Table 2.6.

Table 2.6: Summary of briefs on PEMS provided to Ministers during the project

Date

Minister briefed

Summary

March 2018

Finance

Noted the initial progress of the project and requested signing of a circular to assist in the recruitment of parliamentarians for the user reference group.

June 2018

Finance

Noted an update on the PEMS initial releases and support for early adoption of PEMS.

November 2020

Finance

Brief notes PEMS is not on track due to delays in user acceptance testing, unanticipated need to acquire specialised resources and complexity in build due to unique parliamentary process. Mitigations to this are re-baselining the project release date for office and travel expenses.

March 2021

Finance

Noting an update on the projects and steps taken to progress the project by 1 July 2022.

July 2021

Finance

Noting the two amber ratings received through the Gateway Reviews process (see paragraph 2.61).

September 2022

Special Minister of State

Noting the steps being undertaken to address the issues with PEMS. The brief indicated a ‘minimal viable product’ for expenditure report would be delivered by 31 October 2022.

 

October 2022

Finance

133

133

172

Note: PEMS was also covered in the incoming government brief for the Special Minister of State.

There was a gap in briefings between July 2018 and October 2020. Finance advised that this was in part due to the 2019 election and ministerial and machinery of government changes and the Australian Government and Finance responding to natural disasters and the COVID-19 pandemic.

Source: ANAO review of ministerial briefs provided by Finance.

Secretary briefings

2.52 Three written briefs were provided to the Secretary of Finance (September 2018, November 2020 and June 2021) on the PEMS rollout and project issues. Operational updates on project activities were also provided.27 The Executive Board of Finance received updates on PEMS in February 2021 and when it agreed to provide additional funding to the project in June 2019, December 2020 and March 2021 (see paragraph 2.17). The Secretary is chair of the Executive Board.

Gateway Reviews

2.53 Finance has responsibility for the Gateway Review policy and co-ordination of Gateway Reviews across non-corporate Commonwealth entities. Gateway Reviews aim to facilitate effective planning, management and delivery of major IT projects being delivered in entities.28 Gateway Review reports are intended to be given to the Senior Responsible Officer to provide an assessment of issues that may jeopardise delivery and benefits. Five of the seven PEMS Gateway Reviews were addressed to the Project Executive as Senior Responsible Officer and not the actual Senior Responsible Officer (Deputy Secretary).

2.54 The business case approved by government for PEMS included that the project would be subject to Gateway Reviews. As at October 2023, the project has undergone seven Gateway Reviews as outlined in Table 2.7.

Table 2.7: Summary of Gateway Reviews and outcomes

Focus of the reviewa

Completed

No. of recommendations

Overall ratingb

Combined gate 0–3 (business need, business case, delivery strategy and investment decision)

March 2018

8

Amber

Gate 4 (readiness for service review)

 

July 2018

10

Amber

May 2019

8

Green/Amber

Gate 4/mid-stage program review

 

 

 

February 2021

9

Amber

May 2021

9

Amber

April 2022

8

Green/Amber

December 2022

9

Amber

113

128

152

71

Note a: The Gateway Reviews process includes multiple reviews throughout the project at key points (referred to as ‘gates’ for project or ‘review stages’ for programs which have interrelated projects). Each review has a specific focus and the number of reviews is determined by the complexity and timeframes of the project or program. Depending on structure of the program or project a blended approach between gates and review stages can occur. More information is available in Guidance on Assurance Reviews Process, Resource Management Guide 106 [Internet], Department of Finance, available from https://www.finance.gov.au/publications/resource-management-guides/guid…- [accessed 4 November 2023].

Note b: The PEMS project had a combined review for gate 0–3 in March 2018 given the first deliverable was due in July 2018. There were multiple reviews for stage 4 with each looking at different milestone delivery. A blended approach was adopted from February 2021 to combine the review points of a project and program.

Note c: Red was defined as: ‘Successful delivery of the program/project appears to be unachievable. There are major issues on program/project definition, schedule, budget, quality or benefits delivery. The program/project may need to be re-baselined and/or overall viability re-assessed.’

Note d: Amber was defined as: ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears feasible but significant issues already exist requiring management attention. These need to be addressed promptly.’

Note e: Green/Amber was defined as: ‘Successful delivery of the program to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery.’

Note f: Green was defined as: ‘Successful delivery of the program/project to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly.’

Source: Summarised by ANAO from Gateway Review reports and meeting minutes from the project board and steering committee.

2.55 ANAO analysis of gateway review ratings against project status report ratings for the overall project status (see paragraph 2.48) shows discrepancies as outlined in Figure 2.3.

Figure 2.3: Gateway Reviews and project status reporting timeline

A horizontal timeline of January 2018 till December 2023 showing the seven Gateway Reviews’ overall ratings compared to the overall status of the project as reported in the monthly project status report. The Gateway Reviews are rated green/amber or amber. The overall project status reports are rated red, amber and green.

Note: Project status reports started in September 2018.

The May 2019 red rating in the project status report was due to delays in the schedule for data migration and preparation for the technical cutover (series of activities to prepare for the new system) for Milestone 6. The delay was resolved by the next report.

Source: ANAO analysis of Gateway Review reports and PEMS project status reports.

2.56 Factors impacting the overall rating in each review are outlined below.

  • March 2018 — rated as amber with the report noting that Finance was using a ‘commercial off-the-shelf’ solution and implemented by the SDO that has used the technologies with other agencies, which lowers the technical implementation risk. The review however noted the project was under schedule pressure, it was unclear whether a ‘commercial off-the-shelf’ solution would meet the expectations of parliamentarians and their staff and that extensive communication and organisational change management was required for successful implementation.
  • July 2018 — rated as amber noting the schedule continued to be under pressure and it was still unclear if the scope of the work would meet expectations of all stakeholders or if the budget would be able to meet emerging requirements from business areas. The review noted that if the recommendation to capture and prioritise user needs and requirements is not implemented promptly the project may become unachievable.29
  • May 2019 — rated as green/amber noting progress was being made, including implementation readiness for financial related functions (Milestone 6); the new delivery date of April 2020 is achievable and that it was anticipated that Finance’s Executive Board would fund increased scope.
  • February 2021 — rated as amber noting issues outside the control of the project during 2020 (for example, IT build paused due to loss of technical resources and need to re-hire) that necessitated replanning and re-baselining. The report also noted issues involving clearer responsibilities and accountability within the project, business readiness and operational support. The report did note the considerable work in progress as a mitigating factor.
  • May 2021 — rated as amber due to issues with the readiness for the next phase (Milestone 7 phase 2) and with stakeholders/end-user engagement. The review stated these issues were mitigated with the planned successful delivery of Milestone 7 phase 1 and green ratings for governance and planning and risk management. Although Milestone 7 phase 1 was delivered on time, it was with reduced functionality (see paragraphs 3.10 to 3.16).
  • April 2022 — rated as green/amber with delivery of Milestone 7 phase 2 (due 1 July 2022) a ‘high probability’. The report noted issues in arrangements for post ‘go-live’, deferral of expenditure reporting and testing and defect remediation.
  • December 2022 — rated as amber noting PEMS had ‘been delivered and is operating’ and the schedule for expenditure reporting (Milestone 8) ‘appears reasonable’. However, the report notes considerable stakeholder concerns, the budget is exhausted and work is required to explore usability issues.

2.57 The definition of an overall red rating was met during the project, for example there were ongoing issues with the scope of the project, the schedule (and therefore benefits realisation) was re-baselined several times, and the budget was supplemented by additional funding. However, Gateway Reviews are a point in time assessment and the PEMS reviews occurred after the project was re-baselined. The reviews therefore looked at the overall likelihood of success for the project based on the new parameters. This approach potentially reduced the impact of the Gateway Reviews in being able to appropriately escalate issues for action by the Senior Responsible Officer.

2.58 The December 2022 Gateway Review listed two further reviews that needed to be completed. The first in May 2023 to confirm the readiness of Milestone 8 and the second in December 2023 to conclude the project. The review covering Milestone 8 commenced in September 2023.30

2.59 The ANAO examined the recommendations made in the Gateway Review reports and their level of urgency as set by the Gateway Review teams, shown in Figure 2.4.

Figure 2.4: Number of recommendations raised in review reports for March 2018 to December 2022 (broadly categorised by ANAO) and their assigned urgency

A horizontal bar chart describing the number of recommendations raised in review reports for March 2018 to December 2022. They are categorised into security, risk management, future gateway review planning, business requirements and documentation, business case and benefits, stakeholders and end users, planning and governance. The horizontal bars are broken up by their urgency of critical, essential or recommended.

Note: The Gateway Review recommendation categories are:

Note a: Critical (do now): ‘To increase the likelihood of a successful outcome it is of the greatest importance that the program should take action immediately.’

Note b: Essential (do by): To increase the likelihood of a successful outcome the program should take action in the near future. Whenever possible essential recommendations should be linked to program milestones (for example, before contract signature and/or a specified timeframe i.e. within the next three months).

Note c: Recommended: The program should benefit from the uptake of this recommendation. If possible, recommendations should be linked to program milestones (for example, before contract signature and/or a specified timeframe i.e. within the next three months).

Source: ANAO analysis of Gateway Review reports

2.60 When progress of the recommendations was reviewed in subsequent Gateway Reviews, recommendations in the planning and governance categories were more likely to be fully met, as shown in Figure 2.5. Recommendations in the stakeholders and end users category were predominantly partially met, with the implementation of some recommendations remaining unclear to the gateway reviewers.

Figure 2.5: Outcomes of the Gateway Review recommendations for March 2018 to April 2022 as detailed in the subsequent Gateway Review

A horizontal bar graph depicting the outcomes of the gateway review recommendations from March 2018 to April 2022 as detailed in the subsequent gateway review. The bars are broken up into categories of security, risk management, future gateway review planning, business requirements and documentation, business case and benefits, stakeholders and end users, planning and governance. The horizontal bars are broken up by the outcomes of not addressed, unclear, fully, partial, ongoing or N/A (related to a separat

Note: This figure does not include if recommendations from the December 2022 Gateway Review have been completed as the subsequent Gateway Review has been delayed and not conducted as at October 2023.

The N/A relates to a project to support the COMCAR Automated Resource System (CARS project). This project was included in the PEMS second pass business case but managed as a separate project. The July 2018 Gateway Review was a combined PEMS and CARS review with some recommendations for the CARS project. These are represented as a N/A in the figure as the PEMS project did not need to implement these.

Source: ANAO analysis of Gateway Review reports.

2.61 Gateway Reviews have an enhanced notification process which requires escalation of issues to an agency’s accountable authority and/or Minister. This process is triggered by red ratings or sequential amber or amber/red ratings.31 The project twice received two consecutive amber ratings that were then followed by a green/amber rating, triggering the first level of enhanced notification. Finance complied with the enhanced notification process after both sets of consecutive amber ratings.32

External reporting

External performance reporting

2.62 The Public Governance, Performance and Accountability Act 2013 (PGPA Act) establishes a framework for monitoring and evaluating performance that requires entities to have meaningful performance information which relies upon a clear understanding of the entity’s purpose and expressing it in a way that is related, measurable and complete.

2.63 PEMS was included in the performance measures in Finance’s corporate plan and annual reports from 2017–18 to 2023–24. The 2022–23 measure is depicted in Table 2.8.33

Table 2.8: Finance performance measure for PEMS 2022–23

Program

Key activities

Performance measure and description

Performance target and description

Delivering effective services to, and for, government — key activities and performance measures

Deliver ministerial and parliamentary services: Provide a range of services to Parliamentarians, their employees and others as determined by the Australian Government to assist them in undertaking their duties.

Improve administration of parliamentary work expenses — the timeliness, efficiency, clarity and transparency of the administration of parliamentary work expenses is improved.

Increased usage of PEMS by parliamentarians and their staff.

The target is represented by a ‘✔’.a

106

141

126

133

Note a: The corporate plan defines a tick as: ‘that an assessment approach will be used to measure performance for the given reporting period/s.’ The methodology detailed against the performance measures is ‘Measured through provision of additional functionality and increased number of claims processed through PEMS’.

Source: Finance’s Corporate Plan 2022–23.

2.64 The ANAO assessed the 2022–23 measure and target against section 16EA of the Public Governance Performance and Accountability Rule 2014 (PGPA Rule) as set out in in Table 2.9.

Table 2.9: Assessment of performance measure 2022–23

Performance measure and target

Relateda

Measurableb

Reliable and verifiable

Free from bias

Assessable over time

Measure: Improve administration of parliamentary work expenses — the timeliness, efficiency, clarity and transparency of the administration of parliamentary work expenses is improved

Target: Increased usage of PEMS by parliamentarians and their staff

144

80

110

91

104

Key:  Does not meet requirements.

Partially meets requirements.

Fully and/or mostly meets requirements.

Note a: Related refers to the requirement of subsection 16EA(a) of the PGPA Rule 2014, as amended. In applying the ‘related’ criterion, the ANAO assessed whether the entity’s performance measures:

related directly to one or more of the entity’s purposes or key activities; and

provided a clear link between purposes, key activities and performance measures.

Note b: In applying the ‘measurable’ criterion, the ANAO assessed whether the entity’s performance measures were:

reliable and verifiable — supported by clearly identified data sources and methodologies;

free from bias — provides an unbiased basis for the measurement and assessment of the entity’s performance; and

assessable over time — able to provide a basis for an assessment of performance over time.

Source: ANAO analysis based on the PGPA Act and PGPA Rule.

2.65 The measure, ‘improve administration of parliamentary work expenses — the timeliness, efficiency, clarity and transparency of the administration of parliamentary work expenses is improved’ and the associated target of ‘increased usage of PEMS’ target was determined to be related to a key activity, however, is not measurable as it is not reliable, verifiable, free from bias or assessable over time. This is based on the following factors.

  • The measure relates to Finance’s key activity to ‘deliver ministerial and parliamentary services’ under its purpose to provide services to and for government.
  • The measure and description are not reliable or verifiable with terms such as ‘timeliness’, ‘clarity’ and ‘transparency’ used without any definitions. It is not clear how these would be measured or verified nor how the target or underpinning methodologies correlate to these concepts.
  • In addition, the data source for this measure is a spreadsheet which requires a high level of manual intervention, impairing the reliability and validity through increased potential for errors.
  • The measure and target were not free from bias as the methodology for the target of ‘increased use’ was based on number of registered users and number of certified claims paid. This does not provide a complete view of PEMS performance and how it contributes to improved administration.
  • The measure and target are not assessable over time as the target and methodology does not allow for baseline or benchmarking for improved performance. The single target does not give a view of the effectiveness or efficiency of PEMS, rather it is a target based on registrations for a system that is mandatory (aside from paper-based forms) and for which users register for once.

2.66 For the 2022–23 results, the target was revised to ‘sustained use — 99 per cent of offices submitting and certifying claims using PEMS’. Despite the change to the target mid-way through the reporting cycle, the results continue to be based on user registrations and number of certified claims being paid. The discrepancies between the Corporate Plan and Annual Performance Statements are outlined in Table 2.10.

Table 2.10: Comparison between 2022–23 Corporate Plan and 2022–23 results

 

2022–23 Corporate Plan

Results reported in 2022–23 Annual Performance Statements

Target

Increased usage

Fully achieved

Sustained usagea — 99 per cent of parliamentarian offices have submitted and certified claims using PEMS

Measurement

  • Additional functionality
  • Increased numbers of claims processed
  • Continuing to build expenditure reporting
  • Registration for PEMS is stable and at least 90 per cent and number of claims/transactions processed successfully is stable and at least 95 per cent of all certified claims

113

133

172

Note a: In its 2022–23 Corporate Plan, Finance stated the target was ‘increased usage’. In its 2022–23 Annual Performance Statement it reported against a target of ‘sustained usage’.

Source: Finance’s 2022–23 Corporate Plan and Finance’s 2022–23 Annual Performance Statements.

2.67 Finance reported in its Annual Performance Statements that it fully achieved the target of ‘sustained usage’ in 2022–23 — at least 90 per cent of parliamentarians have registered to use PEMS, and of the 138,730 office claims/transactions lodged in PEMS 137,940 (99.4 per cent) have been paid.

2.68 The PEMS measure in Finance’s 2023–24 Corporate Plan has been updated to ‘Improved administration of parliamentary work expenses — sustained usage of PEMS by parliamentarians and their staff and successful processing of claims in PEMS’, removing reference to timeliness, efficiency, clarity and transparency from the measure description. There are three corresponding targets against the measure: sustained usage of PEMS by parliamentarians and their staff; successful processing of claims in PEMS; and availability of public reporting on parliamentarian work expenses. The new measure description and targets represent an improvement, although they could still be strengthened to better address the intent of the performance measure to improve administration of parliamentary work expenses.

Opportunity for improvement

2.69 There is an opportunity for Finance to review its PEMS performance measures and consider how to measure PEMS performance, for example time taken for end users to lodge claims, manual compared to online claims and customer satisfaction, to provide a more comprehensive view of PEMS capability and usability and how it contributes to improved administration of expenses.

Reporting to the Digital Transformation Agency

2.70 As part of its role to provide advice to the Australian Government on digital investments, the Digital Transformation Agency (DTA) collects information from agencies on the status of their digital and IT projects. Reporting provided to DTA by Finance detailed the PEMS project’s progress, budget, benefits, risks, and a rating of the delivery confidence. Finance’s self-reported ratings for delivery confidence were medium-high for the first year of the project in 2018 followed by a reduction to medium and eventually low by the October 2020 report. Ratings gradually rose again to high by June 2022. The September 2022 rating was medium, followed by two high ratings in January and May 2023. The July 2023 rating reduced to medium-high due to a delay in the schedule caused by further clarification on business and testing requirements.34 The DTA used these reports, along with its membership on the steering committee, to monitor the project’s progress and brief its executive and the Minister of Finance as minister responsible for digital transformation.

3. System deliverables and capabilities

Areas examined

This chapter examined whether the Parliamentary Expenses Management System (PEMS) meets the agreed deliverables and achieves outcomes including alignment with relevant legislative requirements.

Conclusion

The current PEMS capability does not meet all deliverables as agreed in the business case. The functionality delivered meets basic requirements to process office and travel expenses and payroll for parliamentary staff. There is a reliance on manual workarounds particularly for payroll services. Reporting functionality to meet legislative requirements was delayed. Parliamentarians and their staff were not sufficiently consulted throughout the project implementation to ensure the system was simple and easy to use.

3.1 PEMS was to be the key enabling IT system to support better administration, accountability, and transparency of work expenses for Parliamentarians and their staff. As parliamentary expenses and workplaces are covered by a legislative framework, it is critical that PEMS not only aligns with the high-level legislative requirements but also meets the needs of users to facilitate user adoption and makes the system usable and simple to support parliamentarians and their staff to operate within the rules.

PEMS capabilities

3.2 The ANAO assessed the current functionality as of December 2023 against the original PEMS capabilities as outlined in the business case and project brief. The assessment considered the high-level definition of the 13 PEMS capabilities35 and whether the functionality described has been released (see Table 3.1).

Table 3.1: Scope breakdown according to capability and delivery

Capability

Department of Finance’s (Finance) PEMS business case and project brief — capability descriptions

Capability delivered

C1 — Human Resource (HR) Management

HR management facilitates people management focussing on performance, policies and systems, including recruitment, training, performance appraisal and policy management.

Partial

C2 — Payroll Management

A core system to facilitate employee Member of Parliament (Staff) Act 1984 (MOP(S) Act and COMCAR driving staff) payroll and work expense payments.

Partial

C3 — Financial Management

A core system to facilitate financial management of accounts receivable and payable, payroll and the Ministerial and Parliamentary Services (MaPS) and Independent Parliamentary Expenses Authority (IPEA) general ledger.

Delivered

C4 — work expenses management and processing

This is considered the hub of transactions, certifications, and data integration. Work expense management can be developed in such a way that enables automation of more complex, interpretative actions. This could include policy automation through use of a business rules engine.

Partial

C5 — web portal & content management

Provides users with an online entry point into the system and enables information management. Ensures services and work expense policy resources are available to clients, the public and administrators.

Partial

C6 — customer relationship management (CRM)

Critical for client centric service delivery. The CRM will provide the central store for information relating to a client and is the basis by which the solution enables a single client record. Most enterprise grade CRMs offer a wide range of complementary functionality, for example, workflow management.

Partial

C7 — workflow management

Enables the agility of the ICT solution (its ability to respond to change) and promotes efficiencies within and across MaPS and IPEA service lines.

Partial

C8 — reporting and data visualisation

Effective presentation of information improves the quality and timeliness of decisions.

‘Access to all data held within the system for both ad-hoc (bespoke) and scheduled reporting. Audit and data analysis capability to support IPEA in meetings its functions under the IPEA Act.’a

Delivered

C9 — smart forms

Enables Clients and Administrators to conduct self-service functions online. Online data entry reduces errors and service requests, streamlines the processing of transactions and prevents re-work when paired with effective business process management systems.

Delivered

C10 — document & records management

Existing systems are digital but have limitations in functionality as the majority are stored in email. A dedicated document and records management system is required to ensure management of records throughout their life cycle, from the time they are created to their eventual disposal.

Partial

C11 — identity & access management

Underpins all systems to ensure that users have access to the appropriate information and functions for their role.

Delivered

C12 — system integration services

Provides linkages between all critical systems and enforces business rules for streamlined operation.

Delivered

C13 — data services

Provides the business with a unified view of information across a number of systems. Minimises the impacts of changes to individual systems over time.

Delivered

110

208

93

Key: Delivered = Capability delivered.

Partial = Scope was either reduced from original business case or full functionality was not (or is yet to be) delivered.

Note a: The description provided in quotation was provided only in the approved project brief and was not included in the business case.

Note: Capability C8, C12 and C13 were delivered when Milestone 8 was complete, with the Independent Parliamentary Expenses Authority (IPEA) publishing its first report in December 2023.

Source: ANAO analysis against PEMS business case and project brief.

Does the system allow for correct processing of expenses and payroll services?

For the functionality that has been delivered, PEMS correctly processes office expenses, travel claims and payroll, supported by manual processing. Controls around the claim process are largely appropriate, however there are a number of manual workarounds in relation to payroll services. There is a backlog of items that need to be implemented in order for the system to meet its scope as set out in the business case and user requirements.

3.3 The Parliamentary Business Resources Act 2017 states that the Commonwealth must pay for expenses as prescribed in the regulations relating to the conduct of a member’s parliamentary business. Section 66 of the Parliamentary Business Resources Regulations 2017 sets out what office expenses are prescribed and include items such as office supplies, printing and communications, and ICT and telecommunications. Claims must be within a set budget.36

3.4 The Parliamentary Business Resources Act 2017 also provides examples of expenses that can be paid for travel, such as cost of fares and travel allowance. This is supported by the Independent Parliamentary Expenses Authority Act 2017 that requires the Independent Parliamentary Expenses Authority (IPEA) to process claims (travel expenses) and incur expenses (travel allowance) for travel resources for parliamentarians and their staff. Parliamentarians and their staff can claim for travel.37

3.5 The principles outlined in the legislation are supported by regulations, determinations, supporting documentation and procedures.

Business process and system capability

Claiming office and travel expenses

3.6 Office and travel claims are either:

  • created and submitted directly into PEMS by parliamentarians or their staff;
  • submitted by parliamentarians or their staff via a form which is then manually entered into PEMS by Finance or IPEA staff; or
  • paid directly to a supplier under separate contractual arrangements between Finance or IPEA and a preferred supplier.38

3.7 Office and travel claims go through a two-step process of certification by the parliamentarian and verification by Finance or IPEA to process payment. If the parliamentarian submits the claim in PEMS, the parliamentarian certifies the claim in the system. If they submit a manual form, Finance or IPEA creates a new claim and completes the certification in the system based on certification contained in the manual form, with a second officer in Finance or IPEA verifying the claim. Table 3.2 and Table 3.3 summarise the high-level process for office and travel expenses.

Table 3.2: Office expenses claiming process

 

Process

PEMS system steps

1

Parliamentarian or staff arranges quotes, orders the item/service and confirms good/services have been delivered

Parliamentarian is accountable that expense meets obligations and within budget

Parliamentarian or staff orders/good and service off system

Office budget is calculated manually by Finance and entered into PEMS. Parliamentarians can view their office budget in PEMS. As claims are paid the budget automatically reduces in PEMS

2

Parliamentarian enters the claim, uploads required evidence and certifies

Claim details are entered into PEMS

Parliamentarian certifies in PEMSa

Evidence is uploaded into PEMS (evidence may vary depending on claim type)

PEMS checks input controls (for example, all mandatory fields are completed)

3

The claim is sent through to Finance staff to verify

Claim workflows through to Finance in PEMS

4

Finance staff review key claims details for example total invoice amount matches claims, required documentary evidence is supplied

Finance verifies in PEMS

If incorrect claim or part of the claim is incorrect Finance cannot edit the claim but returns to end user in PEMS

Basic system checks, for example, duplicate claim looking for same invoice date, invoice number, total amount and vendor

5

Finance can either pay the vendor directly for an item already received (account details set up in PEMS)b or if already paid reimburse to end user

Claim is processed and paid in PEMS

27

174

210

Note a: If it is a parliamentary staff member arranging the office expense they can create the claim but then submit it to the parliamentarian in PEMS for the parliamentarian to certify.

Note b: Finance has advised that as of 9 May 2023 there are 20,100 vendors set up in PEMS.

Source: ANAO summary of IT functionality observed during IT demonstrations.

Table 3.3: Travel expenses claiming process

 

Process

PEMS system steps

1

Parliamentarian or staff arranges/books travel

Parliamentarian is accountable that expense meets obligations

Off system

2

Parliamentarian or staff creates expenses claim or travel allowance claim, complete details and uploads evidence

Claim details are entered into PEMS

Evidence is uploaded into PEMSa

PEMS checks input controls (for example, all mandatory fields are completed)

3

Parliamentarian certifies travel claim

For parliamentary staff they submit the claim to the parliamentarian to certify (indicating direction and approval)

Claims certified in PEMS

4

Claim is work flowed to IPEA

Claim sent to IPEA in PEMS

5

IPEA reviews the claim and verifies the claim

IPEA staff review key claims details, for example, claim is not to/from work

IPEA verifies in PEMS

If incorrect claim or part of the claim is incorrect IPEA cannot edit the claim but returns to end user in PEMS

6

Claim is processed and paid

Claim paid through PEMS

27

183

210

Note a: For travel allowance claims evidence may not be available at time of entering claim (for example, if travel allowance is claimed prior to travel and the parliamentary staff has not paid for accommodation yet). In these cases, the staff select, ‘available on request’. IPEA then does assurance checks on a sample of these claims based on their assurance processes.

Source: ANAO analysis of IT functionality observed during IT demonstrations.

3.8 Overall, PEMS supports the key steps in claiming office and travel expenses, enabling the payment of office and travel claims submitted by parliamentarians and their staff. There are a range of controls in place to provide for accurate processing of office and travel expenses, noting the parliamentarian has ultimate accountability that the claim meets the parliamentary expenses framework when submitting the claim.

3.9 There is a backlog of undelivered functionality and system incidents/defects for expense management (office and travel) discussed further in paragraph 3.14 to 3.16. This means that the current functionality needs to be supported by manual workarounds by Finance and IPEA.

Payroll services

3.10 Parliamentarians employ staff within their offices under the Members of Parliament (Staff) Act 1984. The conditions of employment are primarily determined by the Commonwealth Members of Parliament Staff Enterprise Agreement. The business case for PEMS acknowledged the complexity of these employment arrangements as a cost driver for the PEMS solutions, for example concurrent employment across parliamentary offices is a practice that is seen in this operating environment that is not common in other organisations.

3.11 Finance supports parliamentarians in administering employment arrangements through a range of payroll and human resource services which PEMS was intended to support. This includes payroll services (including administering leaving and allowances), human resource policy and work, health and safety.

3.12 The payroll component of PEMS uses a SAP-solution that has not been customised to support the unique employment arrangements of parliamentary staff. This has resulted in manual workarounds for key components of payroll services, with examples outlined in Table 3.4.

Table 3.4: PEMS functionality to support payroll services

Payroll services area

PEMS capability

Details

On-boarding/off-boarding

  • This is a manual process, with parliamentarians emailing a form when a new staff member is on-boarded.
  • Finance employee enters data into back-end SAP and notifies supervisor to review data entered (notification is a manual process with no workflow to supervisor).
  • Off-boarding relies on manual processes (form submitted to Finance for Finance to process).
  • PEMS does not support automated access removal based on position when staff move between offices.

Pay processing

  • System capability for Finance staff to process payroll for parliamentary staff.

Leave

  • Some leave types (for example maternity leave) and more complex leave provisions (such as additional annual leave days) use manual forms submitted to Finance.
  • Manual process for calculating long service leave and applying for it.

Work, health and safety (WHS)

  • WHS functionality was originally in scope for PEMS. PEMS was intended to support tracking of work, health and safety roles, training and associated allowances. None of the features were delivered and are now managed through manual workarounds. The backlog includes WHS items and all were assigned with high priority.

108

91

234

Key:  No functionality delivered Partial delivery Functionality delivered through PEMS.

Note: For example, MOP(s) Act employees can access additional leave if they are required to travel from a rural or remote location to Caberra on a Sunday or Public Holiday.

Source: ANAO summary of IT demonstrations and walk-throughs.

3.13 Overall, the system capability delivered for payroll services meets basic requirements, i.e. system capability for pay to be processed and key leave types supported through self-service. However, payroll services are at risk of incorrect processing given the extensive reliance on manual workarounds, for example on-boarding/off-boarding process and manual calculation of more complex leave types such as long service leave. A key driver of the PEMS project was to replace previous systems that had high levels of manual interventions.

Backlog

3.14 During the project, a backlog of in-scope items that were not delivered prior to scheduled release or lower severity incidents/defects from earlier releases was created. There was discussion at governance forums throughout the project, including on the process for managing and prioritising the backlog. For example, in May 2021 the steering committee noted there were 471 items on the backlog. In response, a specific working group — the design authority — was established to prioritise the backlog and manage how it related to the essential items list for Milestone 7 phase 2.

3.15 From April 2023 the new PEMS Working Group (see paragraph 2.7) commenced a prioritisation process of the current backlog. As at August 2023, there were 348 open items in the following areas39:

  • travel expenses (domestic and international) — 172 items;
  • office expenses — 56 items;
  • reporting — 40 items;
  • human resources — 48 items; and
  • other — 32 items.

3.16 The PEMS Working Group assigned backlog items a priority rating from one to four, based on a group of drivers such as client impact, legislative impact and resourcing impact. Human resources, travel and reporting were assessed by Finance as the top priorities. Finance advised the ANAO on 18 December 2023, that ‘members of the 25 October Management Board agreed to prioritise the top 5 PEMS issues from each business group and put these forward to the SDO for remediation’ and that the steering committee noted these items in December 2023. The $4 million allocated in the 2023–24 Federal Budget for further development of PEMS will address these backlog items. Finance further advised that the remaining items on the backlog have been prioritised by business owners and will be developed into a release schedule for the next two to three years.

Internal assurance process

3.17 On 14 August 2023 Finance advised that in July 2023 it had approached the market to engage a system auditor for legal, assurance, fraud and risk advice on PEMS in July 2023. This is in response to a recommendation from the December 2022 Gateway Review that there was no evidence of an assurance plan or assurance activities that demonstrated PEMS conformed with legislative requirements. Finance further advised the ANAO on 18 December 2023, that the final report for this work is due mid-February 2024.

Testing of IT general controls

3.18 The ANAO undertook IT general controls40 assurance testing of PEMS in August 2023. No issues were found in terms of privileged user access or segregation of duties between processes, and system maintenance is managed through enterprise change control.

Is the system designed to facilitate transparent reporting on expenses?

The transparent reporting of expenses was delayed. PEMS expenditure reporting functionality was originally due in January 2020, with the expenditure reporting component released to IPEA in November 2023 and IPEA publishing its first reports in December 2023. Based on the design of the reporting component of PEMS, the system will support the reporting of expenses.

3.19 Under its legislation IPEA is required to prepare and publish regular reports on office and travel expenses, which it does on its website and data.gov.au.41 A core requirement of PEMS was functionality to facilitate this expenditure reporting, including moving to monthly public reporting. Since 1 July 2022 IPEA was unable to publish expenditure figures, until December 2023 where it published its first reports using PEMS expenditure reporting functionality for the period July–September 2022.42 IPEA has advised it uses the data from PEMS to conduct its audit functions off system.

3.20 The expenditure reporting functionality was initially due to be delivered in January 2020, with delays experienced throughout the project as described below.

  • In February 2019, the steering committee agreed that expenditure reporting would become part of the new Milestone 7 with an expected delivery date of April 2020.
  • In May 2020, the steering committee agreed to push the delivery date of Milestone 7 to July 2021 and in December 2021 the steering committee agreed that expenditure reporting would be part of Milestone 7 phase 2.
  • In April 2022, IPEA agreed that public expenditure reporting would be delivered after the July 2022 release43, with other operational reports for parliamentarians and IPEA still being delivered as part of the July 2022 release.
  • In August 2022, the steering committee confirmed the release as 31 October 2022.
  • In November 2022, the steering committee agreed to a revised timeframe — build to be completed by 30 June 2023 and in use by IPEA in September 2023. A formal Milestone 8 was also created in recognition of the work required.

3.21 Given this timing, the analysis of the expenditure reporting is based on ANAO’s observation of the design and build at the time of fieldwork (April to August 2023).

Reporting requirement and system capability

Expenditure reporting

3.22 To support the publishing of office and travel expenses data IPEA had the following high-level requirements for the expenditure reporting component of PEMS: a workflow to enable users to view, query and certify reports; the ability to extract the expenditure report in a format that can be published; and to be able to move to monthly reporting. These high-level requirements were supplemented by several lower-level requirements captured in supporting documentation that was agreed between IPEA and SDO and ICTD. There were also approved change requests to support changes to these requirements as needed.

3.23 If implemented according to design, there are capabilities in PEMS to support public reporting. Features of the process included:

  • the report generating the data from the other office and travel expense modules into the reporting module;
  • a workflow process that includes: IPEA creating the reporting period; parliamentarians certifying the report to be published and then IPEA extracting the approved data and publishing;
  • system capability for IPEA to adjust data deemed ‘not for publication’ which removes the data from the report44;
  • supporting manual options, for example a Parliamentarian signing a printout or PDF softcopy and uploading as evidence of certification;
  • office staff being delegated to review reports prior to parliamentarian certifying report; and
  • data controls.45

Was the system designed and implemented in a user centric manner?

User experience was planned to be considered in project establishment however it was not a focus throughout project implementation. Users of the system have reported that they are not satisfied with the usability of the system.

User considerations through project planning

3.24 PEMS end users were included in initial project planning documentation as outlined in Table 3.5.

Table 3.5: Summary of how end users were represented in planning documentation

Document

Examples of end user experience in documentation

Project brief

  • Project objective included outcomes for end users.
  • Project outcomes included end user outcomes.
  • In scope activities included organisational change management, communication management and stakeholder management.
  • Design principles and considerations included user experience such as the system being user centric, accessible, self-service, pre-population/validation, customised dashboards, customer relationship management capability.
  • Parliamentarians and their staff were listed as stakeholders, although no detail.
  • Benefits included those to parliamentarians and their staff, including client centric service delivery, improved administration, efficiency and effectiveness and increased transparency.
  • Risks included PEMS not meeting expectations and not being utilised.

Project Management Plan (initial March 2018)

  • Design principles of user centric, accessible, and self-service (as per project brief).
  • Benefits included those to parliamentarians and their staff, including client centric service delivery, improved administration, efficiency and effectiveness and increased transparency.
  • The establishment of the user reference group to provide collaboration during the requirements gathering phase is identified as a dependency.
  • Organisational change management is identified as a risk, with resistance to change impacting project scope and implementation. Robust change management strategy is listed as a proposed treatment.
  • Parliamentarians and their staff are identified as stakeholders in the engagement approach matrix with regular email updates, workshops and training material outlined as engagement activities. The user reference group is also listed, with engagement activities including prototypes and regular product presentation.

Project Management Plans (Milestone 7 and updated for phase 1 and phase 2

  • Organisational change management was listed as a work stream.
  • MaPS and IPEA were given responsibility of representing end users.

152

261

Source: PEMS project planning documentation.

Implementation

User reference group

3.25 Finance advised the ANAO in May 2023 that interactions with parliamentarians and their staff was primarily conducted through the PEMS user reference group. The user reference group was also listed as a mitigation to users not utilising PEMS in the initial project brief.

3.26 The user reference group was initially established in July 2018, with the Finance Minister issuing a circular to invite all current parliamentarians and their staff to participate in the group. The purpose of the group as outlined in its terms of reference were to, ‘participate in the design of the new system, making sure it is easy to use and meet their requirements’. The terms of reference indicate that there was originally 29 members (although this number and membership changed throughout the project).

3.27 Following its formation meetings were held on:

  • 29 November 2019 — re-endorsing the terms of reference and updates on the project, preview of PEMS potential front-end interface, future consultations and planned user acceptance testing;
  • 28 February 2020 — project update, PEMS previews (screen shots of potential design for human resources for example leave, payslips, work schedule);
  • 30 July 2020 — user reference group told of new ‘go-live’ date of July 2021 for human resource management and expense management and an update on communication and education;
  • 24 September 2020 — new format of an online drop-in session; and
  • 29 April 2021 — first meeting with new membership, which included discussion on project status and timing and training plan, in particular for the July 2021 release of Milestone 7 phase 1.

3.28 The meetings were primarily informational updates to the reference group members on project status, given the delays in functionality throughout the project.

User acceptance testing

3.29 The user reference group’s terms of reference outlined that group members will have the opportunity to view and comment on system design, training and support materials and participate in system demonstrations aligned to the delivery of milestones. Finance advised the ANAO that parliamentarians and office staff were involved in one round of user acceptance testing between 16 and 29 January 2020 for leave requests/approval, detailed views of electoral staff allocation and payslips. Another round was scheduled for December 2021 for further testing of human resources and office expenses, however Finance advised no user reference group members agreed to participate.

3.30 IPEA and MaPS staff undertook user acceptance testing as business owners and users of the system instead, with parliamentarians or their staff not involved in testing. The test environment delivered for Milestone 7 phase 2 was missing a key feature of system integration capability which impeded the ability to perform full end-to-end user acceptance testing.

Communications and training planning

3.31 The PEMS project brief included organisational change management, communication management and stakeholder management as in-scope for the project. The initial project management plan identified that a change management strategy would be established as well as stakeholder and communication activities. Updated versions of the project management plan for Milestone 7 included a separate work stream for organisational change management.

3.32 Organisational change management planning from the beginning of the project until Milestone 7 was limited. It included a PEMS ‘launch strategy’ document, change impact assessment and one change project status report. There is no evidence that any practical organisational change management activities occurred during this time. Planning for Milestone 7 phase 1 included a communication plan, a training plan and an internal training plan for MaPS staff.

3.33 Detailed planning occurred for Milestone 7 phase 2. For the Milestone 7 phase 2 release, Finance developed communication and training planning documentation including:

  • communication plan — covering communication objectives, audience, key messages, communication matrix, evaluation and risks;
  • training plan — strategy for overall education and training which includes approach, evaluation, goals, risks and stakeholders approved by project board and noted by the steering committee in March 2022; and
  • communication and training planner — operational document showing scheduled training and communications events around PEMS go-live.

3.34 Finance and IPEA have created educational and training products in different forms.

  • Factsheets and demonstration videos — IPEA and Finance have produced several factsheets on the basic topics of PEMS including the creation of claims (travel and office), accessing reports, changes and reminders.
  • Training sessions — Finance provided external training sessions to members of parliament and their staff surrounding the release of Milestone 7 phase 2, July 2022. There were twice daily sessions conducted on the basic functions of PEMS during July and August 2022. Finance did not provide evidence of how many individuals attended these training sessions. Finance positioned a team at Parliament House during sitting week for assistance on site. Individual training sessions were also provided if requested, with Finance conducting 62 appointments with 73 staff members.
  • Websites — Finance and IPEA developed websites that has help-guides, factsheets and information about the system.

Digital Services Standards

3.35 The Digital Transformation Agency (DTA) has a set of best-practice principles for designing and delivering services that are user-friendly, inclusive, adaptive and measurable. The standards include 13 criteria and the DTA website notes that applying the criteria may be an ongoing process.46 Finance confirmed PEMS was subject to the Digital Services Standards and compliance with the standards was part of the original high-level requirements that informed the business case. The commitment to the standards was also reflected in the project board and steering committee terms of reference.47

3.36 Between January 2018 and January 2019, Finance reported to the DTA (as part of regular reporting, see paragraph 2.70) that user interfaces from 2018 would comply with the standards. The July 2023 report stated that the digital transformation benefits of the system would be measured against compliance with the standards. However, there was no discrete assessment of PEMS against the Digital Services Standards throughout the project.

PEMS adoption rates

3.37 As described in paragraph 3.6 and 3.7 office and travel claims can either be directly entered into PEMS by parliamentarians and their staff or submitted using paper forms. Based on data provided by Finance for 1 July 2022 to 31 August 2023, Figure 3.1 shows the percentage of claims directly entered by the end user since the release of the current solution in July 2022.

Figure 3.1: Proportion of claims lodged using PEMS direct entry — 1 July 2022 to 31 August 2023

A line graph depicting the proportion of claims lodged using PEMS direct entry in terms of percentage from 1 July 2022 to 31 August 2023. It is broken into two lines, one showing direct entry for the domestic travel claims and one showing direct entry for office claims.

Note: Data does not include if Finance or IPEA has entered the claim into PEMS on behalf of the parliamentarian or their staff or when Finance or IPEA pays the supplier directly.

Source: ANAO analysis of Finance PEMS data.

End user views on PEMS

ANAO survey of users

3.38 In September 2023, the ANAO sent a survey to 2117 parliamentarians and their staff on their experience using PEMS. 472 parliamentarians and staff responded. Most respondents primarily use PEMS for travel.48 Key results from the survey are presented in Figure 3.2 to Figure 3.4.

Figure 3.2: End user overall satisfaction with PEMS

A column chart showing the end user over satisfaction with PEMS in percentage in the categories of very satisfied, satisfied, neither satisfied nor dissatisfied, dissatisfied and very dissatisfied.

Note: One per cent of survey respondents replied ‘don’t know’ to overall satisfaction with PEMS.

Source: ANAO survey of parliamentarians and staff.

Figure 3.3: Satisfaction of users with key components of PEMS

A clustered column chart depicting the satisfaction of users with key components of PEMS by percentage. The chart is in clusters according to the users’ response to statements. The statements are ‘PEMS travel claims are easy to complete’, ‘PEMS office management claims are easy to complete’, ‘PEMS helps to undertake human resource administration’, ‘PEMS helps to view my human resource record’ and ‘PEMS supports to review and manage my budget’. These clusters are then broken up by the percentage of users tha

Note: The results in Figure 3.3 do not include where the respondent answered ‘don’t know’, as the rate was minimal.

Source: ANAO survey of parliamentarians and staff.

Figure 3.4: End user views on PEMS usability

A clustered column chart shows the end user’s views on the usability of PEMS by percentage. The clusters of the chart are determined by the statements that the users responded to. The statements are ‘PEMS is easy to use’, ‘PEMS is easy to access’, ‘PEMS is easy to navigate’, ‘Processes in PEMS are simple and logical’, ‘I understand the progress and status of my claims’, ‘I can do everything I need to using PEMS’, ‘PEMS helpdesk is helpful’ and ‘PEMS meets expectations’. These are then divided by the respons

Source: ANAO survey of parliamentarians and staff.

IPEA customer satisfaction survey

3.39 IPEA conducts annual customer satisfaction surveys with parliamentarians and their staff covering their key service areas. The 2022 survey was conducted between 8 November and 12 December 2022, five months after the release of office and travel expense management. While the survey covers only the travel expenses component of PEMS, the results provide insight into user views on PEMS functionality as outlined in Table 3.6.

Table 3.6: Summary of PEMS satisfaction scores for travel expenses in IPEA customer satisfaction survey

 

2022 results (either satisfied or very satisfied)

%

2021 results (either satisfied or very satisfied)

%

Satisfactions with submitting travel claims in PEMS

31

78

PEMS was easy to find

54

78

PEMS was easy to access

50

76

PEMS travel claims were easy to submit

39

80

PEMS travel claims were easy to complete

34

73

157

141

141

Note: 132 users completed the 2022 survey. The research reports notes contextual factors that could have impacted the results including change of government and new users, the introduction of PEMS and the delay in expenditure reporting functionality.

98 users completed the survey in 2021. For the 2021 survey parliamentarians and their staff were using online forms for some travel claims with the remainder of claim types supported by manual forms and processing conducted by IPEA in the Entitlements Management System.

Source: IPEA, Customer Satisfaction Survey 2022 Research Report [Internet], IPEA available from https://www.ipea.gov.au/sites/default/files/2023-05/IPEA_Customer%20Satisfaction%20Survey%202022.pdf [accessed 3 September 2023].

Helpdesk calls

3.40 The PEMS helpdesk, run by Finance, received 3586 call in 2022–23, with 96 per cent categorised by Finance as ‘Severity 4 (Low)’.49 In the previous year, 2021–22 (prior to Milestone 7 phase 1 release for office and travel expenses), there were 1647 helpdesk calls, 97 per cent of which were also categorised as ‘Severity 4 (Low).’ In December 2022 Finance analysed the help desk calls it was receiving and identified the most common queries as: the claim approval process for parliamentarians and different workflows for different claim types; issues with the workflow for when claims are returned from MaPS to the parliamentarian’s office for updating; and users being able to find claims in different statuses.

Recent efforts to engage end users

3.41 Since October 2022, Finance has engaged with parliamentarians and their staff, for example:

  • re-establishing the user reference group — terms of reference were approved by the user reference group in May 2023 at the first meeting.50 This meeting also covered updates to the project and upcoming user research work. Meetings were held in June 2023 and in September 2023 to demonstrate expenditure reporting functionality. The user reference group were also involved in user acceptance testing for Milestone 8 in October 2023;
  • conducting user research (workshops and individual feedback session) to better understand PEMS usability issues. Key themes included opportunities to improve process flow, error messages, approach to requesting different access types, mobile experience, understanding where claims are up to and level of information in PEMS (with user having to maintain ‘shadow spreadsheets’). Finance advised it will add these issues to the backlog and consider as part of the prioritisation process; and
  • developing a revised approach to training, awareness and communications with the plan to use a mixture of methods such as face-to-face sessions and ‘train the trainer’ model, updates to web content and development of a new communications plan.

Appendices

Appendix 1 Entity responses

Page one of the response from Department of Finance. A summary of the response can be found in the summary and recommendations chapter.
Page one of the response from Department of Finance. A summary of the response can be found in the summary and recommendations chapter.
Page one of the response from IPEA. A summary of the response can be found in the summary and recommendations chapter.
Page one of the response from DTA. A summary of the response can be found in the summary and recommendations chapter.
Page one of the response from DTA. A summary of the response can be found in the summary and recommendations chapter.

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

  • strengthening governance arrangements;
  • introducing or revising policies, strategies, guidelines or administrative processes; and
  • initiating reviews or investigations.

4. In this context, the below actions were observed by the ANAO during the course of the audit. It is not clear whether these actions and/or the timing of these actions were planned in response to proposed or actual audit activity. The ANAO has not sought to obtain assurance over the source of these actions or whether they have been appropriately implemented.

  • Updates to governance arrangements (see paragraph 2.7).
  • A set of agreed user requirements and improved change management process for Milestone 8 expenditure reporting (see paragraph 3.22).
  • Finance conducted an informal assessment of PEMS in October 2023 on the aspects of the project that is considers have met the standards (see paragraph 3.36).
  • Efforts to engage parliamentarians and their staff on PEMS (see paragraph 3.41).

Appendix 3 Project delays by milestone

Project delays by milestone

Original milestone

Original delivery date

Expected functionality

Actual milestone

Actual delivery date

Actual functionality

1

June 2018

PEMS portal

  • Front facing PEMS portal for users to login.
  • Release of two online forms to allow Parliamentarians and their staff to submit two different claims in PEMS — no data validation or changes to backend functionality.

1

July 2018

PEMS portal

  • Front facing PEMS portal for users to login — no data validation or changes to backend functionality.

2

July to Nov 2018

Online forms

Release of eight additional online forms for Parliamentarians and their staff to submit eight more claim types — no data validation or changes to backend functionality.

2–5a

August to December 2018

Online forms

  • Release of eight online forms (instead of 10 in total) for Parliamentarians and their staff to enter claims. Two moved from Milestone 1.b
  • Supporting functionality such as ‘office manager’ (view claims for an office), ability to print forms, notifications and re-assignment of claims.

3

Feb 2019

PEMS financial related functions

  • Claims are paid using new PEMS finance system.

6

July 2019

PEMS financial related functions

  • Claims are paid using new PEMS finance system.
  • Additional development to support continued use of previous systems.c

4

Mar 2019

Business rule validation

  • Data is validated on entry for standard claims.

7 phase 2

July 2022

Business rule validation

  • Data validation for certain fields in the claiming process. Finance and IPEA staff verify claims are appropriate to process payment.

5

May 2019

Human resources

  • MaPS can process pay using the PEMS SAP human resources (HR) system.

7 phase 1

7 phase 2

July 2021

July 2022

Human resourcesd

MaPS staff can on-board staff, process payroll and terminate staff in backend of SAP HR, supported by manual processes.

6

July 2019

ERP operational

  • Parliamentarians and their staff use IPEA portal for leave, pay variations and enquiries.
  • Electronic certification of all claims forms submitted via PEMS.

7 phase 1

7 phase 2

July 2021

July 2022

ERP operationald

  • Parliamentarians and their staff use IPEA portal for certain leave types.
  • Office expenses and travel claims can be entered in PEMS and verified by IPEA.

7 e

Sept 2019

Single view profile

Parliamentarians and their staff have a single view of all details for all offices within their responsibility.

7 phase 1

7 phase 2

July 2021

July 2022

Single view profile

Parliamentarians and their staff have a single view of all details for all offices within their responsibility, including:

  • organisational information about their office;
  • budget consumption report for all expenses/claims in their office; and
  • list of all claims relevant to their office.

8

Jan 2020

Reporting

  • MaPS and IPEA, Parliamentarians and their staff can generate reports from PEMS.
  • Access to reporting for the public.

7 phase 1

8

July 2022

December 2023

Reporting

  • Reporting capability for ad-hoc reporting (IPEA) and budget reports are available (Parliamentarians and their staff).
  • Public expenditure reporting delivered in December 2023.

91

91

155

91

128

159

Note a: Milestones 3, 4 and 5 from the original project documentation were internal milestones that went towards the delivery of ERP operational. Their deliverables and scope were moved to milestone 6 (ERP Operational) and the mini releases for Milestone 2 were called Milestone 2–5 in future documentation.

Note b: The steering committee agreed in April 2018 to move two of the forms to the first release of Milestone 2 (August 2018).

Note c: Previous systems (Entitlements Management System and Chris 21) interfaces were updated to allow for continued use.

Note d: Scope and user requirements for these milestones were not fully delivered during the project. See paragraphs 3.8 to 3.16.

Note e: In February 2019, the steering committee agreed to split the scope of Milestone 6 so that the financial related functions were implemented in July 2019 and the remaining deliverables across expense management, including single view profile, and reporting were moved into Milestone 7. A new Milestone 8 was then created in November 2022 for expenditure reporting.

Source: ANAO analysis of scope and deliverables timeline.

Footnotes

1 The Special Minister of State, Senator the Hon Don Farrell requested an audit of the administration of PEMS on 7 November 2022. See Australian National Audit Office, Request for audit - Administration of the Parliamentary Expenses Management System [Internet], ANAO, Canberra, 2023, available from https://www.anao.gov.au/work/request/administration-the-parliamentary-expenses-management-system [accessed 30 October 2023].

3 Independent Parliamentary Expenses Authority, available from https://www.ipea.gov.au/travel/parliamentarian-travel/travel-within-australia [accessed 12 August 2023].

4 Department of Finance, available from https://maps.finance.gov.au/pay-and-employment/mops-act-employment [accessed 12 August 2023].

5 SAP is an enterprise resource planning software package meant to automate day-to-day processes of an entity’s core corporate business, such as human resources and finance and accounting.

6 Review of Parliamentary Entitlements Committee, An Independent Parliamentary Entitlements System Review, February 2016, available from https://www.finance.gov.au/publications/reviews/independent-parliamentary-entitlements-system-review [accessed 23 October 2023].

7 Finance advised the ANAO on 18 December 2023 that the PEMS steering committee noted in December 2023 the top priority items from the backlog that will be fixed by the end of 2023–24. Remaining items on the backlog will inform a two to three year strategy for improvements to PEMS, with funding provided in the 2023–24 Federal Budget for changes to PEMS in 2023–24 only.

8 In February 2021, Finance’s Executive Board agreed to move the project delivery from the SDO to the Information and Communication Technology (ICT) Division and establish a specific Branch in the ICT Division to support the project.

9 Figures provided to the ANAO by Finance on 4 October 2023 for 2022–23. It is based on claims and payroll transactions processed in PEMS. Adjustment can be made retrospectively to claims in past financial years and it may differ from other public reporting by Finance and IPEA for the same period.

10 Finance advised that the project board ceased in August 2022 because of increased involvement from the steering committee from this time. Between September 2022 and December 2022, the project board members were consulted out of session on the support arrangements for users following the July 2022 release.

11 Throughout the five-year project there were three individuals in the Senior Responsible Office role.

12 In this context, the ANAO has interpreted the decisions of the steering committee as decisions of the Senior Responsible Officer as chair of the steering committee.

13 The steering committee terms of reference note that once Milestone 8 (expenditure reporting) is complete the committee will transition to an executive management committee to manage the ongoing support and enhancements of PEMS.

14 The project management plan for Milestone 7 was finalised following approval from the project board by the Project Executive not the Project Sponsor. The updates for phase 1 and 2 were not finalised following approval by the project board and were provided to the ANAO in ‘draft’.

15 The funding was available through the Parliamentary Expenses Management System measure. The $38.1 million included: $30.2 million for capital for PEMS; $6.7 million for ongoing operating funding to support operating and maintenance costs of the system to 2020-21; $0.5 million for PEMS Gateway Reviews; and $0.6 million for project to support the COMCAR Automated Resource System (CARS project). This project was included in the PEMS second pass business case but managed as a separate project.

16 Outcome 1 in the IPEA Portfolio Budget Statement is defined as, ‘support for current and former parliamentarians and others as required by the Australian Government through the delivery of, independent oversight and advice on, work resources and travel resources.’

17 Department of Finance, PGPA glossary [Internet], Finance, available from https://www.finance.gov.au/about-us/glossary/pgpa [accessed 13 November 2023].

The glossary defined administered as ‘administered items are usually those managed by an entity on behalf of the Government. Entities do not have control over these items, which are normally related to activities governed by eligibility rules and conditions established by the legislation (for example, grants, subsidies and benefit payments).’ It defined departmental as ‘departmental items are usually appropriations managed by an entity, and over which the entity has control. That is, the entity’s accountable authority has discretion in delivering the activities and/or allocating resources.’

18 The Auditor-General Report No. 33 2020–21 Administration of Parliamentary Expenses by the Independent Parliamentary Expenses Authority previously noted that legal advice was not sought regarding the transfer of $5 million in June 2020.

19 Contractor expenditure includes the payment of contract staff engaged directly by Finance to fill specific non-SES positions. Consultant expenditure is for business consultant services. Outsourcing expenses include expenses incurred in providing IT facilities, software maintenance and operational related IT expenses. Employee expenditure is for Finance employees.

20 Milestone 1 delivered the client portal released on 1 July 2018. Milestone 2 was intended to be the progressive release of online forms between July and November 2018. Eight forms in total (not ten as per original business case) were delivered by December 2018, along with supporting functionality such as office manager (view claims for an office), ability to print forms, notifications and re-assignment of claims. Milestones 3, 4 and 5 from the original business case and project documentation were internal milestones that went towards the delivery of the ERP solution. The deliverables and scope were moved to Milestone 6 and the mini releases for Milestone 2 were re-named Milestones 2–5.

21 The Project Management Plan for Milestone 7 phase 2 included definitions to support the prioritisation process and development of ‘essential items’ list.

Essential — necessary for the system or business to perform the business process.

Important — Not essential. An acceptable work around can be developed for the short to medium term.

Desirable — Useful but not important or essential for go-live.

The plan also outlined that essential items will be included in the work plan, important items placed on the backlog with high priority and desirable will be placed on the backlog for post go-live prioritisation.

22 The steering committee meeting minutes noted that in practice ‘baselining’ meant the SDO would build to their understanding of requirements and design with gaps and defects addressed during user acceptance testing.

23 Further details were provided in updated versions following Milestone 7 to capture the specific steps in the change request process.

24 The ANAO has observed improvements in Finance’s approach to the change management process for Milestone 8 including an agreed set of requirements (see paragraph 3.22).

25 Project issues have been raised in the risk register throughout the project. Example issue descriptions include ‘July 2020 schedule slippage’, ‘The project does not have secured budget for the 2020/2021 financial year’ and ‘Significant user experience issues exist and negatively impact on end users.’

26 The Major Projects Committee is a subcommittee of the Executive Board. Its terms of reference state its role is to provide oversight and advice on the implementation of projects that present higher risk.

27 Finance provided the Secretary operational updates as part of weekly reporting. However, these updates were routine, for example dates of upcoming governance forums, rather than detailed briefings at the times the project was experiencing issues.

28 Department of Finance, Gateway Review Process [Internet], available from https://www.finance.gov.au/government/assurance-reviews-and-risk-assessment/gateway-reviews-process [accessed 16 August 2023].

29 The minutes from the August 2018 steering committee outline that in response to the recommendation Finance would work with IPEA to document a full set of user stories (see paragraph 2.27). The subsequent Gateway Review in May 2019 noted the work that was being done to agree on design documents and prioritise requirements. However, the review team also noted concerns that requirements were still not agreed.

30 In April 2023 the steering committee agreed to delay the May 2023 Gateway Review until further progress had been made on the December 2022 review recommendations. This was approved by the Assurance Review Team within Finance (that manages the overall Gateway Review process) and PEMS Senior Responsible Office in August 2023.

31 The first level of the enhanced notification process is triggered by one red or two sequential amber/red rating and requires the Secretary of Finance to write to the relevant accountable authority to escalate awareness of the project issues. The second level of enhanced notification is triggered by two sequential red ratings or three sequential amber ratings. The Secretary of Finance will write to the accountable authority of the non- corporate commonwealth entity and the entity is required to develop an action plan and inform the responsible Minister and the secretaries of Prime Minister and Cabinet and Finance. The third level is triggered by three sequential red ratings or four sequential amber or amber/red ratings. The Secretary of Finance will write to the accountable authority of the non-corporate commonwealth entity and the entity is required to undertake an independent review and inform the responsible minister and the secretaries of Prime Minister and Cabinet and Finance.

32 In the case of the PEMS project the Secretary of Finance was also the accountable authority. In this case the Deputy Secretary who is responsible for the Gateway Review process in Finance wrote to the Secretary of Finance.

33 Finance’s 2023–24 Corporate Plan includes PEMS with an updated performance measure description and targets. To allow for analysis of results and a full performance measurement cycle the 2022–23 Corporate Plan measures are the basis of the assessment in this report.

34 Low is defined as ‘successful delivery of the project required changes to budget, schedule, scope or benefits. There are major issues with project definition, schedule, budget, quality and/or benefits delivery, which don’t appear to be manageable or resolvable without changes being made’.

Medium is defined as ‘successful delivery of the project against budget, schedule, scope and benefits, appears feasible but significant issues already exist, requiring management attention. These appear resolvable at this stage and, if addressed promptly, should not present a cost/schedule overrun or loss/delay of benefits.’

Medium-high is defined as, ‘successful delivery of the project to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery’.

High is defined as ‘successful delivery of the project to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly.’

35 The business case outlined 15 ICT capabilities. The project brief and project management plan defined 13 capabilities were in scope for PEMS. The remaining two capabilities were out of scope for PEMS project (14-location intelligence and 15-fleet reservation and allocation) and to be separately provided by the COMCAR Automated Resource System (CARS) project. Capability 14 description included, ‘assisting with work expense compliance and certification to verify travel claims’. Finance has advised there is some functionality to support location intelligence in the travel expenses claiming process. Documents did not specify why the capability was initially de-scoped for PEMS.

36 The budget is calculated by Finance based on amounts set in the Parliamentary Business Resources Regulations 2017. Certain items such as work, health and safety equipment are not deducted from the budget.

37 Paragraph 12(j) provides for IPEA to process travel expenses for parliamentarians and their staff. Travel expenses claims include reimbursed travel related costs to parliamentarians and their staff for example flights, parking, taxi fares, hire cars, fuel for hire cars, rail, bus and ferry fares, tolls. Paragraph (k) and (l) provides for IPEA to incur expenses for parliamentarians and their staff. Travel allowance covers accommodation, meals and incidental expenses for overnight stays.

38 Finance has these arrangements in place for services such as office stationery and suppliers, property services, fleet management, mail folding and sorting equipment and copying services. IPEA has these arrangements in place with a preferred travel service provider for parliamentarians and their staff to book transport such as flights. If the parliamentarian or staff order through a preferred supplier they do not need to enter a claim as well. MaPS or IPEA will process and pay the supplier separately in PEMS.

39 Each ‘item’ is an identified instance of deficiency that requires IT change, for example undelivered user requirement, functionality issues (IT defect) or system enhancements identified by users.

40 IT general controls are controls over the entity’s IT processes that support the continued proper operation of the IT environment, including the integrity of information. The domains of IT general controls are: user access management, change management, and computer operations. Privileged access refers to processes that allow administrators to perform their duties such as adding new users.

41 Paragraphs 12(1) (e) and (f) of the Independent Parliamentary Expenses Authority Act 2017.

42 IPEA’s website states that subsequent reporting periods will be published on a rolling schedule through to mid-2024. Independent Parliamentary Expenses Authority, Release of the 1 July to 30 September 2022 expenditure reports [Internet], available at https://www.ipea.gov.au/news-articles/release-1-july-30-september-2022-expenditure-reports [accessed 20 December 2023].

43 This decision was made in the context that IPEA members agreed to retain quarterly reporting so the first report was not due until after 1 July 2022.

44 For example, parliamentarians with only staff travel costs and where the travel is prior to 1 April 2017 (when staff costs were first reported).

45 Examples of these data controls include: system enforced mandatory fields during the claim process and claim data automatically flowing from the front-end interface to the database to populate the reports.

46 The key criteria that relate to end users include: understand user needs; consistent and responsive design; make it accessible; measure performance; don’t forget the non-digital experience; and encourage everyone to use the digital service.

47 Aligning PEMS to the standards was a recommendation from the initial inquiry into parliamentary expenses that provided the business case for PEMS (recommendation 31).

48 44 per cent of respondents used PEMS primarily for travel, 38 per cent for office expenses, 14 per cent for payroll services and five per cent for budget management.

49 Priority levels are assigned to service requests/incidents as follows: Severity 1 (critical) = serious consequence for normal business for example PEMS unavailable for all users; Severity 2 (high) = normal business processes are seriously affected; Severity 3 (medium) = normal business processes are affected; and Severity 4 (Low) = problem has little or no effect on normal business processes for example PEMS service is available with a work around.

50 The new terms of reference outline its roles to discuss user experience, participate in user acceptance testing, review and provide feedback on education and training and discuss further enhancements. The group is to meet quarterly.