The audit objective was to assess the effectiveness of the Department of Health’s administration of the Medical Specialist Training Program.

Summary

Introduction

1. State and territory governments currently fund over 90 per cent of all medical specialist training in Australia through specialist trainee1 positions in public teaching hospitals. Over the past 15 years the Australian Government has also contributed to medical specialist training through a range of Commonwealth-funded programs administered by the Department of Health (Health).2

2. Australian Government programs were consolidated in 2009 as the medical Specialist Training Program (STP), in order to create a simpler and more flexible funding program. In March 2010 the then Government announced additional funding of $144.5 million over four years to increase the number of specialist training positions under the STP from around 360 to 900 by 2014.3 The funding increase—part of a wider National Health and Hospitals Network initiative—was intended to assist in addressing a forecast shortage of specialists in Australia by drawing on the private sector and other non-traditional avenues for training.4

Specialist Training Program

3. The Government intended that medical specialties with shortages were to be targeted through the expanded STP, including general surgery, pathology, radiology, dermatology, obstetrics and gynaecology. Priority was also given to providing training positions ‘where Australians need them, such as in rural and regional areas’.5

4. The objectives of the STP are to6:

  • increase the capacity of the health care sector to provide high quality, appropriate training opportunities to facilitate the required educational experiences for specialists in training;
  • supplement the available specialist workforce in outer metropolitan, rural and remote locations; and
  • develop specialist training arrangements beyond traditional inner metropolitan teaching hospitals.

5. The STP is an executive grants program7 involving annual competitive funding rounds. The Australian Government provides financial assistance to hospitals and other medical facilities or health organisations (known as ‘settings’) to employ specialist trainees. Depending on the circumstances, STP grants to settings can consist of a number of components, as follows:

  • the primary component is in the form of salary support to settings to assist with the cost of employing a specialist trainee in a specified training position. Salary support is set at $100 0008 per full-time equivalent (FTE) per year;
  • training positions outside metropolitan areas are eligible for an additional rural loading of $20 000 per FTE per year; and
  • training positions in a private sector setting are eligible for the Private Infrastructure and Clinical Supervision allowance, consisting of a supervision allowance of $30 000 per FTE per year9 and a training infrastructure allowance of $10 000 per FTE once every three years.

6. As a consequence, an STP grant will typically provide financial assistance of between $100 000 to $153 333 per FTE per year for a training position.

7. Under the STP, specialist trainees are placed in specified training positions within public or private hospitals, other medical facilities or health organisations that are accredited for the purpose of specialist training by the relevant specialist medical college (college). Specialist trainees will generally rotate through a number of training positions at different hospitals or facilities during their specialist training in order to gain a broad range of experience and skills. Completion of specialist training normally takes between three and six years depending on the speciality involved. Subject to meeting any other college requirements, specialist trainees are then eligible to apply for Fellowship10 of the relevant college and be recognised as a fully qualified specialist.

8. While Health has overall responsibility for STP administration, the department does not have a direct contractual relationship with the settings, which are the grant recipients. The administration of STP grant funds is managed through separate agreements between Health and the colleges.11 Under this ‘college administration’ model, all grant funding for STP training positions for a particular specialty is provided by Health to the relevant college.

9. Overall, STP expenditure from 1 July 2010 to 31 December 2014 has been $379 million. While the STP is an ongoing initiative, current funding agreements with colleges expire at the end of 2015. As at January 2015, the Government has not made a decision on future STP funding.

Previous audit coverage

10. The ANAO has not previously examined the STP. ANAO Performance Audit Report No.34 2010–11 General Practice Education and Training examined the management of general practice vocational education and training programs by General Practice Education and Training Limited (GPET), then a Commonwealth company.

Audit objective and criteria

11. The audit objective was to assess the effectiveness of the Department of Health’s (Health) administration of the STP. The audit focused on key aspects of Health’s administration of the STP since the consolidation of funding programs in 2009, and the achievement of key program targets and objectives. To assess the department’s grants administration, the ANAO focused on the fourth annual grant funding round (the 2014 round) which was completed in December 2013 and funded training positions from the beginning of 2014.

Overall conclusion

12. Australian Government programs providing support for medical specialist training were consolidated in 2009 as the Specialist Training Program (STP), and in March 2010 the then Government announced additional funding of $144.5 million over four years to increase the number of specialist training positions funded under the STP from around 360 to 900 by 2014. The funding increase was intended to help address a forecast shortage of specialists in Australia by tapping into the private sector and other non-traditional training settings. The STP is a grants program, with four annual competitive funding rounds conducted since its expansion in 2010. While the Department of Health (Health) has overall responsibility for STP administration, the department receives advice from state health services12 and specialist medical colleges as part of the grants assessment process, and disburses grants through the colleges.

13. Health has made substantial progress towards achieving the key STP targets and objectives, adopting a generally sound administrative approach which has improved over time. The STP training targets established in March 2010 have largely been met, with college reports indicating that some 93 per cent of training positions have been filled13, and some 89 per cent of funded training positions have been located in non-traditional settings. However, in the 2014 grant funding round Health adopted an internal review and rescoring procedure which was not documented, and the department did not strictly adhere to the published selection criteria; an approach which affected the transparency and to an extent the equity of the assessment process when viewed in terms of the application form and other explanatory material that informed applicants’ expectations about how grants would be selected.

14. The ANAO’s analysis of specialist medical college reports provided to Health indicates that, on a full-time equivalent (FTE) basis, around 833 training positions were filled as at 30 June 2014, representing some 93 per cent of the target of 900 training positions announced in March 2010. Further, college reporting indicates that the STP has been successful in utilising non-traditional settings to expand the number of specialist training opportunities. The most recent (July 2014) reporting indicates that 800 (89 per cent) of the training positions are in ‘expanded’ (non-traditional) settings, with 369 (41 per cent) in regional or rural areas.

15. The 2014 funding round, which was completed in December 2013 with a view to funding training positions from early 2014, increased the number of funded training positions from 750 to 900. The first stage of the assessment process for the 2014 round was soundly-based and benefited from third-party assessment of applications by state health services and colleges. However, Health decided not to fund some highly-rated applications recommended by the state health services and colleges as it sought to obtain a relatively even distribution of the new training positions against the population. In adopting this approach, which also featured in the previous (2013) funding round14, the department effectively applied a selection criterion that was not documented in the application form or other explanatory material made available to applicants. Although the program’s funding priorities, which underpinned the assessment criteria, were reviewed between rounds, Health did not take the opportunity to incorporate a reference to the approach adopted on population distribution in the 2014 round application form or explanatory material. Nevertheless, when considered in the context of the program’s intended outcomes (which include achieving a better geographical distribution of specialist services) the department’s approach in relation to this matter was not unreasonable.15

16. Further, the department advised the ANAO that it also adopted an internal review and rescoring procedure in the 2014 round16, to address differences in assessment scores from the state health services and colleges17; another departure from documented processes. Health’s approach was also inconsistent with information provided to applicants that the department would ‘collate’ assessment results received from state health services and the colleges, and advice to the Health Minister in March 2013 that the department’s selection of applicants was ‘essentially administrative’ as individual assessments were based on recommendations received from the state health services and colleges.

17. The ANAO has made one recommendation aimed at improving the transparency and equity of Health’s grants administration by: reviewing program guidelines and assessment criteria to incorporate lessons learned from funding rounds; and providing operational guidance to staff on moderation or other quality control processes to be applied to assessments by third-party advisers.

Key findings by chapter

Assessment and Selection of Applications (Chapter 2)

18. In the 2014 funding round, applications for STP grants were open to a broad range of organisations, consistent with the general program objective that specialist training occur beyond traditional teaching settings. The assessment process for the selection of grants was also outlined in explanatory material to the application form. Stakeholders informed the ANAO that the opening of the 2014 round was well publicised amongst potential applicants, and in the event, some 467 applications were received for the 150 available grants.

19. Health trialled the use of shared services for the 2014 funding round, to make the application process easier for applicants and to achieve administrative efficiencies. However, stakeholders advised the ANAO that while the electronic application process had improved overall administration compared to previous rounds, the trial suffered from insufficient testing prior to implementation. In particular, the trial encountered significant technical problems relating to the receipt and processing of applications, and the anticipated benefits and efficiencies were not fully realised.

20. The assessment criteria and selection processes outlined in the department’s assessment plan, application form and other publicly available explanatory material for the 2014 round reflected Australian Government priorities, which were informed by published research of Health Workforce Australia. Further, the involvement of specialist medical colleges and state health services in the assessment of applications strengthened the assessment process. In particular, the state health services and colleges provided third-party advice to Health on the educational merit of applications, the potential impact of applications on health services, and the extent to which applications met program funding priorities.

21. ANAO testing of Health’s final assessment of applications for the 2014 funding round indicated that the department adopted an internal review and rescoring process after receiving input from state health services and the colleges. Around 250 of the 467 applications received in the 2014 STP funding round were reviewed as part of this process, which was not documented. ANAO analysis indicates that the rescoring directly affected funding outcomes for 13 applications, representing some 2.8 per cent of all (467) 2014 round applications. For 12 of these applications, the final score assigned by Health was above that which was calculated by the ANAO based on input from the specialist medical colleges and state health services, indicating these applications likely benefited from the review and rescoring process to the extent that they were offered grants. As there were only 150 grants available through the 2014 round, the elevation of the 12 applications meant that some applicants that may have otherwise been offered a grant were not. One application was scored down by the department and as a consequence of this, was not offered a grant.

22. Further, the department decided not to fund some highly-rated applications received as part of the 2014 funding round. Specifically, 13 applications were placed on a ‘reserve’ list18, as the relevant setting had submitted applications for two or more training positions in the same specialty.19 In these cases, Health funded only one place in order to obtain ‘a relatively even distribution of the new training positions against the population data’. In adopting this approach, Health effectively applied a selection criterion that was not documented in the application form or other explanatory material made available to applicants for the 2014 funding round. Health’s approach was inconsistent with the instructions provided to applicants for completing the program application form, which indicated that the department would ‘collate’ assessment results received from state health services and the colleges. Further, the approach adopted was not consistent with advice provided to the Health Minister in March 2013 that the department’s selection of applicants to be funded under the STP was ‘essentially administrative’, and that individual assessments were based on recommendations received from the state health services and colleges.

23. The lack of appropriate record-keeping and quality control in the internal review and rescoring process, and the use of a selection criterion that was not contained in the 2014 round application form or other explanatory material available to applicants, affected the transparency and to an extent the equity of the assessment process when viewed in terms of the application form and other explanatory material that informed applicants’ expectations about how grants would be selected.20 However, as previously noted, the department’s approach in using a selection criterion that incorporated population distribution considerations was not unreasonable in the context of the program’s intended outcomes.

Administration of Funding Agreements (Chapter 3)

24. A feature of the STP is that while Health has overall responsibility for its administration, the department does not have a direct contractual relationship with individual settings, which are the grant recipients. Rather, the administration of STP grant funds is managed through separate agreements between Health and the respective colleges. Under this ‘college administration’ model, developed by the department in 2010, all grant funding for STP training positions within a particular specialty is provided by Health to the relevant college.

25. The colleges and settings interviewed by the ANAO indicated that the ‘college administration’ model generally worked well. However, some potential risks were not fully assessed by Health when developing the model, including the risk that the Australian Government’s financial framework requirements might apply to the colleges if they handled public money.21 It is prudent for government entities to consider, at the design stage, the full implications of complex financial and administrative arrangements—such as those involving third-party administration of government programs—so as to avoid potential compliance and reputational risks.22

26. The department relies on reports received from the colleges to inform its oversight of the college administration model.23 Since 2012 Health has sought additional management and performance information from the colleges, particularly relating to financial issues. However, the presentation of financial information by colleges—particularly income and expenditure—has varied significantly, sometimes making it difficult for Health to assess how funds have been spent. Where variations in reporting have occurred, Health has undertaken follow-up communication with colleges to determine their actual financial position.

27. As at 31 December 2013, total surpluses of STP funds held by the colleges were $36.28 million.24 By 30 June 2014, total surpluses had risen to $56.31 million. The surpluses can be partly attributed to timing issues, including delays in the submission of invoices by training settings to the colleges. During 2014, Health responded more actively where surpluses were identified, by withholding a proportion of scheduled progress payments. In consequence, $23.89 million that was due to be paid following receipt of the July 2014 college reports was withheld.

Program Performance and Evaluation (Chapter 4)

28. The STP has had key performance indicators (KPIs) in place since the consolidation of the program in 2009. However, explicit outcome-linked KPIs were only developed in 2013, and the colleges have reported against these KPIs since January 2014.

29. College reporting against the program’s KPIs indicates that the STP has been successful in utilising non-traditional settings to expand the number of training positions for specialist trainees, with 89 percent of STP-funded positions being located in non-traditional settings. In discussions with the ANAO, stakeholders also suggested that the expanded range of work environments has contributed to the overall quality of training.

30. ANAO analysis of college reporting indicates that on a FTE basis, around 833 training positions were filled as at 30 June 2014, representing 93 per cent of the target of 900 positions. Overall, the additional specialist trainee positions funded by the STP have boosted the availability of specialist services, including in regional and rural areas. However, it remains unclear to what extent the STP has, or will, contribute to an improved geographical distribution of specialist services to meet community need, over the longer term.

Summary of entity response

31. The Department of Health agrees with the audit recommendation. The findings of the audit will be of value in the future administration of the Medical Specialist Training Programme.

32. Health’s full response is provided at Appendix 1.

Recommendation

Recommendation No. 1

Paragraph 2.45

To improve transparency and equity in the administration of grants, the ANAO recommends that the Department of Health:

  • review program guidelines and assessment criteria at the conclusion of grant funding rounds, to incorporate lessons learned; and
  • provide operational guidance to staff on moderation or other quality control processes to be applied where applications have been assessed by third-party advisers.

Department of Health response: Agreed.

1. Introduction

This chapter provides the background and context for the audit including an overview of the Medical Specialist Training Program. The audit objective, criteria, scope and methodology are also outlined.

Training medical specialists in Australia

1.1 In Australia, medical graduates must undergo a 12 month period of additional training before becoming fully qualified as medical practitioners (doctors) and receiving a provider number to enable billing under Medicare.25 Most commonly, this training involves an internship at a public hospital, followed by a further 12 months as a resident medical officer. After this, doctors may undertake further training, either in general practice or in a particular medical specialty.26 In the latter case, a doctor will enter a training program under the auspices of the relevant specialist medical college (college).27

1.2 Under these college programs, specialist trainees (also known as registrars) are placed at specified training positions28 within public or private hospitals, other medical facilities or health organisations that are accredited for the purpose of the specialist training programs by the relevant college.29 Specialist trainees will generally rotate through a number of training positions at different hospitals or facilities (which are called ‘settings’) during their specialist training in order to gain a broad range of experience and skills.

1.3 On completion of specialist training—which may take between three and six years30—and subject to meeting any other requirements of the relevant college, specialist trainees are eligible to apply for Fellowship31 of the college and be recognised as a fully qualified specialist. Whilst undergoing training, specialist trainees may choose to focus on a particular sub-speciality or discipline.32 Alternatively, they may train across a broader spectrum: in some cases this may lead to specialist trainees qualifying within a ‘generalist’ specialist stream.33 An illustrative medical education and training pathway for a specialist is shown at Figure 1.1.

Figure 1.1: Medical specialist education pathway in Australia

Source: Health Workforce 2025—Volume 3 Medical Specialties, p. 406.

1.4 There has been substantial growth in the overall number of fully qualified specialists in recent years. Excluding general practice specialists, the number of actively-practising college Fellows has risen from 26 946 in 2008 to 32 702 in 2012, an increase of 21.4 per cent.34,35 Excluding general practice, the number of specialist trainees undergoing specialist training increased from 10 649 in 2009 to 13 801 in 2013, an increase of 29.6 per cent.36 In comparison, Australia’s population increased by 7.7 per cent between 2008 and 2013.37

The Specialist Training Program

Background

1.5 State and territory governments currently fund over 90 per cent of all medical specialist training in Australia through specialist trainee positions in public teaching hospitals. Over the last 15 years the Australian Government has also contributed medical specialist training through a range of programs administered by the Department of Health (Health). These programs have funded a broad range of activities, including those with a specific geographic focus38, directed at a particular specialty39, or dealing with doctors trained outside Australia.40 In 2009, programs funded by the Australian Government were consolidated into the medical Specialist Training Program (STP) in order to create a ‘simpler, more flexible funding program’.41

1.6 In March 2010, the then Australian Government announced a range of significant health policy and funding measures under the National Health and Hospitals Network initiative. A component of the initiative was additional funding of $144.5 million over four years to increase the number of specialist training positions under the STP from around 360 to 900 by 2014.42

1.7 The 2010 Australian Government announcement noted that work by the Australian Medical Workforce Advisory Committee43 and the colleges suggested there would be a shortage of 1280 specialists in Australia by 2020. The expansion of the STP aimed to deliver 680 additional specialists by 2020 by drawing on the private sector and other non-traditional avenues for training. The remainder of the projected shortfall was to be addressed by state health services44 increasing the number of training positions in their public teaching hospitals. Specialties where shortages existed were to be targeted through the expanded STP, including general surgery, pathology, radiology, dermatology, obstetrics and gynaecology. Priority was also given to providing training positions ‘where Australians need them, such as in rural and regional areas’.45

1.8 The objectives of the STP are to46:

  • increase the capacity of the health care sector to provide high quality, appropriate training opportunities to facilitate the required educational experiences for specialists in training;
  • supplement the available specialist workforce in outer metropolitan, rural and remote locations; and
  • develop specialist training arrangements beyond traditional inner metropolitan teaching hospitals.

1.9 The objectives outlined above were to be ‘achieved without an associated loss to the capacity of the public health care system to deliver services’.4748

1.10 Associated with the STP objectives are nine ‘expected outcomes’49:

  • rotation of specialist trainees through an integrated range of settings beyond traditional inner metropolitan teaching hospitals, including a range of public settings (including regional, rural and ambulatory settings), the private sector (hospitals and rooms), community settings and non-clinical environments;
  • increased number and better distribution of specialist services;
  • increased capacity within the sector to train specialists;
  • improved quality of specialist training with trainees gaining appropriate skills not otherwise available through traditional settings;
  • development of system wide education and infrastructure support projects to enhance training opportunities for eligible trainees;
  • improved access to appropriate training for overseas trained specialists seeking Fellowship with a college;
  • increased flexibility within the specialist workforce;
  • development of specialist training initiatives that complement those currently provided by state health services; and
  • establishment of processes which enable effective and efficient administration of specialist training positions, with reduced complexity for both stakeholders and the administering department.

1.11 The STP is administered by the Australian Government through the Department of Health (Health).50 Formal stakeholder input into the operation of the STP is through two main sources: the Medical Training Review Panel51 and the STP inter-college forum.52

How the program is delivered

1.12 The STP is an executive grants program53 involving annual competitive funding rounds.

Funding

1.13 The grants are provided to hospitals and other medical facilities or health organisations to employ specialist trainees. Depending on the circumstances, STP grants to settings can consist of a number of components, as follows:

  • The primary element is in the form of salary support to settings to financially assist them to employ a specialist trainee at a specified training position. Salary support is set at $100 000 (ex GST)54 per full-time equivalent (FTE) per year.55
  • Training positions outside metropolitan areas are eligible for an additional rural loading of $20 000 per FTE per year.56 ANAO discussions with settings indicate this ‘rural loading’ is used to subsidise a range of expenses, including covering transport costs or, in some cases, providing specialist trainees with accommodation and/or a vehicle.
  • Where the position is in a private sector setting, it is also eligible for the Private Infrastructure and Clinical Supervision allowance. This consists of a clinical supervision allowance of $30 000 per FTE per position, per year—which is intended to recognise the time and effort involved in a specialist supervising a specialist trainee whilst they are undertaking training—and a training infrastructure allowance of $10 000 per FTE once every three years.57 Consistent with the intent of the STP to utilise non-traditional avenues for specialist training, the Private Infrastructure and Clinical Supervision provides an incentive for the private sector to participate in the program.

1.14 As a consequence, an STP grant will typically provide financial assistance of between $100 000 to $153 333 per FTE per year for a training position.

1.15 In order to avoid cost-shifting58, the STP only funds ‘new’ training positions. Training positions that have been funded from any source for more than 12 months out of the last three years are ineligible for STP grants.59,60 Applications must also be accompanied by a letter of support from the local hospital network as well as the relevant college.

Administration

1.16 A feature of the STP is that while Health has overall responsibility for its administration, the department does not have a direct contractual relationship with the settings, which are the grant recipients. Rather, the administration of STP grant funds is managed through separate agreements between Health and the colleges. Under this ‘college administration’ model, all grant funding for STP training positions within a particular specialty is provided by Health to the relevant college. Under their respective funding agreements with Health, the colleges have responsibility for:

  • disbursing the STP grant funds to the relevant settings through periodic payments;
  • oversight of the conduct of the funded training positions through contractual and liaison arrangements with the settings;
  • the development and implementation of strategic training and education projects to support the network of STP training positions;
  • promoting the integration of training provided through STP training positions with that provided by state health services; and
  • providing progress and financial reporting to the department.

1.17 Colleges receive funding from Health to undertake the administrative functions referred to in paragraph 1.16.61 The amount of administrative funding provided by Health varies between specialist medical colleges when measured on a ‘per position’ basis. However, for the majority of specialist medical colleges the amount of administrative funding is approximately $5000 to $10 000 per position, per year.62

Progress in implementing the Specialist Training Program

1.18 Since the expansion of the STP announced in 201063, four annual STP competitive funding rounds have been conducted by Health.64 The 2014 round65, which was completed in December 2013, increased the number of training positions funded under the program from 750 to 900. The progressive expansion in the number of STP training positions through the four funding rounds is shown in Table 1.1. As at December 2014, the Australian Government has not announced whether any further funding rounds will be undertaken, or whether supported training positions will have funding extended beyond the current contractual commitments that cease at the end of December 2015.

Table 1.1: Expansion of STP training positions 2011–2014

Funding Round

2011

2012

2013

2014

New training positions funded

158

82

150

150

Cumulative training positions funded

518

600

750

900

Source: Department of Health Annual Reports.

1.19 While the STP has achieved the target of 900 training positions, some funded settings have subsequently withdrawn from the program or have had periodic difficulties in attracting or retaining specialist trainees in the training positions. In such cases, colleges have utilised so-called ‘reserve lists’ of highly ranked unsuccessful applicants from the previous funding round to boost the number of occupied training positions. While calculating the exact number of occupied training positions is difficult due to some variations in the relevant college reports and/or late reporting by settings, ANAO analysis indicates that around 833 of the 900 training positions66 (93 per cent) were occupied during the January to June 2014 reporting period. Several colleges with higher vacancy rates67, or which experienced underspends of previously provided STP funds68, indicated in their July 2014 progress reports that they will draw on reserve lists from the 2014 funding round to increase the number of occupied training positions in the second half of 2014 and in 2015.

1.20 ANAO analysis of Health’s financial records indicates that over the four years from 2010–11 to 2013–14, some $336.53 million has been expended on the STP, as shown in Table 1.2. In the first six months of 2014–15, expenditure has slowed over that of the previous year, with $42.46 million spent to 31 December 2014. This reflects the department’s decision to withhold a proportion of scheduled progress payments in response to the build–up of significant surpluses of STP funds held by the colleges.69

Table 1.2: STP expenditure 2010–11 to 2014–15

2010–11

2011–12

2012–13

2013–14

2014–15 (to 31 December 2014)

$58.06 million

$71.34 million

$96.25 million

$110.88 million

$42.46 million

Source: Department of Health data.

Notes: The above figures do not include the amounts for two other initiatives, Emergency Department Workforce (Doctors and Nurses) and Specialist Training in the Tasmanian public health system, which are largely delivered under STP agreements with the colleges. These two initiatives provide funds, including to support specialist training positions, of up to $123.83 million from 2010–11 to 2015–16. Further details are at paragraphs 1.23–1.28.

Distribution of Specialist Training Program training positions

1.21 The national distribution of all funded STP training positions is shown in Figure 1.2. The distribution broadly corresponds to population distribution, with the exception of the Northern Territory, which has 4.5 per cent of STP training positions but only one per cent of Australia’s population.70

Figure 1.2: Funded STP training positions 2010–14 by State and Territory

Source: ANAO analysis of Department of Health data.

1.22 The distribution of STP training positions by medical speciality is shown in Figure 1.3. Some 40.9 per cent of training positions are in specialties that fall within the auspices of the Royal Australasian College of Physicians. These cover a broad range of specialties, including public health medicine, paediatrics, geriatric medicine, medical oncology and cardiology. The significant proportion of psychiatry training positions (18.2 per cent) is consistent with the increased emphasis on mental health in Australian health policy in recent years.

Figure 1.3: Funded STP training positions 2010–14 by medical specialty

Source: ANAO analysis of Department of Health data.

Note: Percentages do not equal 100 per cent due to rounding up and down of original data. Not shown are the two training positions funded through the STP that are administered by the Australasian College for Emergency Medicine. These represent 0.2 per cent of all STP training positions.

Delivery of other activities through the Specialist Training Program

Emergency Department Workforce (Doctors and Nurses)

1.23 In July 2010, the then Australian Government announced the $96 million Emergency Department Workforce (Doctors and Nurses) measure. This was intended to increase the capacity of the healthcare sector to train emergency department specialists, nurses and support staff, as well as training general practitioners in emergency medicine.

1.24 A significant component of this measure has been delivered through the STP. Grant applications for new training positions for specialist trainees specialising in emergency medicine have been considered through the annual STP funding rounds.71 Each year, 22 additional training positions have been funded through this process—these are additional to the 900 training positions targeted by the STP process. During the January to June 2014 reporting period, 87 training positions were filled72, an outcome very close to the target of 90 training positions specified in the relevant funding agreement.73 The grants are administered through a funding agreement between Health and the Australasian College for Emergency Medicine (ACEM) using the STP model, although ACEM’s reporting obligations are slightly different to the other colleges.74

Specialist Training in the Tasmanian public health system

1.25 In June 2012, the Australian Government announced a $325 million Tasmanian ‘health assistance’ package. One element of the package was the $39.6 million Training more Specialist Doctors in Tasmania initiative (the ‘Tasmanian project’), which provided targeted grants to specifically address the public health needs of Tasmania and the medical specialist workforce in Tasmania’s public hospitals.75 Tasmania’s difficulty in recruiting and retaining its health workforce, including specialist trainees and specialists, was a key issue in the 2004 review of the Tasmanian health system by an expert advisory group.76 The provision of specialist services also featured in the 2012 preliminary report of The Commission on Delivery of Health Services in Tasmania.77

1.26 Following discussions between Health, the Tasmanian Department of Health and Human Services, Tasmanian regional health organisations and colleges, a detailed implementation plan for the Tasmanian project was approved by the Australian Health Minister in June 2013. The project involves Health providing grants to fund an initial 38 FTE specialist trainees and 10.7 FTE specialist supervisor positions in specified settings in the Tasmanian public sector health system in 2013–14, rising to 51 FTE and 14.5 FTE positions in 2015–16.78 The Tasmanian project explicitly funds the full cost of the relevant positions.79 On a FTE basis, this equates to around $120 000–180 000 per year for a specialist trainee, depending on their speciality and seniority, and an average of $365 000 for a specialist supervisor.

1.27 STP funding agreements were amended in 2013 to provide for delivery and administration of Tasmanian project funding through the agreements. As is the case for STP funds, Tasmanian project funds are administered by the relevant college, but there are different reporting requirements and payment schedules applying to the Tasmanian project funds. The colleges do not receive further administration funding for the Tasmanian project over and above the STP funding that they receive from Health.

1.28 As at June 2014, progress in filling funded Tasmanian positions has been relatively slow. In part, this was due to a previously unrecognised legal issue concerning medical indemnity, with resolution of this matter also delayed by a change of government in Tasmania in March 2014. During the January to June 2014 reporting period, 31.35 FTE specialist trainees and 7.46 FTE supervisors were in place, around 80 per cent of the target for 2013–14.

Grants administration framework

1.29 The STP is subject to the Australian Government’s framework for grants administration, which has operated since 2009. Commonwealth Grant Guidelines were introduced under the Financial Management and Accountability Act 1997 (FMA Act) and Financial Management and Accountability Regulations (FMA Regulations) in July 2009 and updated in June 2013. Since July 2014, the Commonwealth Grants Rules and Guidelines have operated pursuant to the Public Governance, Performance and Accountability Act 2013 (PGPA Act) and related Rules. Since 2009, the Commonwealth Grant Guidelines and subsequently the Commonwealth Grants Rules and Guidelines have outlined a generally consistent legislative, policy and reporting framework for administering grants, and seven key principles for better grants administration. For the purposes of this audit, clear reference is made to which set of guidelines and/or rules (2009, 2013 or 2014) are relevant to the discussion of Health’s administration of the STP at any one time.

Health grants policy

1.30 In 2011, following the completion of a review of administrative arrangements in the Health and Ageing portfolio, the Australian Government decided to implement a range of changes within the portfolio, including consolidating 159 programs (many of them grant programs) into 18 internal ‘flexible funds’.80 The flexible funds concept was intended to ‘reduce red tape, provide increased flexibility to respond to emerging issues and deliver better value with public money’81 The STP falls within one of these flexible funds, the Health Workforce Fund. The primary objective of the Health Workforce Fund is to strengthen the capacity of the health workforce to deliver high quality care.82 Policy responsibility for the STP, as for other Health Workforce Fund matters, lies with the department’s Health Workforce Division. However, administration of the STP funding agreements, including the assessment of college reporting and the authorisation of progress payments, is undertaken by the department’s Grant Services Division.83

Previous reviews of the Specialist Training Program and specialist training

1.31 The ANAO has not previously examined the STP. ANAO Performance Audit Report No.34 2010–11 General Practice Education and Training examined the management of general practice vocational education and training programs by General Practice Education and Training Limited (GPET), then a Commonwealth company.

1.32 A review of Health’s administration of the funding agreement with one of the colleges in receipt of STP grant funding was carried out by the department’s internal audit unit in 2012. The review indicated that ‘substantial improvement’ was required in key aspects of the department’s administration. Key findings related to unspent grant funds, a substantial shortfall in the number of training positions actually filled, and inconsistencies in financial reporting.

1.33 In 2012, the then Australian Government commissioned a review (the Mason Review) to assess the appropriateness, effectiveness and efficiency of the health workforce programs and activities, and their alignment with Australia’s workforce priorities.84 The review’s April 2013 report observed that:

The STP has been highly successful in extending vocational training into new settings, particularly in the rural and private sectors. It has also demonstrated that specialist colleges can take a flexible approach to accrediting new positions and to supporting networked training arrangements involving multiple health care settings, sometimes in different regions.85

1.34 However, the review also made some observations about career pathways for medical graduates wishing to specialise, and recommended a full review of the STP to inform its future direction and consider whether existing training positions were meeting the program’s objectives.

1.35 Observations about the overall coordination of training pathways for specialists were also made by Health Workforce Australia in its 2012 report Health Workforce 2025:

While better organised and targeted at a national level, the Commonwealth [specialist and GP training programs] lack the requisite level of coordination and alignment with state and territory approaches to best leverage the collective tax payer funded training investments of both levels of government. Rectifying these shortcomings is essential to achieving the long-term workforce outcomes required.86

1.36 The coordination of medical training is being considered through the National Medical Training Advisory Network, formed in 2012 under Health Workforce Australia.87 Health advised that the National Medical Training Advisory Network is developing a series of rolling medical training plans, including in relation to specialists, to inform the Australian Government, health and education sectors.88 The department further advised that the first specialist training plan (for psychiatrists) is expected to be provided to Health Ministers for their consideration around February or March 2015.

Audit objective, criteria and methodology

1.37 The audit objective was to assess the effectiveness of the Department of Health’s (Health) administration of the Specialist Training Program (STP). The audit focused on key aspects of Health’s administration of the STP since the consolidation of funding programs in 2009, and the achievement of key program targets and objectives. To assess the department’s grants administration, the ANAO focused on the fourth annual grant funding round (the 2014 round) which was completed in December 2013 and funded training positions from the beginning of 2014.

1.38 The audit methodology involved:

  • conducting over 30 interviews with primary stakeholders of the STP, including specialist medical colleges, state health services, the Australian Medical Association, and settings that received STP grants;
  • undertaking a detailed review of key aspects of the application and assessment process for the 2014 funding round, including how Health handled the assessment results received from the colleges and jurisdictions;
  • reviewing the process through which existing funding agreements were varied to take account of the outcomes of the 2014 funding round; and
  • examining college reporting for 2013–14, Health’s analysis of that reporting and the making of progress payments.

1.39 The audit reviewed Health’s administration of the STP against the Australian Government’s resource management and grants frameworks and the ANAO’s grants administration better practice guide.89

1.40 The audit was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $450 140.

Report structure

1.41 The structure of the audit report is outlined in Table 1.3.

Table 1.3: Structure of the audit report

Chapter 2 – Assessment and Selection of Applications

Examines the conduct of the 2014 Specialist Training Program funding round, including the development of program guidelines and funding priorities, the application, assessment and selection process.

Chapter 3 – Administration of Funding Agreements

Examines Health’s administration of the Specialist Training Program funding agreements with the specialist medical colleges.

Chapter 4 – Program Performance and Evaluation

Examines performance monitoring and evaluation for the Specialist Training Program.

2. Assessment and Selection of Applications

This chapter examines the conduct of the 2014 Specialist Training Program funding round, including the development of program guidelines and funding priorities, the application, assessment and selection process.

Introduction

2.1 The 2014 funding round was the fourth to be conducted following the Australian Government’s decision in 2010 to significantly expand the Specialist Training Program (STP). The 2014 round was intended to fund an additional 150 training positions, so as to bring the total number of STP training positions up to the target of 900.

2.2 The application, assessment and selection process for the 2014 funding round broadly followed the model established in previous rounds. The only significant change was that applications were to be completed and submitted electronically utilising a ‘smartform’, with storage and processing of the applications occurring through an existing Department of Social Services (Social Services) online funding management system, FOFMS. This process was intended to improve Health’s grants administration by making it easier to apply and achieving efficiencies in the department’s handling of applications during the assessment and selection stages.

Development of grant guidelines and application documentation

2.3 Under the Australian Government’s grants framework, all grant activities must be administered according to a set of approved guidelines. These guidelines must also be made publicly available.90 The STP guidelines were originally developed by Health in 2009 and subsequently approved by the Expenditure Review Committee of Cabinet in December 2009, as then required by Australian Government policy. Consistent with changes in Australian Government policy, subsequent revisions of the STP guidelines (including in relation to the 2014 round) were subject to a risk-based assessment, rather than Expenditure Review Committee consideration, involving consultation with the Department of the Prime Minister and Cabinet and the then Department of Finance and Deregulation.

2.4 The STP grant guidelines were subsequently revised and renamed as the STP Operational Framework (Operational Framework) in 2011.91 The Operational Framework sets out: the program objectives and expected outcomes; governance arrangements; available funding amounts; eligibility requirements; and an overview of the application and assessment process. The Operational Framework is supplemented by the STP Priority Framework (Priority Framework), which sets out the funding priorities for each round.

2.5 Stakeholder input into the Operational Framework and Priority Framework for the 2014 round was received through the Medical Training Review Panel. In addition, work by Health Workforce Australia informed the development of the Priority Framework.92

2.6 Priorities for the 2014 round were:

  • private sector healthcare settings;
  • settings in regional, rural and remote areas;
  • non-hospital settings including aged care, community health and aboriginal medical services;
  • specific areas of medicine (obstetrics and gynaecology, ophthalmology, anatomical pathology, diagnostic radiology, radiation oncology, medical oncology, geriatric medicine and psychiatry);
  • generalist training93; and
  • dual training.94

2.7 In addition, the Priority Framework specified that preference would be given in the assessment process to training positions which demonstrated:

  • their capacity to be filled with Indigenous trainees;
  • specialist trainee involvement with clinical academic research or teaching junior doctors and/or medical students; or
  • capacity for an individual trainee to complete the majority of training requirements for Fellowship95 in an on-going position in a rural, regional or remote setting.

2.8 The Priority Framework for the 2014 round was approved by the Minister for Health in early March 2013. As with the previous round, the Minister delegated authority to approve the 2014 round application documentation to the department’s relevant First Assistant Secretary. The Minister also delegated authority to the First Assistant Secretary to approve the ‘outcomes’ of the round—that is, the selection of applications to be funded. In seeking the Minister’s approval, the department advised that ‘the Priority Framework is the key document that determines the composition of the outcomes’ with the actual approval of outcomes being ‘essentially administrative, as the assessment is based on the recommendations from the colleges and [state] health services’.

2.9 Immediately after receiving the Minister’s approval, the First Assistant Secretary approved the application documentation. This documentation included both public material (the Operational Framework, the Priority Framework, the application form and the associated guidance material for applicants) as well as the internal assessment plan. The documentation was also reviewed by Health’s internal grants advisory area96, although the relevant clearance indicates the review was done before the assessment plan was drafted. Other than a slightly different set of priorities, the only material change in the 2014 assessment plan over the 2013 round was the insertion of a section on ‘probity, accountability and ethics’.

Application process

2.10 As in previous rounds, a broad range of organisations were eligible under the Operational Framework to apply for an STP grant, including medical education providers, state health services (including local hospital networks and regional hospitals), community health organisations, private healthcare organisations and settings, and Aboriginal community–controlled health services.

2.11 The application process opened on 20 March 2013 and closed on 1 May 2013. Stakeholders advised the ANAO that the opening of the 2014 round was well publicised amongst potential applicants, including through the colleges advising their Fellows and other contacts. Stakeholders also considered that the six week period was generally sufficient to develop an application with all necessary supporting documentation97 and the information required in the application was proportionate to the nature of the grants on offer and the objectives of the STP. Some 467 applications were received for the 150 training positions on offer, with another 44 applications for the 22 Emergency Department Workforce training positions.98 The number of applications was similar to previous rounds, indicating on-going demand for STP funding.

Use of shared services to support the 2014 round

2.12 As noted by the 2014 National Commission of Audit99, one way that agencies can achieve efficiencies and reduce costs is through the appropriate use of shared services. Shared services involve the single provision of particular functions to more than one agency. They commonly cover activities in the areas of human resources, information management, communication, technology, procurement and financial management.

2.13 Since 2005, the Department of Social Services100 has administered the FaHCSIA Online Funding Management System (FOFMS). One of the main capabilities of FOFMS is the administration of grant activities. In 2012, under an agreement with Social Services, Health started to progressively roll out the use of FOFMS to manage a range of payments it administered.101

2.14 In late 2012, Health decided to use FOFMS to support the upcoming STP 2014 funding round. This involved the development of a tailored ‘smartform’ by Social Services102 that allowed applicants to complete and submit applications electronically, including letters of support from the relevant college and the local hospital network. Development of the smartform and integrated FOFMS process took place from October 2012 to March 2013. It was originally anticipated that the project would enable applications to open around early March, however the operational smartform was not delivered to Health until 18 March 2013.

2.15 Stakeholders advised the ANAO that they generally considered the electronic application process to be an improvement on previous rounds, although some problems were encountered when unexplained error messages were received when attempting to complete the form or where there were difficulties in attaching the required supporting documentation.

2.16 However, a number of more significant technical problems arose when receiving and processing applications, which reduced the overall efficiency and effectiveness of the application process. Notably, FOFMS experienced an outage on the day immediately before applications closed, and applications could not be submitted during this time. Some applicants instead emailed applications to the Health STP mailbox, and the volume of data caused the mailbox to crash. As a consequence, these applications had to be manually loaded into FOFMS by departmental staff. Due to the various technical issues encountered, a large number of duplicate applications were stored in and extracted from FOFMS, requiring significant work by departmental staff to identify which applications were duplicates.103 Further, Health had anticipated that it could implement automated screening of ineligible applications by FOFMS—for example, where applications did not include the required letters of support—but the technical problems precluded this.

2.17 A post-implementation review of the project by Social Services, with the involvement of Health, resulted in 23 recommendations intended to inform any future activity of this sort. Further development of the smartform application concept to support a FOFMS-based grants funding round for the Rural and Regional Teaching Infrastructure Program was undertaken by Health in 2014. A specific risk management process, derived directly from the review of the STP pilot, and involving more extensive testing, training and technical support, was developed to inform the use of FOFMS for the Rural and Regional Teaching Infrastructure Program funding round. However, in October 2014, Health advised the ANAO that the Rural and Regional Teaching Infrastructure Program round, which was due to open for applications in September 2014, did not go ahead as the program was put ‘on hold’.

Assessment process

2.18 A key consideration in administering granting activities is to implement a selection process that identifies and recommends for funding those applications that will provide greatest value with public money in the context of government objectives for the granting activity, as set out in the grant guidelines.104 Well-designed assessment processes can provide confidence that decision-makers have equitably and transparently selected applicants that best represent value for public money in the context of the program objectives and outcomes.105

Assessment criteria

2.19 The instructions provided to applicants for completing the application form stated:

Each eligible application will be assessed by:

  • the relevant specialist medical college for rating in terms of the position’s ability to meet the appropriate educational imperative; and
  • the relevant state or territory government for rating in terms of jurisdictional areas of workforce need.

These assessments will be provided to the Department of Health and Ageing who will collate assessment results for the decision maker. The outcome of the 2014 STP Application Round will be based on the 2014 Priorities and further application weightings as detailed in the 2014 STP Priority Framework (Attachment B). These weightings are in no particular order. Additional information may be sought from the relevant college or jurisdiction to assist with this process.

2.20 Other sections of the lengthy instructions indicated that the colleges and state health services would also be involved in assessing applications against the Priority Framework, and would not be restricted to assessing educational and workforce need aspects.

2.21 In summary, applications would be assessed by third parties—the colleges and state health services—and the assessments collated by the department prior to consideration by a departmental decision-maker.

Assessment of applications by the colleges and state health services

2.22 Health’s internal assessment plan provided for an initial screening of applications by the department against mandatory requirements. However, the initial screening did not occur due to the problems with FOFMS, discussed above, and all applications were accepted as compliant and proceeded to assessment. 106

2.23 The assessment process was conducted in two stages. The first stage was an assessment by the colleges and state health services. Health provided the colleges and state health services with a four–page template to assist in applying a consistent approach to assessments and a spreadsheet on which their assessment responses could be recorded. The template and spreadsheet provided for assessment against two key criteria:

  • The extent to which the application met each of the Commonwealth priorities and preferences in the Priority Framework outlined in paragraph 2.6–2.7. Assessment responses were required to indicate ‘yes’ or ‘no’ against each priority and preference.
  • An overall ‘global rating’ of the application using a four point sliding scale—strong support, moderate support, minimal support or no support. Health’s advice to the colleges and state health services was that, in addition to the application’s strength in addressing the Priority Framework, the global rating should reflect (in the case of the state health services assessments) ‘the benefit to health services’ and (for college assessments) the ‘educational merit’ of the application.107

2.24 Applications were provided to the relevant colleges and state health services by Health on 17 May 2013.108 Assessment responses were generally returned to the department by mid-June, although one was not received until late June.109

2.25 The colleges and state health services advised the ANAO that they took a number of different approaches to how they assessed applications, including through established or ad hoc committees, providing the applications to local health networks or, in the case of the colleges, having senior officers or Fellows of the college advise on the applications. Similarly, the approaches adopted by the colleges and state health services in managing potential conflicts of interest varied. Most colleges and state health services told the ANAO that either formal or informal mechanisms were in place to manage the risk of real or perceived conflicts of interest.110 Nonetheless, in one instance, a senior manager of a public hospital was directly involved in advising the jurisdiction’s Chief Medical Officer on the assessment of applications from the public hospital at which that manager was employed. As such, the manager had a direct interest in the outcome of these applications.

2.26 The assessment documentation provided to colleges and state health services by Health did not address conflict of interest issues. Health advised the ANAO that:

The Department did not specifically request the colleges/jurisdictions to ensure that reasonable conflict of interest provisions were in place for the 2014 funding round. It should be noted that most colleges and jurisdictions had completed three assessment processes before with the operational framework and assessment guidelines distributed to the colleges/jurisdictions clearly explaining roles and included explanatory notes. The Invitation to Apply [documentation] also included a section on how the applications would be assessed and described the STP Complaints Handling Procedures.

No issues have been raised with the Department about possible conflict of interest issues arising during the 2014 funding round.

2.27 Since its introduction in 2009, the Australian Government’s grants administration framework has consistently highlighted the need to manage the risk of real or perceived conflicts of interest.111 A recent ANAO performance audit of Commonwealth entities reviewed how conflicts of interests were managed for grants programs where applications were subject to assessment by third parties.112 That audit observed that by its very nature, third-party assessment through peer review processes presents inherent conflict of interest issues. Appropriate management is therefore necessary to satisfy grants framework requirements that agencies ensure the impartiality of decision-making.113 While Health was the ultimate decision-maker regarding the selection of successful applicants, the college and state health services assessments informed Health’s decisions and hence funding outcomes. Should any future funding rounds be conducted and assessed in a similar way, there would be merit in the department providing appropriate guidance to colleges and state health services regarding its expectations around the management of potential conflicts of interest.

Calculation of scores by Health

2.28 The second stage of the assessment was undertaken by Health. The assessment plan provided that, subject to receiving at least a moderate global rating from both the college and state health services, applications would be assessed against the Priority Framework with the ‘final ranking based on the number of priorities the proposed training position addresses’.

2.29 In fact, all applications were assessed against the Priority Framework. As in the previous round, Health did this by taking the college and state health services assessment responses—relating to whether individual applications met the priorities and preferences set out in the Priority Framework—and converting them into a score. Table 2.1 shows the relative weighting of the various priorities and preferences used to calculate an applicant’s score. The weightings were not included in the application documents provided to applicants or the departmental assessment plan.

Table 2.1: Relative weighting of priorities and preferences

Priority or preference

Score

Priority setting

3 points

Priority specialty

1 point

Generalist training

2 points

Dual training

1 point

Indigenous health component or commitment to support indigenous trainee

2 points

Trainee involved in teaching or research

1 point

Majority of Fellowship training requirement able to be completed in a rural, regional or remote location

2 points

Maximum total score

12 points

Source: 2014 round Health assessment report.

Notes: A priority setting means a setting in the private health sector, in a regional, rural or remote area, or an aged care, community health or Aboriginal medical service or facility.

2.30 By the end of the second stage of assessment, each application had received two scores out of 12 from Health: one based on the college assessment, and one on the relevant state health service assessment.

2.31 To assess the accuracy of Health’s calculation of scores, the ANAO replicated this scoring process based on the Health weightings reported in Table 2.1. The ANAO analysis was based on the college and state health service assessment responses for 449 out of 467 applications assessed in the 2014 round.114 The analysis identified 178 applications (39.6 per cent of the applications included in the ANAO analysis) in which the score recorded by Health on its assessment database varied from that calculated by the ANAO.

2.32 The ANAO raised the issue of variances with Health, as there was nothing in the assessment report or other records examined by the ANAO to explain the variances. Health advised that for some applications, the scores initially calculated by the departmental assessment team (based on the college assessment response) were different to those calculated from the state health service assessment response. For example, using the weightings in Table 2.1, a college assessment response might equate to a score of eight, but the relevant state health service assessment for the same application might only equate to a score of six. In effect, this meant that the relevant college and state health service had different views on the extent to which an application met the various priorities and preferences contained in the Priority Framework.

2.33 Health advised the ANAO that the differences in the calculated scores had prompted the department’s assessment team to review the ‘initial’ scores of those applications receiving either strong or moderate global ratings, totalling around 250 applications. Following this review, the Health team then assigned a ‘final’ score to the application that they considered most accurately reflected the information in the application. In relation to this ‘review and rescoring’ procedure, Health was able to provide the ANAO with working papers that recorded the initially assigned application scores. However, Health did not document any of the key decisions involved in this procedure–in particular no records were kept of the specific reasons for making changes to the scores of individual applications. The department acknowledged that in hindsight, these changes should have been recorded on the assessment database. The Health assessment team advised that they could not recall any discussions with the colleges or state health services about this subsequent departmental review and rescoring process115, nor could they recall any discussion with the relevant Assistant Secretary within Health.

2.34 The department also acknowledged that there was no further quality control undertaken during the review and rescoring process and that this may have led to some variations or input errors that affected the final scores assigned by Health.

Ranking of applications by Health

2.35 Under the 2014 round, funding was available for 150 new training positions. While the global rating provided by the colleges and state health services was the primary factor in ranking applications and determining funding outcomes, the scores assigned by Health also played a role in the ranking process, and in some cases the department exercised discretion to not fund applications.116

2.36 As Table 2.2 illustrates, 147 applications of the total of 467 applications received a strong global rating by both the relevant college and state health services (a ‘strong/strong’ rating). Of these, 121 (82.3 per cent of all strong/strong applications) were ranked by Health in the ‘top 150’—that is, selected as one of the 150 applications to be offered STP grant funding.117 A further 22 strong/strong applications (15.0 per cent of all strong/strong applications) were placed on the reserve list, and four (2.7 per cent) were unsuccessful. Of the remaining 29 applications that were ranked in the top 150, all received a strong rating from the relevant college and a moderate rating from the relevant state health services, or vice-versa (a ‘strong/moderate’ rating). These 29 successful applications represented 27.1 per cent of all strong/moderate applications. The remaining 78 strong/moderate applications were placed on the reserve list.

Table 2.2: Ranking of 2014 round applications

Global

Total number of applications

Number of applications in top 150 (offered funding)

Number of applications placed on reserve list

Number of applications placed on unsuccessful list

Strong/Strong

147

121

22

4

Strong/Moderate

107

29

78

0

Strong/Other

103

0

0

103

Moderate/Moderate

20

0

20

0

Moderate/Other

57

0

0

57

Minimal/Other

33

0

0

33

Total

467

150

120

197

Source: 2014 round Health assessment report.

Note: The term ‘Other’ in the rating column means either minimal support or no support.

2.37 As noted in paragraph 2.35, in some cases the department exercised a discretion to not fund applications. Of the 22 strong/strong rated applications that were placed on the reserve list, 13 (representing 8.8 per cent of all strong/strong applications) involved situations where the hospital or medical organisation had submitted applications for two or more training positions in the same specialty.118 In such cases, Health funded only one place in order to obtain ‘a relatively even distribution of the new training positions against the population data’, an approach which also featured in the previous (2013) funding round.119 As a result, some relatively high–scoring, strongly–rated applications were not offered funding. The remainder of the unfunded strongly–rated applications were either relatively low–scoring (between two and four), or, in the case of the four unsuccessful applications, were from Tasmanian public hospitals and were not funded on the basis that specialist training in the Tasmanian public sector was being separately supported through the Tasmanian project.

2.38 The 29 ‘strong/moderate applications ranked by Health in the top 150 had scores ranging from a high of nine to a low of five, with three applications scoring five. However there were also 29 applications with the same ‘strong/moderate’ rating that scored either six or seven and which were placed on the reserve list. Health was unable to provide any specific evidence as to why the three applications scoring five were funded in preference to the 29 applications scoring six or seven.

Transparency of the scoring and ranking process

2.39 As noted in paragraph 2.8, Health had advised its Minister in March 2013 that the approval of assessment outcomes by the departmental delegate was ‘essentially administrative, as the assessment is based on the recommendations from the colleges and [state] health services’. While the college and state health service global ratings were the single most influential factor in deciding which applications received funding, the relevant score subsequently assigned by Health was also a factor. As such, the assessment team’s decision to change the initial scores across a substantial proportion of the applications through a review and rescoring procedure that was not documented was a feature of the assessment process.

2.40 Health’s approach was inconsistent with information provided to applicants—in the instructions for completing the program application form—that the department would collate assessment results received from the state health services and colleges.120 The failure to document the additional review and rescoring procedure in the program guidelines and instructions; the lack of record–keeping regarding the reasons for changing initial assessment scores; and the failure to provide advice on the procedure in the assessment report121 provided to the decision-maker, meant that the procedure was not consistent with sound practice.122 As discussed, Health acknowledged also that that there were no specific quality control measures in place for the additional review and rescoring, further increasing the risks introduced by the procedure.

2.41 As a result of the variances observed by the ANAO, the department undertook a complete rescoring of applications for the 2014 round.123 As at December 2014, this work remained incomplete.

2.42 Assessing the impact of the undocumented review and rescoring process was made difficult as there was not always a strict ‘cut off’ score for funding. However, there were 12 applications in the top 150 for which the scores calculated by the ANAO124 were between two and three and a half points out of 12 but were ultimately assigned a score by Health of between four and seven. In these cases, the department’s rescoring increased the prospects of applications being funded. Conversely, there was one application on the reserve list (which received a strong/strong rating) where the ANAO calculated a score of four but Health assigned a score of two. Had that application been assigned a score of four, it may have been placed in the top 150 and offered funding.

2.43 The ANAO’s analysis of the rescoring of the 13 applications discussed above (which together represented 2.8 per cent of the 467 applications received in the 2014 round) is illustrated in Figure 2.1. The hatched horizontal line at the score of four represents the general minimum score for applications placed in the top 150. As the figure shows, there were 12 applications where the score calculated by the ANAO, based on the college and state health service assessment responses (the ANAO score), was under four, but which were given a final score by Health of four or more; with one application having an ANAO score of four but a Health score of two.

Figure 2.1: Effect of rescoring process on selected applications

Source: ANAO analysis of college and state health services assessment responses and Health data.

2.44 In addition to the review and rescoring process, the department also decided not to fund some applications that received a strong/strong global rating and a relatively high score.125 As discussed in paragraph 2.37, 13 applications were placed on the reserve list where the setting had submitted applications for two or more training positions in the same specialty.126 In such cases, Health funded only one place in order to obtain ‘a relatively even distribution of the new training positions against the population data’. In adopting this approach, Health effectively applied a selection criterion that was not documented in the 2014 round application form or other explanatory material available to applicants. In adopting this approach, which also featured in the previous (2013) funding round127, the department effectively applied a selection criterion that was not documented in the application form or other explanatory material made available to applicants.128 Although the program’s funding priorities, which underpinned the assessment criteria, were reviewed between rounds, Health did not take the opportunity to incorporate a reference to the approach adopted on population distribution in the 2014 round application form or explanatory material. Nevertheless, when considered in the context of the program’s intended outcomes (which include achieving a better geographical distribution of specialist services) the department’s approach in relation to this matter was not unreasonable.

Recommendation No.1

2.45 To improve transparency and equity in the administration of grants, the ANAO recommends that the Department of Health:

  • review program guidelines and assessment criteria at the conclusion of grant funding rounds, to incorporate lessons learned; and
  • provide operational guidance to staff on moderation or other quality control processes to be applied where applications have been assessed by third-party advisers.

Department of Health Response:

2.46 Agreed.

2014 round outcomes

2.47 A total of 150 STP training positions were funded nationally in the 2014 round. Figure 2.2 shows the proportion of training positions located in each state and territory jurisdiction. The proportions broadly correspond to each state and territory’s share of population.129 However, the Northern Territory was funded for nine training positions; representing six per cent of all 150 training positions, in contrast to its population share of about one per cent.

Figure 2.2: Funded 2014 round STP training positions by State and Territory

Source: ANAO analysis of Department of Health data.

2.48 The distribution of STP training positions by medical speciality is shown in Figure 2.3. The proportion of specialties funded under the program generally aligned with the number of applications received for each specialty. For example, approximately half of all applications fell within the auspices of the Royal Australasian College of Physicians, and these specialties received 47.3 per cent of training positions.

Figure 2.3: Funded 2014 round STP training positions by medical specialty

Source: ANAO analysis of Department of Health data.

2.49 Approximately 45.3 per cent of successful applications were in medical specialties listed in the Priority Framework for the 2014 round with a similar number (47.3 per cent) incorporating generalist training, another specified priority.130 Applications having a rural or regional component were well represented (53.3 per cent)) with applications for training positions wholly or partly in the private sector at around the same levels (55.3 per cent). Fortythree percent involved a component of training in non-hospital settings. Eight per cent of successful applications identified a specific capacity to be filled by an indigenous trainee.

Advising applicants and the provision of feedback

2.50 The selection of the top 150 applications was approved by the departmental delegate on 19 July 2013. Immediately thereafter, letters were sent to all applicants advising whether they were successful or not (in the latter case, applicants were also advised whether they were placed on the reserve list and thus may be potentially funded at a later stage). The letters to successful applicants stated that a formal offer of grant funding was dependent on a contract first being executed between the department and the relevant college.

2.51 The letters also provided the STP mailbox email address should applicants ‘wish to discuss this matter further’. Departmental records indicate that 25 unsuccessful applicants sought feedback. In response to an invitation by the ANAO, some applicants contacted the ANAO in relation to the usefulness of the verbal feedback they received from Health through this process. Views were mixed, although the small number of applicants responding to the ANAO made it difficult to draw any firm conclusions.

2.52 Lists of successful applicants were also provided by the department to colleges and state health services. The colleges and state health services informed the ANAO that they considered the outcomes of the round to be generally consistent with their assessments, although several commented they would have appreciated additional feedback on the reasons why applications within their (state or college) jurisdiction were not successful.

Briefing the Health Minister

2.53 The Minister received departmental advice on the outcomes of the 2014 round through a minute submitted immediately after the Health delegate approved the outcomes. The minute did not require any decision by the Minister—it was for ‘noting’ only.131 Health advised the ANAO that this was the department’s only communication with the Minister or Minister’s office about the grant outcomes before the applicants were advised by correspondence.

Approval of funding

2.54 Following the delegate’s approval of the outcomes of the assessment process and the applicants being advised, Health started negotiations with the colleges to vary the existing funding agreements. However, this process was not completed before the caretaker period for the September 2013 Australian Federal election took effect.

2.55 The 2014 round STP grants were subsequently considered by the incoming Australian Government, and their implementation was agreed initially by the Finance Minister and then the Health Minister in late 2013, as part of a wider consideration of proposed grants. Following the review process, the departmental delegate gave financial approval under FMA Regulation 9 before formal offers were made to vary the existing funding agreements.132 The department provided the delegate with advice on the 2014 funding round assessment process and the role of the colleges in administering the STP training positions selected through that process.133

Conclusion

2.56 In the 2014 funding round, applications for STP grants were open to a broad range of organisations, consistent with the general program objective that specialist training occur beyond traditional teaching settings. The assessment process for the selection of grants was outlined in explanatory material to the application form. Applications were received from: state health services (including local hospital networks and regional hospitals), community health organisations, private healthcare organisations and settings, and Aboriginal community–controlled health services. Stakeholders informed the ANAO that the opening of the 2014 round was well publicised amongst potential applicants, and in the event, some 467 applications were received for the 150 available grants.

2.57 Health trialled the use of shared services for the 2014 funding round, to make the application process easier for applicants and to achieve administrative efficiencies. Applications could be completed and submitted electronically utilising a ‘smartform’, and were subsequently stored and processed through an existing Department of Social Services funding management system, FOFMS.134 However, there were some shortcomings in the approach adopted by Health. Stakeholders advised the ANAO that while the electronic application process had improved overall administration compared to previous rounds, the trial suffered from insufficient testing prior to implementation. In particular, the trial encountered a number of significant technical problems relating to the receipt and processing of applications, and the anticipated benefits and efficiencies were not fully realised.

2.58 The assessment criteria and selection processes outlined in the department’s assessment plan for the 2014 round reflected Australian Government priorities, which were informed by published research of Health Workforce Australia. Further, the involvement of specialist medical colleges and state health services in the assessment of applications strengthened the assessment process. In particular, the state health services and colleges provided third-party advice to Health on the educational merit of applications, the potential impact of applications on health services, and the extent to which applications met program funding priorities.

2.59 ANAO testing of Health’s final assessment of applications for the 2014 funding round indicated that the department adopted an internal review and rescoring process after receiving input from state health services and the colleges. Around 250 of the 467 applications received in the 2014 STP funding round were reviewed as part of this process, which was not documented. ANAO analysis indicates that the rescoring directly affected funding outcomes for 13 applications, representing some 2.8 per cent of all (467) 2014 round applications. For 12 of these applications, the final score assigned by Health was above that which was calculated by the ANAO based on input from the specialist medical colleges and state health services, indicating these applications likely benefitted from the review and rescoring process to the extent that they were offered grants. As there were only 150 grants available through the 2014 round, the elevation of the 12 applications meant that some applicants that may have otherwise been offered a grant were not. One application was scored down by the department and as a consequence of this, was not offered a grant.

2.60 The department decided not to fund some highly-rated applications received as part of the 2014 funding round. Specifically, 13 applications were placed on a ‘reserve’ list135, as the relevant setting had submitted applications for two or more training positions in the same specialty.136 In these cases, Health funded only one place in order to obtain ‘a relatively even distribution of the new training positions against the population data’. In adopting this approach, Health effectively applied a selection criterion that was not documented in the application form or other explanatory material made available to applicants for the 2014 funding round.137 Health’s approach was inconsistent with instructions provided to applicants for completing the application form, which indicated that the department would ‘collate’ assessment results received from state health services and the colleges. Further, the approach adopted was not consistent with advice provided to the Health Minister in March 2013 that the department’s selection of applicants to be funded under the STP was ‘essentially administrative’, as individual assessments were based on recommendations received from the state health services and colleges.

2.61 The lack of appropriate record-keeping and quality control in the internal review and rescoring process, and the use of a selection criterion that was not contained in the 2014 round application form or other explanatory material available to applicants, affected the transparency and to an extent the equity of the assessment process when viewed in terms of the application form and other explanatory material that informed applicants’ expectations about how grants would be selected.138 However, the department’s approach in relation to this matter—of using a selection criterion that incorporated population distribution considerations—was not unreasonable in the context of the program’s intended outcomes.

3. Administration of Funding Agreements

This chapter examines Health’s administration of the Specialist Training Program funding agreements with the specialist medical colleges.

Introduction

3.1 The Specialist Training Program (STP) funds139 are disbursed under 13 individual funding agreements between Health and each college participating in the program.140 These agreements give effect to a ‘college administration’ model as discussed in the following paragraphs.

The college administration model

3.2 Health’s administration of STP grants has undergone a considerable transition over the life of the program. Initially, some of the grants were administered by colleges141, while the majority were administered by Health, including through its state offices. However, from 2010 the department relinquished its direct role in dealing with the settings and the relevant colleges assumed responsibility in this regard.142 When seeking approval to move to the ‘college administration’ model in 2009, the department advised the Minister that:

… the current arrangement [through which the Department manages funding agreements primarily with individual practice settings] is fragmented and whilst it delivers a range of training posts, these are not integrated in a network arrangement or closely aligned with the skill needs of individual registrars. The new arrangement brings together the management of training posts function within organisations that are already responsible for the quality of training arrangements, responsible for the delivery of training curriculum and assessment of registrars for Fellowship. It is proposed that this arrangement provides a better platform for the delivery of networks of high quality training positions.

3.3 Under this model, the administration of STP grant funds is governed through separate funding agreements between Health and the respective colleges. For example, all grant funding for STP training positions providing training in surgery is provided to the Royal Australasian College of Surgeons (RACS). Under the relevant funding agreement, RACS is responsible for:

  • disbursing funds to the individual settings143 that operate STP training positions;
  • oversighting the conduct of the STP training positions through contractual144 and liaison arrangements with the individual settings;
  • the development and implementation of strategic training and education projects to support the network of STP positions;
  • promoting the integration of training provided through STP training positions with that provided by state health services; and
  • providing periodic progress and financial reporting to Health.

3.4 The STP funding agreements each consist of a ‘heads of agreement’ based on Health’s standard–form grant contract, and a project schedule. The project schedule contains the detailed STP–specific obligations, mentioned above, lists the training positions administered by the college, the amounts of the various funding elements145, the progress payment schedule, and details of the periodic and final reporting required from the colleges.146

3.5 The project schedules accompanying funding agreements are amended as necessary through formal deeds of variation. Variations are typically made after every funding round (to reflect increases to the number of training positions being administered by the relevant college), when major changes have been made to reporting requirements and the like, or when additional specialist training grant funding is provided. The most recent variations were made between late 2013 and early 2014 to incorporate the results of the 2014 funding round.

3.6 Over 2010–2016147, the colleges may receive up to $33.68 million148 from the Australian Government for the administration of training positions and related tasks.

The application of the Financial Management and Accountability Act 1997 to the specialist training colleges

Section 12 of the FMA Act

3.7 The purpose of the Australian Government’s resource management framework is to promote the proper use of public resources.149 Where the Commonwealth uses third parties to make payments on its behalf, it is necessary for Australian Government entities to assess the possible application of framework requirements.

3.8 Section 12 of the Financial Management and Accountability Act 1997 (FMA Act) established special requirements for Australian agencies that entered into agreements or arrangements for the receipt, custody or payment of public money by ‘outsiders’, including third parties involved in government administration such as the colleges. Agencies either had to ensure that outsiders complied with all the requirements of the Commonwealth financial framework, or alternatively could make special arrangements through a ‘Section 12 agreement’, which specified a set of requirements to be met by the outsider.150

3.9 Health did not fully assess the potential implications of section 12 for the college administration model when it was first proposed in 2009, and the department did not seek legal advice on the issue at that time.151 The department sought legal advice from the Australian Government Solicitor during the course of the audit, after the matter was raised by the ANAO. Health’s advice indicated that while there were ‘arguments either way’ on the matter, ‘on balance’, college staff were not subject to the FMA Act in administering STP grant funds.

3.10 In July 2014, the FMA Act was replaced by the Public Governance, Performance and Accountability Act 2013 (PGPA Act). While the PGPA Act does not include a provision comparable to section 12 of the FMA Act, it places a responsibility on entities to assess risks before entering into financial arrangements.152 It is prudent for Australian Government entities to consider, at the design stage, the full implications of complex financial and administrative arrangements.153

How Specialist Training Program administration funding is calculated

3.11 In 2010, Health decided that the administration funding provided to colleges would be set at a maximum of 10 per cent of the total salary support component of the training positions being administered by the colleges. This equated to an average upper limit of $10 000 per position, per year, although the actual amount payable to each college was to be negotiated on a case-by-case basis to ‘ensure a reasonable allocation’ for each college. The relevant record of that decision does not indicate as to why $10 000 was considered an appropriate upper limit.

3.12 With each successive funding round increasing the total number of training positions funded by the STP, individual colleges had a corresponding rise in the number of training positions they administered. As a consequence, the level of administrative funding provided by Health to the colleges also generally increased. However, for the majority of colleges, the annual funding measured on a ‘per position’ basis has generally declined over time, indicating efficiencies have been achieved by the relevant colleges over the course of the program. Figure 3.1 illustrates this trend through the last four funding rounds, with funding levels as at the 2014 round ranging from $5098 to $10 263 per position, per year. Two further colleges (not shown in Figure 3.1), with relatively small numbers of administered training positions, are funded at a somewhat higher level ($11 053 and $15 914), with the remaining college being funded at $32 800 per position, per year. This last college had only commenced participation in the STP from the 2013 funding round, and was also considered by Health to require additional funding for the administration of strategic support projects.

Figure 3.1: Trends in STP administration funding 2011–2014

Source: Department of Health data.

Notes: The Department of Health administered the Anaesthetic STP training positions in 2011. The Australian and New Zealand College of Anaesthetists administered all relevant STP training positions from 2012.

3.13 As part of negotiations for additional administrative funding flowing to colleges from the 2014 round, Health required colleges to provide estimates of forward budget requirements and reviewed 2013 college expenditure reports. While the level of detail in the expenditure reports and budget estimates varied significantly, email trails of the negotiation processes examined by the ANAO indicate that in a number of cases, the department negotiated amounts downwards from initial college budget estimates. In some cases no additional funding was provided where underspends of administrative funding provided under previous STP funding rounds were identified.

3.14 To maintain consistency of approach in its funding of colleges, there would be benefit in the department clearly documenting the basis for agreed administration funding at the conclusion of negotiation processes.

Salary support and rural loadings

3.15 Participating colleges receive two discrete elements of STP funding from Health which are then on-paid to the settings154:

  • salary support; and
  • rural loading.

3.16 Salary support funding is provided through the colleges to settings to assist them with the cost of specialist trainee salaries. Each full-time equivalent (FTE) position receives $100 000 per year. Salary support is by far the largest funding element of the STP.155 Collectively, the funding agreements provide that up to $387.55 million in salary support is payable to settings via the colleges between 2010 and 2016.156

3.17 The salary support funding provided to each college is based on a simple calculation of the number of FTE training positions administered by the college. In relation to the additional salary support funding flowing from the 2014 round, ANAO analysis indicated that the amounts in the funding agreement variations were, with one minor exception157, calculated in accordance with Health’s funding methodology.158

3.18 Before making a salary support payment, colleges require STP settings to provide them with invoices or other documentation showing that the position was filled by a specialist trainee. Payments are generally made twice a year in arrears. The six-monthly college reporting provided to Health in July 2014 indicates that a small number of settings were late in providing relevant documentation, delaying payments and contributing to underspends by colleges against their projected annual expenditure.

3.19 The settings consulted by the ANAO indicated that there was an increasing gap between the full cost of employing specialist trainees (particularly more senior specialist trainees) and the amount of the STP grant.159 Grant funding amounts for salary support and rural loading (at $100 000 and $20 000 respectively) were set by the then Australian Government in 2009 through the program guidelines and have not been revised. A review of the STP ‘funding model’ for 2011–2014 (including the individual components such as salary support and rural loading) was undertaken by Health in 2010. However, the review was silent on whether the funding levels of these components remained appropriate going forward.160

3.20 Private sector settings advised the ANAO that where the specialist trainee was seconded to a private sector STP position from a public sector hospital, the specialist trainee generally remained on the public sector payroll. In such cases, the public hospital would require the private sector setting to reimburse the specialist trainee’s salary (and in most instances, on-costs such as superannuation) so that the public hospital could fund the ‘backfilling’ of the specialist trainee’s position.161 In effect, the STP grant funds were used by the private sector setting towards these reimbursement costs. A simplified illustration162 of this process is shown in Figure 3.2.

Figure 3.2: Simplified STP funding flow

Source: STP Operational Framework and ANAO discussions with stakeholders.

3.21 If the position is partly or wholly in a regional or rural area163, settings are eligible to receive a rural loading allowance of up to $20 000 per FTE per year. Collectively, the funding agreements provide that up to $31.83 million164 is payable for rural loading to eligible settings via the colleges for the period 2010–2016.165

3.22 In relation to the rural loading funding flowing from the 2014 round, ANAO analysis indicated some relatively small variations between the funding amounts contained in the deeds of variation and the information contained in relevant appendices (which form the basis for calculating these amounts).

3.23 Discussions with stakeholders also indicated that colleges adopted different approaches in the way that rural loading is paid to settings for STP training positions. Some colleges indicated they will not pay the rural loading allowance unless settings provide appropriate receipts or other documentation, while other colleges allocate the rural loading allowance automatically. Where STP training involves rotations between metropolitan and rural or regional settings, one college commented that significant effort was required to confirm whether the rotations166 had in fact taken place as originally planned, and checks were not done on a regular basis.

Execution of the 2014 round variations

3.24 The Commonwealth financial framework in operation until 1 July 2014 required that all proposals to commit public money receive formal approval under FMA Regulation 9 before Australian Government agencies entered into contracts or funding agreements.167 Funding agreements were also required to be consistent with the Regulation 9 approval.168 Financial approvals for the 2014 STP funding round were for sums ‘up to’ certain specified amounts. ANAO analysis indicates that the amounts contained in Health’s deeds of variation with the colleges did not exceed the maximum limit approved, and the number of new training positions to be funded was consistent with the relevant approvals.

Public grants reporting

3.25 Effective disclosure and reporting arrangements for grants administration is essential for reasons of transparency and public accountability. Reliable and timely information on grants awarded is a precondition for public and parliamentary confidence in the quality and integrity of grants administration.169

3.26 Under the 2013 Commonwealth Grant Guidelines, which applied at the time the 2014 round funding agreements were entered into, an agency was required to publish, on its website, information on individual grants no later than 14 working days after the grant agreement took effect.170 Where existing agreements were varied—as with the STP—this timeframe did not apply, but it was considered ‘better practice for agency staff to amend the website to reflect the variation as soon as practical’.171

3.27 The ANAO examined the department’s grants and tenders homepage as at 31 July 2014, and found that only a minority of the variations had been reported on Health’s grants reporting website. The department advised the ANAO that the exact publication dates had not been recorded for the variations prior to January 2014—only the financial year the data was published. Health advised the ANAO that variations data cannot be extracted from FOFMS, and the department is therefore reliant on manual reporting of variations. Health further advised the ANAO that a system for manual data capture was put in place in January 2014 by its Grant Services Division pending the introduction of an automated system, implemented from 9 January 2015. The new system is intended to include variations in grant reporting, to ensure that all new and varied grants are published within the reporting timeframes required by the Commonwealth Grants Rules and Guidelines. 172

Administration of the reporting and payment regime

The reporting regime

3.28 The STP and the Tasmanian project funds are administered by the same colleges, through a single funding agreement between each of the relevant colleges and Health. The funding agreements outline the responsibilities of all parties, including: payment of funding; use of funding; audit and monitoring; reporting requirements; and termination of the grant. Under the funding agreements, the specialist medical colleges are required to provide separate sixmonthly progress reports for both STP and the Tasmanian project by the end of January and July each year.

3.29 The funding agreement terms for payment of STP and the Tasmanian project funding differ. Under the funding agreement the terms for the payment of STP funding to the colleges are dependent on Health’s acceptance of the report provided. Whereas the terms for the Tasmanian project funding payments are not explicitly linked to satisfactory reporting or any other defined deliverable from the specialist medical colleges—rather the payments are made on a set date in June and December each year.

3.30 Despite these differences in payment conditions, the funding agreement terms provide Health with the discretionary power to:

  • defer, reduce or not make any payment under the funding agreement if the specialist medical college has either overspent or underspent funding; and
  • request repayments of funds in situations where the department has formed a reasonable opinion that funds have been spent or committed other than in accordance with the terms of the funding agreement.173,174

3.31 The reporting requirements for STP training positions have evolved since the first agreements were executed in 2010. Increased detail in financial reporting175 was required from late 2012, and specific key performance indicator (KPI) reporting was developed in late 2013.

3.32 More detailed financial reporting was introduced following an internal Health review of its administration of one STP funding agreement. The review highlighted several areas of concern including:

  • a significant level of unspent STP funds being held by the relevant college;
  • STP funds being mixed with other college funds in banking and investment arrangements; and
  • a substantial level of vacancies in STP training positions managed by the college.

3.33 Reporting against the Tasmanian project is relatively light touch, requiring colleges to advise on:

  • current expenditure and funds ‘committed’;
  • whether planned specialist trainees, supervisor and coordinator positions have been filled;
  • the extent to which outcomes have been achieved;
  • key issues and barriers to successful implementation of the project (including a ‘handling strategy’ in respect of these);
  • commentary on uncommitted funds; and
  • anticipated key actions for the next six months.

3.34 Annual college STP audited financial statements must also cover Tasmanian project revenue and expenditure.

3.35 The timing of the periodic reports allows colleges to collect the required information from the settings in order to provide the necessary information to Health on the preceding period—reports submitted to Health in July will cover the preceding January-June period. Colleges generally report that they are able to obtain most of the information required, although some reports examined by the ANAO stated that up to five percent of settings had not provided relevant information by the reporting date. ANAO analysis of the January and July 2014 college reports indicated that around 90 per cent of reports were received by Health within a day or two of the due date, although on a number of occasions the department has subsequently sought additional information.

3.36 The general view from the settings interviewed by the ANAO was that the reporting requirements were not excessive, and that relationships with the colleges on STP matters were good. The only negative issue raised by settings on a fairly regular basis was that, if a setting was hosting multiple STP training positions across different specialties, they had to provide different forms of reports to the relevant colleges, as each college had somewhat different reporting requirements.

3.37 On balance, the department’s reporting requirements have had regard to the proportionality principle of the Australian Government’s grants framework, which provides that grants processes should be commensurate with the scale, nature, complexity and risk involved in the granting activity.176

Assessment of college reporting by the Department

3.38 A well-drafted funding agreement alone is not sufficient to ensure the objectives of the grant are met. Agency staff should ensure that funding agreements are supported by ongoing communication, active grants management and performance monitoring requirements, which are proportional to the risks involved.177

3.39 Under the STP, assessment of college reports is undertaken by a departmental grants officer, who makes a written recommendation to a decision maker as to whether the report should be accepted. As noted above, the grants officer sometimes seeks further information from the college before making a recommendation. The assessments are based on a standardised template and have two separate components: assessment against the KPIs, and assessment of the financial information. Typically the assessments are completed and then considered by the decision maker within one to two weeks of all the required information being received by Health. Acceptance of a report by Health triggers the execution of a progress payment in FOFMS, although as noted in paragraph 3.43, Health may decide to withhold part (or the whole) of the scheduled payment due to underspends by the college.

ANAO analysis of reporting and assessments

3.40 A key element of the department’s STP monitoring approach is college reporting against outcome-oriented KPIs that were developed in consultation with the colleges during 2013.178 While these were required to be reported on from January 2014, it was evident from the ANAO’s analysis of the July 2014 STP reports that for some KPIs, a minority of colleges were unable to collect the relevant information to allow reporting. The colleges raised this issue with Health at the February 2014 annual intercollege STP meeting. While at the time the department agreed to review the issue, there has been no further work by Health on this.179

3.41 A fundamental KPI against which the colleges must report is the number of STP training positions filled and the corresponding vacancy rate. The department advised the ANAO that the colleges were required to report against this KPI on a FTE basis, but the actual reports did not always do so. There were also occasions where the spreadsheet showing the status of each position was inconsistent with the level of vacancies contained in the KPI reporting template, although the inconsistencies were relatively minor in terms of the number of training positions involved. In some cases, a small number of settings had not reported to colleges, and these reports noted that the actual overall vacancy rate might vary somewhat from the rate in the July 2014 report.

3.42 ANAO analysis indicates that around 833 of the 900 FTE training positions were filled as at 30 June 2014, equating to 93 per cent of training positions filled on an FTE basis. Health’s assessment of the July reports rated a number of colleges as ‘unsatisfactory’ on this issue, but recommended acceptance of the report on the basis there was evidence that the relevant college had committed to a specific strategy to fill significant numbers of additional STP training positions in the second half of 2014 and/or 2015.

3.43 As at 31 December 2013, collective surpluses of STP funds held by the colleges were $36.28 million.180 By 30 June 2014, this had risen to $56.31 million, equivalent to 16.4 per cent of the total STP funds that had been paid to the colleges by Health under the relevant agreements to 30 June 2014.181 While the surpluses can be partly attributed to timing issues182, Health has responded by withholding a proportion of the scheduled progress payments. Some $23.89 million that was due to be paid following receipt of the July 2014 reports was withheld.183

3.44 Expenditure of 2013–14 funds for the Tasmanian project was also relatively low.184 Colleges received $10.81 million in funding in 2013–14 with around half received in June 2014, but progress reports indicate that only $1.59 million had been spent. However, the level of ‘committed’ funding through contracts with the relevant settings was considerably higher.

3.45 The presentation of financial information by colleges, particularly income and expenditure, varied significantly from college to college and sometimes made it difficult to assess how funds had been spent, particularly on administration. In one case, Health concluded in its assessment that the relevant college’s reported cash surplus was some $600 000 below that indicated in the college’s bank statements included in its financial report.185

Progress payments to colleges

3.46 Progress payments are managed through FOFMS, in part to allow the department to automate payment processing. The department advised that some other payments are date based and automatically released to reduce the administrative burden on Health and the colleges.

3.47 The ANAO examined FOFMS payment records over the last two years. Only one payment was not explicitly approved by Health, and this was a result of the department deciding to withhold a portion of an earlier scheduled payment to a college (some $453 000) until the next progress report was submitted. The release of the withheld funds was to be considered as part of the next progress report assessment by Health. Through an oversight, there was no reference to the withheld amount (and hence no recommendation as to whether it should be released) in the subsequent assessment report approved by the Health, however the amount was released in FOFMS regardless. The department’s experience in this case is a reminder of the need to apply relevant financial controls.

3.48 As noted in paragraph 3.43 Health decided to withhold some $23.89 million in payments scheduled to be made on acceptance of the July 2014 college progress reports. The ANAO’s examination of FOFMS records indicates that the subsequent payments were reduced consistent with the amounts approved by the decision maker.

Conclusion

3.49 A feature of the STP is that while Health has overall responsibility for its administration, the department does not have a direct contractual relationship with individual settings, which are the grant recipients. Rather, the administration of STP grant funds is managed through separate agreements between Health and the respective colleges. Under this ‘college administration’ model, developed by the department in 2010, all grant funding for STP training positions within a particular specialty is provided by Health to the relevant college. The colleges may receive up to $33.68 million in 2010–2016 for the administration of training positions and related tasks. Most colleges receive between $5 098 and $10 263 per position for administrative purposes, although colleges with relatively small numbers of training positions to administer tend to receive somewhat higher funding.

3.50 The colleges and the settings interviewed by the ANAO indicated that the ‘college administration’ model generally worked well. However, some potential risks were not fully assessed by Health when developing the model, including the risk that the Australian Government’s financial framework requirements might apply to colleges if they handled public money. It is prudent for Australian Government entities to consider, at the design stage, the full implications of complex financial administrative arrangements—such as those involving third-party administration of government programs—so as to avoid potential compliance and reputational risks.

3.51 The department relies on reports received from the colleges to inform its oversight of the college administration model. Since 2012 Health has sought additional management and performance information from the colleges, particularly relating to financial issues. On balance, the department’s reporting requirements have had regard to the proportionality principle of the Australian Government’s grants framework, which provides that grants processes should be commensurate with the scale, nature, complexity and risk involved in the granting activity.

3.52 However, the presentation of financial information by colleges–particularly income and expenditure–has varied significantly, sometimes making it difficult for Health to assess how funds have been spent. Where variations in reporting have occurred, Health has undertaken follow-up communication with colleges to determine their actual financial position.

3.53 As at 31 December 2013, total surpluses of STP funds held by the colleges were $36.28 million. By 30 June 2014, total surpluses had risen to $56.31 million. The surpluses can be partly attributed to timing issues, including delays in the submission of invoices by training settings to the colleges. During 2014, Health has responded more actively where surpluses were identified by withholding a proportion of scheduled progress payments. In consequence, $23.89 million that was due to be paid following receipt of the July 2014 college reports was withheld.

4. Program Performance and Evaluation

This chapter examines performance monitoring and evaluation for the Specialist Training Program.

Introduction

4.1 A key principle of grants administration is the achievement of government policy outcomes.186 An effective performance framework facilitates assessment of the extent to which grants of financial assistance have contributed towards achieving the stated program outcomes and objectives.187

4.2 The Parliament and the public’s consideration of a program’s performance, in relation to its impact and cost effectiveness, rely heavily on reliable and appropriate performance information.188 Within the context of the Australian Government’s Outcomes and Programs Framework, key performance indicators (KPIs) are established to provide information on the effectiveness of programs in achieving objectives in support of outcomes.

Development of key performance indicators

4.3 When the then Australian Government was considering the proposed expansion of the STP in 2010, Health advised Ministers that the key performance measure for the program was the ‘take up rate and completion of vocational training placements’ which would be ‘assessed annually’. This simple performance measure was reflected in the first of several KPIs included in the program guidelines for the first funding round of the expanded program:

  • training positions identified under each funding agreement being filled by an eligible specialist trainee, for the duration of the term specified.

4.4 Other KPIs adopted for the first funding round were:

  • appropriate flow of funds identified as a contribution to salary, to the employer of the specialist trainee;
  • improved quality of specialist training with specialist trainees gaining appropriate skills not otherwise available through traditional settings;
  • the number of specialist international medical graduates supported and attaining College Fellowship;
  • demonstration that the specialist training initiatives developed through resources supplied as part of the Agreement complement initiatives currently provided within the state health services; and
  • processes established which enable effective and efficient administration of specialist training positions.

4.5 Apart from noting that Health would require periodic information from STP training positions, the program guidelines were silent on how the KPI data would be collected and assessed. This was addressed through a STP funding agreement variation executed in late 2012, which contained a commitment from colleges to work with Health to develop new KPIs that would form part of the college’s periodic reporting obligations. This general approach was consistent with a finding of the 2013 Mason Review of health workforce programs that more ‘sophisticated’ KPIs should be developed for the STP and included in funding agreements, with a ‘broader focus on sustainability, health workforce distribution outcomes and impact on trainee career decisions’.189

4.6 The new KPIs, information on how they were linked with existing STP outcomes, and reporting requirements, were incorporated into the funding variations flowing from the 2014 funding round. The KPIs are shown in Table 4.1. Reporting against the KPIs started from January 2014, and form part of the colleges’ six-monthly reporting obligations.190

Table 4.1: Current STP key performance indicators

Key Performance Indicator

How this is reported

Link to outcome

A. Contracts in place for all existing STP posts.

B. Contracts for new STP posts under development as soon as Agreement with the Department is formally varied.

C. Maximise opportunities for funding by minimising vacancies.

  • Number of posts awarded / filled / vacancy rates.
  • Number of trainees trained under STP.
  • Ratio of reserve posts: awarded posts that are funded.
  • Increased number of specialists.

D. Number of posts in expanded health care settings.

  • Percentage of STP training that occurs in expanded settings.
  • Percentage of STP training that occurs in rural settings.
  • Specialist trainees rotating through an integrated range of settings.
  • Better distribution of specialist services.

E. Strategic support projects increase trainees’ and specialist international medical graduates (SIMGs) access to appropriate training.

  • Strategic support projects – timeliness and outcomes.
  • Increased capacity within the sector to train specialists.
  • Improved quality of training with trainees and SIMGs gaining appropriate skills not otherwise available through traditional settings.
  • Improved access to appropriate training for overseas trained specialists seeking Fellowship with the College.

F. All STP posts are appropriately accredited for training.

  • Number of accredited STP training posts, compared with number of all training posts.
  • Number and percentage of trainees who have passed all training requirements while in an STP post.
  • Increased number of specialists (Fellows of Colleges).

G. Trainees rotate through network.

  • Number and percentage of STP posts that are part of a training network.
  • No specific outcome

H. Evidence of STP training linking with state/territory training.

  • Number and percentage of STP posts in expanded settings that are integrated with training provided by state health services.
  • Specialist training initiatives that complement those currently provided within the States and Territories.

Source: Department of Health.

Note: KPIs measuring the timely provision of complete periodic reports, and participation in meetings or teleconferences involving the department, have been excluded from the above table as they are not directly related to the achievement of STP outcomes.191

4.7 The new KPIs are clearly linked to STP outcomes. The majority are also quantifiable, allowing trends to be measured over time, although Health has not set any targets for the desired level of achievement. In most cases the KPIs constitute proxy measures, which are indirect measures of effectiveness. Health confirmed that it did not undertake selective verification activities regarding the accuracy of KPI reporting by the colleges192, nor is this identified as a risk in the department’s 2014 STP risk management plan.

Program performance and the future of the Specialist Training Program

Program performance

4.8 Colleges report against the KPIs through a template provided by Health. Based on the July 2014 reports received from colleges, the majority of KPIs were clearly reported against. For other KPIs, a number of colleges qualified their figures in some way or stated that they were unable to report because they did not collect relevant information.193 The ANAO aggregated the July 2014 KPI reporting from the eleven colleges, with the results shown in Table 4.2.

Table 4.2: Aggregated college KPI results from July 2014 reporting

How KPI is reported

Percentage or total number

ANAO comment

Percentage of STP posts filled.

93 per cent

All colleges reported on this KPI.

Number of specialist trainees trained in STP posts.

1011 specialist trainees (some posts have more than one specialist trainee rotating through the post during the six month reporting period hence the number of specialist trainees trained is greater than the number of STP posts).

One college did not report against this KPI.

Percentage of STP training in expanded settings.

89 per cent

All colleges reported on this KPI.

Percentage of STP training in regional and rural settings.

41 per cent

All colleges reported on this KPI.

Number of accredited STP training posts, compared with number of all specialist trainee posts.

403/3749

Four colleges did not report against this KPI, with two others qualifying their reports as they do not record the total number of training posts.

Percentage of STP trainees who have passed all training requirements.

100 per cent

Four colleges did not report against this KPI.

Percentage of STP posts that are part of a training network.

87 per cent

Two colleges only reported on posts funded in the 2013 and 2014 round, meaning that reporting is only based on about two-thirds of total STP posts.

Percentage of STP posts in expanded settings that are integrated with training provided by state health service providers.

82 per cent

One college did not report against this KPI.

Source: ANAO analysis of July 2014 STP progress reports from colleges.

Note: The above percentages have been rounded to the nearest whole number.

4.9 A key program outcome relates to exposing specialist trainees to an increased range of work environments and patient needs, to improve the quality of their training. Overall, based on July 2014 college reports, ANAO analysis indicated that 89 per cent194 of STP training took place in expanded settings—that is, outside traditional metropolitan teaching hospitals. This aspect of the STP was consistently rated as one of the program’s strengths by the settings in discussions with the ANAO, a point also noted in specialist trainee surveys conducted by the colleges.

4.10 One setting advised the ANAO that the focus of STP on expanded settings—which are generally smaller in size than metropolitan teaching hospitals—had been influential in encouraging the relevant college to adopt a more flexible approach in accrediting training positions in non-traditional settings for specialist training purposes. A similar point was made in the Mason Review findings, which cited examples of ‘hub and spoke’ STP training models. Under this model, STP funding has facilitated an increased range of training opportunities by tapping into training capacity in smaller training settings that would not otherwise be able to support a specialist training position on its own. STP–funded ‘strategic support’ projects have also contributed in this regard.195 Collectively, these developments have contributed to the achievement of another key program outcome—increased capacity within the healthcare sector to train specialists.

4.11 In addition to exposing specialist trainees to a more diverse range of healthcare working environments, STP training positions in regional and rural areas are intended to contribute to a better distribution of specialist services, a program outcome of direct benefit to the community.196 Overall, 41 per cent of STP training positions were in rural or regional settings, with the figures for individual colleges ranging from 17 per cent to 75 per cent. The settings interviewed by the ANAO noted that it could sometimes be challenging to attract specialist trainees to rural or regional locations, and that STP funding (including the availability of the $20 000 rural loading) was of substantial assistance in this respect.

4.12 ANAO analysis of July 2014 college reports indicated that the overall proportion of STP training positions in expanded settings that are integrated with training provided by the relevant state health service providers is high at 82 per cent.197 While college reporting generally does not provide any substantial information on the nature of the ‘integration’, the figures suggest that the STP may be making a contribution towards improved coordination between the Commonwealth, states and territories in terms of specialist training pathways. This issue was highlighted as requiring attention by the Mason Review findings in 2013 and Health Workforce Australia in its 2012 Health Workforce 2025198 report. The broader coordination issue is now being examined as part of the work of the National Medical Training Advisory Network. 199

Evaluation and the future of the Specialist Training Program

4.13 In March 2012, the Health Minister’s policy approval was sought to extend STP funding to 2015–16, and as part of that request the department advised that:

it is intended that specialist training activities…will be evaluated in 2013 followed by annual reviews of individual training posts that will commence in the 2014 academic year. This evaluation will examine the effectiveness of the STP…in achieving the Government target of delivering an additional 680 additional specialists into the health sector by 2020.200

4.14 A high-level evaluation of the STP was undertaken in 2012–13 as part of the wider Mason Review of health workforce programs. The Mason Review recommended that201:

While STP has been a well received and apparently successful program, it is important that a full evaluation of the program should be carried out to verify that settings such as the mix of positions are optimal, and to inform the future development of the scheme.

In addition, existing STP posts should be reviewed by colleges (in discussion with the Department and other program stakeholders) to ensure they are meeting the objectives of the program. This may provide the opportunity to redirect funds to new training posts that may better meet emerging workforce priorities.

4.15 In publicly releasing the findings of the Mason Review in May 2013, the then government announced that it would take steps to implement a number of the Review’s recommendations. However, none of these directly related to the STP.202 The issue of a more detailed evaluation of the STP was not subsequently raised in departmental advice to the Minister, although the possibility of such an evaluation was discussed with colleges in early 2014 at the STP inter-college forum.203 Health advised the ANAO that in view of the commencement of the ANAO’s performance audit the department did not progress a planned evaluation, particularly as it may have duplicated the collection of information and had a significant impact on the colleges.

Conclusion

4.16 The STP has had KPIs in place since the consolidation of the program in 2009. However, explicit outcome-linked KPIs were only developed in 2013, and the colleges having reported against these KPIs since January 2014.

4.17 College reporting against the program’s KPIs indicates that the STP has been successful in utilising non-traditional settings to expand the number of training places for specialist trainees, with 89 per cent of STP-funded positions being located in non-traditional settings. In discussions with the ANAO, stakeholders have also suggested that the expanded range of work environments has contributed to the overall quality of training.

4.18 ANAO analysis of college reporting indicates that on a full-time equivalent basis, around 833 positions were filled as at 30 June 2014, representing 93 per cent of the target of 900 positions. Overall, the additional specialist trainee positions funded by the STP have boosted the availability of specialist services, including in regional and rural areas. However, it remains unclear to what extent the STP has, or will, contribute to an improved geographical distribution of specialist services to meet community need, over the longer term.

4.19 A relatively high-level evaluation of the STP was undertaken in 2012–13 as part of the wider Mason Review204 of health workforce programs. While the review concluded that the STP has been a well received and apparently successful program, it recommended a full evaluation to verify that policy settings, such as the mix of positions, are optimal and to inform the scheme’s future development. The Review also recommended that existing STP posts be reviewed by colleges (in discussion with the department and other program stakeholders) to: ensure they were meeting the objectives of the program; and with a view to redirecting funds to new training posts that may better meet emerging workforce priorities. In publicly releasing the findings of the Mason Review in May 2013, the then government announced that it would take steps to implement a number of the Review’s recommendations. However, none of these directly related to the STP.205

4.20 As discussed, Health advised the ANAO that it did not progress a planned evaluation in view of the commencement of this performance audit.

Appendices

Appendices

Please refer to the attached PDF for the Appendices:

  • Appendix 1: Entity Response
  • Appendix 2: Electoral Status of Selected 2014 round Grant Applications

Abbreviations

ACEM

The Australasian College for Emergency Medicine

ANAO

Australian National Audit Office

ASGC-RA

Australian Standard Geographical Classification—Remoteness Area

FaHCSIA

Department of Families, Housing, Community Services and Indigenous Affairs

FOFMS

FaHCSIA Online Funding Management System

FTE

Full-time equivalent

GST

Goods and Services Tax

Health

Department of Health

KPI

Key Performance Indicator

MTRP

Medical Training Review Panel

PICS

Private Infrastructure and Clinical Supervision

PGPA Act

Public Governance, Performance and Accountability Act 2013

RACMA

Royal Australasian College of Medical Administrators

SIMGs

Specialist International Medical Graduates

Social Services

Department of Social Services

STP

Specialist Training Program

Footnotes

1 People undertaking specialist medical training are generally known as registrars. For the purpose of this audit the term specialist trainees is used to mean registrars.

2 Known as the Department of Health and Ageing until machinery of government changes in September 2013.

3 N Roxon (Minister for Health and Ageing), ‘Training Record Numbers of Specialist Doctors’, media release, Canberra, 15 March 2010.

4 The remainder of the projected shortfall was to be addressed by state and territory governments increasing the number of training positions in their public teaching hospitals.

5 N Roxon, op cit.

6 Department of Health, Specialist Training Program Operational Framework, January 2013. The objectives have remained unchanged since the consolidation of programs into the STP in 2009.

7 The STP does not have specific enabling legislation.

8 Unless otherwise specified, all figures in this audit are goods and service tax (GST) exclusive.

9 The allowance is intended to recognise the time and effort involved in a qualified specialist supervising a trainee.

10 Traditionally, a ‘Fellow’ is the most senior grade of membership of most professional or learned societies.

11 The colleges contract separately with each setting.

12 In this audit, the term ‘state health services’ includes state and territory health departments and the regional health organisations that form part of the state and territory public health system within most of these jurisdictions.

13 Vacancies in STP training positions have occurred as some funded settings have subsequently withdrawn from the program or have had periodic difficulties in attracting or retaining specialist trainees.

14 Department of Health, Assessment Report for the 2013 funding round, 4 July 2012.

15 Health advised the ANAO that it considered that the possibility of such geographical ‘balancing’ had been covered in the application material for the 2014 round, which stated that funding decisions will be ‘made on a complex range of considerations, including the endorsement of a position by both the jurisdiction and the relevant College, as well as its capacity to meet the priorities for the round.’ However, this wording did not explicitly advise potential applicants that Health would apply an additional population-based criterion as part of a final internal review process.

16 The procedure was not referenced in the application form or other explanatory material made available to applicants, and implementation of the procedure lacked appropriate record-keeping and quality control processes.

17 While the scores applying to individual applications were calculated by Health, these scores were based on the relevant state health service and college assessments.

18 Applications placed on the reserve list could potentially be offered funding at a later stage if other STP training positions could not be filled by a specialist trainee for some reason or the setting withdrew from the STP.

19 A separate application was required for each proposed training position.

20 Separate from the review and rescoring process, there were also three applications that were funded in preference to other applications that scored slightly higher and Health was unable to provide any records documenting the reasons for this.

21 Health sought legal advice on this issue during this audit, after the matter was raised by the ANAO. The department’s advice indicated that in the period prior to 1 July 2014 there are arguments either way as to whether the STP contracts involved third parties handling public money under the Financial Management and Accountability Act 1997 (FMA Act), which was then in operation. The advice concluded that on balance, 12 of the 13 contracts did not involve colleges handling public money, and college personnel were therefore not considered to be allocated officials of the department under the FMA Act. Had they been allocated officials, the FMA Act provisions relating to the handling of public money would have applied.

22 Australian National Audit Office and Department of the Prime Minister and Cabinet, Successful Implementation of Policy Initiatives—Better Practice Guide, Canberra, October 2014, p. 31.

23 Health does not receive reports directly from the training settings.

24 This figure includes the Australasian College for Emergency Medicine in respect of its implementation of the Emergency Department Workforce (Doctors and Nurses) initiative, but not the Royal Australasian College of Medical Administrators in its capacity as administrator of Private Infrastructure and Clinical Supervision funding (which has a separate reporting schedule), or the Specialist Training in the Tasmanian public health system initiative. Background on the Emergency Department Workforce and Tasmanian initiatives is outlined in paragraphs 1.23–1.28.

25 Medicare is Australia’s universal healthcare system which provides people with access to free or subsidised health and hospital care.

26 In some literature, general practice is also treated as a discrete medical specialty. However, for the purpose of this audit, general practice falls outside the meaning of medical specialty.

27 A full list of colleges is available from [internet] <http://www.medicalboard.gov.au/News/Useful-Contacts.aspx> [accessed December 2014].

28 STP training positions are also sometimes called ‘training posts’. For the purposes of this audit, the term ‘training positions’ is used, except when quoting from documents that explicitly use the term ‘training posts’.

29 The accreditation approach varies between colleges. Notably, some colleges accredit specific training positions whilst others accredit a hospital or other facility rather than a specific position within it. Specialist trainee positions do not necessarily have to be accredited, but employment in unaccredited positions may not count towards completion of specialist training.

30 For example, the training period to qualify for Fellowship of the Royal Australasian College of Medical Administrators is normally three years; for the Royal Australasian College of Surgeons, it is up to six years. Overseas trained specialists that are assessed as having comparable qualifications may become Fellows without going through the relevant training program.

31 Traditionally, a ‘Fellow’ is the most senior grade of membership of most professional or learned societies.

32 Pathology, for instance, includes the disciplines of anatomical pathology (tissue diagnosis of disease) haematology (diseases which affect the blood) and microbiology (diseases caused by infectious agents).

33 Notably general physicians, general surgeons and general pathologists.

34 Figures derived by the ANAO from the 13th (2010) and 17th (2014) annual reports of the Medical Training Review Panel [internet] available from <http://www.health.gov.au/internet/main/ publishing.nsf/Content/work-pubs-mtrp> [accessed August 2014]. The MTRP is a statutory body with membership including representatives from all state and territory health services, specialist medical colleges, medical schools, the Australian Medical Council, and a range of other professional, industry and stakeholder bodies.

35 Figures are all specialist trainees—not just STP—excluding general practice specialist trainees.

36 Medical Training Review Panel, op. cit.

37 Australian Bureau of Statistics, Australian Demographic Statistics 3101.0 [internet], December quarter 2008, ABS, available from <http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/3101.0Dec %202008?OpenDocument>, and December quarter 2013, ABS, available from <http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/3101.0Dec%202013?OpenDocument> [accessed August 2014].

38 The Advanced Specialist Training Posts in Rural Areas program.

39 The Psychiatry Training Outside Teaching Hospitals program.

40 The Overseas Trained Specialists Upskilling program.

41 Australian Government, Portfolio Budget Statement 2009–10: Budget related paper 1.10 [internet], Health and Ageing portfolio, p. 314 [internet] available from <http://www.health.gov.au/internet/budget/ publishing.nsf/Content/2009-2010_Health_PBS_sup1/$File/Department%20of%20Health%20and%20 Ageing%20PBS.pdf> [accessed August 2014].

42 N Roxon (Minister for Health and Ageing) ‘Training Record Numbers of Specialist Doctors’, media release, Canberra, 15 March 2010.

43 This committee, which reported to the Australian Health Ministers’ Conference, operated from 1995 to 2006 with a mandate to assist with the development of a more strategic focus on medical workforce planning in Australia and to advise on national medical workforce matters, including workforce supply, distribution and future requirements. See [internet] <http://www.health.gov.au/internet/publications/ publishing.nsf/Content/work-res-ruraud-toc~work-res-ruraud-lis~work-res-ruraud-lis-b~work-res-ruraud-lis-b-4> [accessed September 2014].

44 In this audit, the term ‘state health services’ includes state and territory health departments and the regional health organisations that form part of the state and territory public health system within most of these jurisdictions.

45 N Roxon, op cit.

46 Department of Health, Specialist Training Program Operational Framework, January 2013. The objectives have remained unchanged since the consolidation of programs into the STP in 2009.

47 ibid.

48 ANAO interviews with stakeholders indicate that the majority of specialist trainees training in STP private sector settings tend to be seconded from the state and territory public health sector, particularly metropolitan and regional hospitals and health networks. In such cases, as a condition of releasing the specialist trainee for the secondment, the relevant hospital or network will invoice the private sector setting for the period that a specialist trainee is in the STP position. The funds received by the hospital or network from the private sector setting can then be used to ‘backfill’ the absent specialist trainee, allowing the level of health services at the public hospital or network to be maintained.

49Specialist Training Program Operational Framework, January 2013.

50 Although as noted in paragraph 1.16, the relevant colleges also have a significant role in the delivery of the STP.

51 Membership of MTRP includes representatives from all state and territory health services, specialist medical colleges, medical schools, the Australian Medical Council, and a range of other professional, industry and stakeholder bodies. Previously, the advisory role was undertaken by the Enhanced Medical Education Advisory Committee (EMEAC), which had a broadly similar membership but was focussed on the STP. EMEAC ceased operation in 2012.

52 The forum is an annual meeting of representatives from the colleges involved in the administration of the STP. A representative from Health also attends the forum.

53 The STP does not have specific enabling legislation. In the context of the Australian Government’s response to the High Court decision of 20 June 2012 in Williams v Commonwealth [No.1], the STP falls within the scope of item 415.035 (‘Workforce and rural distribution’) of Schedule 1AA, Part 4 of the Financial Framework (Supplementary Powers) Regulations 1997. Following the High Court decision of 19 June 2014 in Williams v Commonwealth [No.2], Health advised the ANAO that it had sought advice from the Australian Government Solicitor on the decision’s implications for the STP.

54 Unless otherwise specified, all figures in this audit are goods and service tax (GST) exclusive.

55 A training position that is not filled on an FTE basis during the year receives a pro-rata amount.

56 To qualify for this $20 000 ‘rural loading’, positions must be in locations which fall within an Australian Standard Geographical Classification – Remoteness Area (ASGC-RA) 2 to 5. This includes all areas except ‘major cities of Australia’ See http://www.phcris.org.au/fastfacts/fact.php?id=8290. Hobart and Darwin are classified as RA 2 and RA 3 respectively and thus qualify for rural loadings.

57 The allowance can be spent on medical training equipment, on-line educational training software, video conferencing facilities and other office equipment used by specialist trainees or their supervisors.

58 Cost-shifting involves an entity using funding received from an outside source (through a grant for example) to cover the costs of an activity that the entity would otherwise fund itself. Australian Government grants policy requires agencies designing and administering grants to put in place procedures to minimise opportunities for cost-shifting: Commonwealth Grant Guidelines, 2009, p. 35; Commonwealth Grant Guidelines, 2013, paragraph 11.4; and Commonwealth Grants Rules and Guidelines, 2014, paragraph 11.4.

59 However, once a setting is successfully selected for an STP grant, it does not have to reapply in subsequent rounds (see paragraph 1.18) in order to maintain its funding. The majority of current STP training positions have been in continuous receipt of STP grant funding for several years.

60 In addition, applications for positions that have been funded any time in the last 12 months must show that this was non-ongoing (temporary) funding.

61 This administrative funding is separate from the grant funding elements outlined in paragraphs 1.13–1.14 that colleges disburse to the settings.

62 Colleges also receive funding from Health to develop and implement strategic support projects referred to in paragraph 1.16.

63 See paragraph 1.6.

64 As previously noted, once a setting is successfully selected for an STP grant, it does not have to reapply in subsequent rounds in order to maintain its funding.

65 The annual funding rounds are named according to the calendar year in which positions that receive funding will commence operating. Thus new positions funded from the 2014 round commenced in January 2014 (unless the relevant setting could not recruit a suitable specialist trainee to fill the position).

66 Measured on a FTE basis.

67 Most recruitment activity for specialist trainee positions commences around August, with specialist trainees starting in new positions in the following January or February. The 2014 funding round process could not be completed by the department before the caretaker period took effect on 5 August 2013. With the change of government in September 2013, completion of the round (and hence confirmation of funding) did not occur until December 2013. As a consequence, some successful applicants from the 2014 round were unable to recruit specialist trainees for the first half of 2014 but most expected to have a specialist trainee in place for the second half.

68 This issue is discussed in Chapter 3.

69 See discussion at paragraph 3.43.

70 The STP has a significant emphasis on funding training positions in regional and rural areas. Under the Australian Standard Geographical Classification – Remoteness Area system, Darwin is categorised as a regional area, which partly explains the number of training positions located in the Northern Territory. The STP guidelines do not make any reference to equity of distribution between states and territories as being a factor in selecting STP training positions.

71 A fifth application round for emergency medicine positions was undertaken in calendar 2014 by the Australasian College for Emergency Medicine (ACEM). This was managed by the ACEM rather than Health as there was no STP round for that year.

72 The 87 training positions include five positions funded through underspends of Emergency Department Workforce grant funding in 2012 and 2013.

73 Two of these training positions are actually funded through the STP, however these are administered alongside the Emergency Department Workforce training positions under the same funding agreement.

74 In particular, the ACEM is not required to report on the key performance indicators (KPIs) that were introduced for STP progress reporting by colleges from 2014.

75 More details are contained in: Department of Health, Tasmania Health Assistance Package—Key Facts and Figures.

76 Expert Advisory Group, The Tasmanian Hospital System: Reforms for the 21st Century [internet] available at <http://www.dhhs.tas.gov.au/__data/assets/pdf_file/0004/8563/2004-06-Richardson-Report-v2.pdf> [accessed August 2014].

77 The Commission on Delivery of Health Services in Tasmania, Preliminary Report to the Australian Government and Tasmanian Government Health Ministers, 21 December 2012.

78 The Tasmanian project also funds a small number of coordinator and support positions.

79 In contrast, STP funding does not generally cover the full cost of employing a specialist trainee.

80 Australian Government, Portfolio Budget Statement 2011–12: Budget related paper 1.10 [internet], Health and Ageing portfolio, p. 47 [internet] available from <http://www.health.gov.au/internet/budget/ publishing.nsf/Content/70A1872C465D6D96CA257CA0003FF528/$File/2011-12_Health_PBS_05_ StrategicReview.pdf> [accessed January 2015].

81 Department of Health and Ageing, Annual Report 2011-12.

82 Department of Health, Flexible Fund Guidelines: Health Workforce Fund, January 2014 [internet] available from <http://www.health.gov.au/internet/main/publishing.nsf/Content/ 80B6DECBAF9FFAA3CA257BF0001C95A2/$File/Health%20Workforce%20flexible%20Fund%20Guidelines%20January%202014.pdf> [accessed September 2014].

83 The Grant Services Division was established in October 2013. A key rationale behind its formation was to bring greater consistency to all aspects of departmental grants administration.

84Review of Australian Government Health Workforce Programs, 2013 [internet] available from <http://www.health.gov.au/internet/main/publishing.nsf/Content/D26858F4B68834EACA257BF0001A8DDC/$File/Review%20of%20Health%20Workforce%20programs.pdf> [accessed August 2014].

85 ibid., p. 95.

86 Health Workforce Australia, Health Workforce 2025, Volume 3, p. 2. [internet], available from <https://www.hwa.gov.au/sites/uploads/HW2025_V3_FinalReport20121109.pdf> [accessed March 2014].

87 See Standing Ministerial Council on Health, Communique, 9 November 2012 [internet], available from <http://www.ahmac.gov.au/site/media_releases.aspx> [accessed August 2014].

88 National Medical Training Advisory Network, Concept of Operations: Implementation plan for the Network, December 2013 [internet] available from <https://www.hwa.gov.au/sites/uploads/HWA%20 MNTAN%20concept%20of%20operations_LR.pdf> [accessed August 2014]. Health advised the ANAO that ongoing Health Workforce Australia functions were transferred to the department in August 2014.

89 Australian National Audit Office, Implementing Better Practice Grants Administration, June 2010. A 2nd edition was published in December 2013.

90 Department of Finance, Commonwealth Grant Guidelines, 2009, paragraph 3.24. This version of the guidelines applied at the commencement of the 2014 funding round in March 2013.

91 The change followed the incorporation of the STP into the then Department of Health and Ageing’s Health Workforce Fund in 2011. This change was part of a wider reform by Health whereby 159 departmental programs were consolidated into 18 broader–based funding pools called ‘flexible funds’. The Health Workforce Fund’s primary objective is to strengthen the capacity of the health workforce to deliver high quality care. Department of Health, Flexible Fund Guidelines, Health Workforce Fund, January 2014 [internet] available from <http://www.health.gov.au/internet/main/publishing.nsf/Content/ 80B6DECBAF9FFAA3CA257BF0001C95A2/$File/Health%20Workforce%20flexible%20Fund%20Guidelines%20January%202014.pdf> [accessed September 2014].

92 In particular, the question of which medical specialties should be prioritised for STP funding. See Health Workforce Australia, Health Workforce 2025 —Volume 3 Medical Specialties, 2012, particularly pp 8-9 and 15 [internet] available from <https://www.hwa.gov.au/sites/default/files/HW2025_V3_ FinalReport20121109_0.pdf> [accessed August 2014]. Health Workforce Australia ceased operation in August 2014, with its functions being transferred to the Department of Health.

93 For example, general surgery, general paediatrics, general obstetrics and gynaecology, and geriatric medicine.

94 This combines general medicine and an additional specialty.

95 That is, the potential to complete the entire training program to enable the specialist trainee to become fully qualified as a specialist.

96 At that time called the Program Funding and Procurement Service (PFPS).

97 Some stakeholders commented that making arrangements to get a position accredited was a significant task. However, applications only had to demonstrate ‘progress’ towards accreditation of the proposed training position, with accreditation only required by the time the specialist trainee took up the position.

98 See paragraphs 1.23-1.24 in Chapter 1 for background on the Emergency Department Workforce initiative.

99 See Towards Responsible Government: The Report of the National Commission of Audit, Phase One, February 2014 p. 242 [internet], available from <http://www.ncoa.gov.au/report/docs/phase_one_report.pdf> [accessed August 2014].

100 At the time that FOFMS was introduced in 2005, the Department of Social Services was called the Department of Family and Community Services (FaCS) and subsequently the Department of Families, Housing, Community Services and Indigenous Affairs (FaHSCIA).

101 Under these arrangements, the main FOFMS information technology (IT) framework remains hosted by Social Services, but Health staff are able to access relevant elements of FOFMS through their IT network. A number of other departments also use FOFMS.

102 Health advised the ANAO that the development cost of the smartform was $12 400.

103 Separate applications were required for each proposed training position. Some settings submitted multiple applications as they were seeking funding for a number of training positions, sometimes within the same or related specialties.

104 See Australian National Audit Office, Implementing Better Practice Grants Administration, June 2010, p. 72; Implementing Better Practice Grants Administration, December 2013, p. 56, [internet] available from <http://www.anao.gov.au/Publications/Better-Practice-Guides?topic=68E513ABC79546BB82B4B2593 E46D19C> [accessed August 2014].

105Commonwealth Grant Guidelines, 2009, p. 29; Commonwealth Grant Guidelines, 2013, paragraph 13.9.

106 Following advice from its internal grants advisory area, Health decided to deem all applications as compliant with the mandatory requirements in order to satisfy the ‘department’s obligations to reasonable and fair access for an applicant to submit an application’.

107 In terms of the assignment of global ratings to applications, ANAO discussions with state health services indicated that some state health services emphasised workforce shortage considerations in assigning a global rating, with the Priority Framework playing only a subsidiary role in this decision-making process. Funding an STP position in locations where there was a workforce shortage in that specialty would directly benefit the provision of relevant health services in that location. The concept of educational merit included how the position would enable specialist trainees to acquire appropriate skills and experience, including through appropriate exposure to practices commonly undertaken outside major public teaching hospitals and/or which could not be optimally provided to trainees in major public teaching hospitals.

108 For example, an application for a training position in surgery at a Wollongong hospital would be provided both to the Royal Australasian College of Surgeons and the NSW Department of Health.

109 Most assessment responses were generally returned using the provided spreadsheet, although in some instances, particularly with the smaller state health services or colleges, the responses were in the form of the completed templates.

110 Within colleges, Fellows or other college staff were generally required to declare any conflicts of interests relating to applications and not participate in the assessment of those applications. In respect of the state health services, most said that assessments were done by an area of the relevant Health department that was organisationally (and sometimes legally) separate from the regionally or locally-based health networks or providers that generally submitted applications.

111Commonwealth Grant Guidelines, 2009 p. 33; Commonwealth Grant Guidelines, 2013, paragraphs 13.6-13.8; Commonwealth Grants Rules and Guidelines, 2014, paragraphs 13.6-13.8.

112 ANAO Audit Report No.47 2013–14 Managing Conflicts of Interest in FMA Agencies, [internet],, available from <http://www.anao.gov.au/Publications/Audit-Reports/2013-2014/Managing-Conflicts-of-Interest-in-FMA-Agencies> [accessed August 2014].

113 ibid., pp 61–62.

114 Assessment responses for 18 applications were excluded from the analysis because it was not possible to interpret the meaning of one or more aspects of the college or state health service assessment response due to the lack of clear information in the relevant section of the response provided to Health.

115 Health advised the ANAO that it considered that it was the department’s responsibility to assign scores and it was correct to undertake this review and rescoring process and this was not a task for the involvement of the colleges and state health services—rather that the key role of the colleges and state health services in the assessment process was assigning an overall ‘global rating’ for each application. The assessment plan, while making it clear that the selection of successful applicants was a decision for Health, was silent on the issue of what would occur if there was a difference of opinion between the relevant college and state health service on the extent to which an application met the various priorities and preferences contained in the Priority Framework.

116 In such cases, applications might be placed on either a ‘reserve’ list or an ‘unsuccessful’ list. Those placed on the reserve list could potentially be offered funding at a later stage if other STP training positions could not be filled by a specialist trainee for some reason or the setting withdrew from the STP.

117 Table 2.2 excludes applications for Emergency Department Workforce training positions as these were subject to a separate ranking process which falls outside the scope of this audit.

118 A separate application was required for each proposed training position.

119 Department of Health, Assessment Report for the 2013 funding round, 4 July 2012.

120 See paragraph 2.19.

121 Health’s assessment report provided information that assessments were undertaken by the department ‘using further detail provided in each application as well as assessment information provided by the colleges and state health services’.

122 Further, the departmental assessment plan provided that the assessment report must ‘include details and a risk assessment of any significant issues arising from the assessment’. The lack of any records regarding the reasons for the changes made was also inconsistent with the requirements of the assessment plan that ‘all assessor meetings should be minuted and filed … as part of the audit trail for the assessment’.

123 The department engaged external contractors in August 2014 to rescore applications at a cost of $43 000.

124 As noted in paragraph 2.31, these calculations were based on the college and state health services assessment responses.

125 The ANAO undertook an electoral analysis of a number of applications, including the 13 applications discussed in this paragraph. The results of the analysis are in Appendix 2.

126 A separate application was required for each proposed training position.

127 Department of Health, Assessment Report for the 2013 funding round, 4 July 2012.

128 Health advised the ANAO it considered that the possibility of such geographical ‘balancing’ had been covered by the statement in the application material that funding decisions will be ‘made on a complex range of considerations, including the endorsement of a position by both the jurisdiction and the relevant college, as well as its capacity to meet the priorities for the round.’ However, this wording did not explicitly advise potential applicants that Health would apply an additional population-based criterion as part of its final assessment process.

129 The respective proportional outcomes in Figure 2.2 are similar to the equivalent figure for the entire 900 training positions (Figure 1.2 in Chapter 1), although the 2014 outcomes are somewhat higher for the Northern Territory and lower for Tasmania. However, the latter is more than offset by the specialist trainee and specialist supervisor positions due to be funded by the Commonwealth under the Tasmanian project from 2013–14 to 2015–16 (see paragraphs 1.25–1.28).

130 An application could involve training in both a priority specialty and relevant generalist elements.

131 The minute was noted by the Minister on 5 August 2013.

132 Approvals under FMA Regulation 10 for funding to 2015–16 had been provided in December 2012 to allow for negotiation of multi-year funding agreements variations under the 2013 funding round. The then Health Minister gave policy approval in March 2012 for the commitment of funds out to 2015–16. Regulation 10 required that the Finance Minister or a delegate give written agreement to an arrangement, such as a grant, where there was insufficient available appropriation for expenditure that might become payable under the arrangement. This was typically the case for multi–year arrangements, such as grants paid over a number of years.

133 FMA Regulation 9 required a decision–maker to be satisfied, after making reasonable enquiries, that giving effect to the relevant spending proposal would be a proper use of Commonwealth resources. Subsection 44(3) of the FMA Act defined proper use to mean efficient, effective, economical and ethical use, not inconsistent with the policies of the Government.

134 The FaHCSIA Online Funding Management System (FOFMS) administered by the Department of Social Services is gradually being adopted by Health and a number of other Australian Government entities for grants administration.

135 Those placed on the reserve list could potentially be offered funding at a later stage if other STP training positions could not be filled by a specialist trainee for some reason or the setting withdrew from the STP.

136 A separate application was required for each proposed training position.

137 Health advised the ANAO that it considered that the possibility of such geographical ‘balancing’ had been covered in the application material, which stated that funding decisions will be ‘made on a complex range of considerations, including the endorsement of a position by both the state health services and the relevant college, as well as its capacity to meet the priorities for the round.’ However, this wording did not explicitly advise potential applicants that Health would apply an additional population-based criterion as part of its final assessment process.

138 Separate from the review and rescoring process, there were three applications that were funded in preference to other applications that scored slightly higher and Health was unable provide any records documenting the reasons for this–see paragraph 2.38.

139 In this chapter, unless otherwise specified, ‘STP funds’ include those payable under the 2010 Emergency Department Workforce initiative and the 2012 Tasmanian project.

140 There are two separate agreements between Health and the Royal Australasian College of Medical Administrators (RACMA). The first is similar to the other college funding agreements regarding the STP training positions administered by it. The other is a unique agreement dealing with the administration of the Private Infrastructure and Clinical Supervision (PICS) allowance.

141 This was largely due to the continuation of administrative arrangements that existed for some programs before they were consolidated into the STP.

142 Health currently provides a small number of grants direct to settings under the Emergency Department Workforce initiative. However, these represent less than five per cent of total grant funding under this initiative.

143 The hospital, facility or organisation hosting an accredited training position is known as a ‘setting’.

144 Each setting signs a contract with the college before the setting receives its initial payment of grant funds from the college.

145 These elements are: salary support, rural loading, private infrastructure and clinical supervision (all of which are on-paid to settings), administration and governance and strategic support projects (these are funds for the college’s own use).

146 The department largely relies on the aggregated reporting it receives from the colleges to assess the implementation and performance of the STP—it does not receive reports directly from the training settings.

147 As of the end of 2014, STP training positions are only funded to December 2015, with final progress payments to colleges due to be paid by Health in February 2016. Training positions funded under the Tasmanian Project are funded to June 2016.

148 This figure includes funding provided to the Royal Australasian College of Medical Administrators to administer PICs, and to the Australasian College for Emergency Medicine to administer the Emergency Department Workforce specialist trainee positions.

149 Prior to 1 July 2014, the primary legislation establishing resource management requirements for departments was the Financial Management and Accountability Act 1997 (the FMA Act). From 1 July 2014, the FMA Act was replaced by the Public Governance, Performance and Accountability Act 2013 (the PGPA Act). Fundamentally both the FMA Act and the PGPA Act require Commonwealth entities to make proper use of public resources.

150 The FMA Act made provision for ‘outsiders’ to undertake tasks related to the handling of public money under authorised section 12 agreements, which could be used to establish a framework for the handling of public money. An outsider was defined as ‘any person other than the Commonwealth, an official or a Minister’. An outsider who performed financial tasks in relation to public money and who was not doing so under an authorised section 12 agreement was deemed to be an ‘allocated official’ of the relevant agency in relation to the tasks undertaken, and was therefore subject to all relevant provisions of the FMA Act and FMA Regulations. See Financial Circular 2011/01 Commitments to spend public money (FMA Regulations 7 to 12), pp. 40–42.

151 Departmental records indicate that in 2011 Health considered the implications of section 12 in relation to the Royal Australasian College of Medical Administrators, Private Infrastructure and Clinical Supervision funding agreement. The department provided the ANAO with a document indicating that it consulted with the office of Health’s Chief Financial Officer (CFO). The CFO’s office advised that the determination of whether section 12 applied was primarily a legal question, and did not provide further advice on this issue. There is no indication that the department subsequently sought legal advice on the Royal Australasian College of Medical Administrators, Private Infrastructure and Clinical Supervision agreement.

152 See Department of Finance, Other CRF Money, Resource Management Guide No. 303 [internet], July 2014, paragraph 23, available from < http://www.finance.gov.au/sites/default/files/rmg-303-other-CRF-money.pdf> [accessed October 2014]. Section 16 of the PGPA Act also requires Commonwealth departments to ‘establish and maintain an appropriate system of risk oversight and management’ in relation to its operations.

153 ANAO Best Practice Guide, Successful Implementation of Policy Initiatives - October 2014, p. 31.

154 The Royal Australasian College of Medical Administrators also receives specific funding for on-payment to private sector settings.

155 Training positions filled less than full-time receive a pro-rata amount.

156 While the program currently funds STP training positions to the end of calendar year 2015, some funding will continue to flow until mid-2016 for some colleges to cover the STP related expenses.

157 The ANAO identified an administrative error in the recording of the number of training positions to receive rural loading in 2014. Health has confirmed the error and advised that it is liaising with the relevant college to address the issue.

158 In one case a college had surplus funds at the end of a reporting period (December 31, 2013). This resulted in a reduction of salary support funding, as the college had sufficient funds to cover its salary support payments.

159 Based on costing supplied for 2013 for the Tasmanian project (see paragraphs 1.25-1.28) total employment costs (base salary, superannuation, ‘on-call’ and overtime allowances (where applicable) and other minor costs) ranged from around $120 000 to $180 000 per annum, depending on the seniority of the specialist trainee and the specialty in which they are training.

160 The 2013 Mason Review recommended that the amount be indexed for ‘future [STP] posts’. As previously noted, posts refer to training positions. Health advised the ANAO that the current STP is to cease at the end of 2015 and no new funding rounds are planned. Health further advised that in considering possible options for the future of the STP beyond 2015 it will consider the issue of indexing the financial support for each position. As at January 2015, the Government has not made a decision on future STP funding.

161 Some private sector settings also indicated, in discussions with the ANAO, that some public hospitals charged an additional administrative or management fee between five and 12 per cent above the salary component for the specialist trainee.

162 From discussions with stakeholders the ANAO found that there are a range of different circumstances that exist as a result of individual contract arrangements between the various public and private settings involved in the STP, therefore the ANAO has simplified the basic concept of public-private funding flow in Figure 3.2.

163 Classified under the Australian Standard Geographic Classification – Remoteness Areas (ASGC–RA) as RA2, 3, 4 or 5.

164 This figure includes rural loading funding provided to the Australasian College for Emergency Medicine for Emergency Department Workforce specialist trainee positions.

165 Health does not undertake a separate process to verify the location of positions with respect to its ASGC–RA status. However, the issue of whether a position is in a rural or regional area is considered as part of the initial assessment of grant applications by the colleges and state health services.

166 These rotations affect the eligibility of a setting to receive a pro-rata amount of rural loading.

167Commonwealth Grant Guidelines, 2013, pp. 16–7.

168 The 2014 round deeds of variation were executed under the FMA Act financial framework.

169Commonwealth Grant Guidelines, 2013, p. 27.

170 ibid.

171 Department of Finance and Deregulation, Finance Circular 2013/02, Australian Government Grants: Briefing and Reporting, p14.

172 Health also advised that the design of the system had taken into consideration work by the Department of Finance to develop grants.gov.au, a proposed whole-of-government grant advertising, lodgement and reporting system.

173 The Australasian College for Emergency Medicine reports on the funded training positions under the same arrangements as the other colleges, but it is not required to report against STP key performance indicators. The Royal Australasian College of Medical Administrators also provides a separate report on its administration of PICs.

174 The Royal Australasian College of Medical Administrators reporting is due in April and October each year.

175 Required reporting includes the provision of income and expenditure statements for the last six months, projected budgets for the next six months, copies of bank statements, and a declaration by the Chief Executive Officer or Chief Financial Officer of the college regarding the proper expenditure of funds. A statement from an external auditor on the college STP accounts must also be provided once a year.

176 ‘Proportionality’ has been one of seven key principles for grants administration since the introduction of the grants framework in July 2009.

177 Department of Finance, Commonwealth Grant Guidelines, 2013, p. 56.

178 A more detailed discussion of KPIs is contained in Chapter 4.

179 In the course of the audit, Health advised the ANAO that ‘in light of workload [issues], including new Government Budget measures and the ANAO performance audit, a review of the KPIs has not yet occurred. It is our intention to review the KPIs and related college reports during 2016.’

180 This includes the Australasian College for Emergency Medicine in respect of its implementation of the Emergency Department Workforce project, but not the Royal Australasian College of Medical Administrators in its capacity as administrator of PICS funding (which has a separate reporting schedule) or the Tasmanian project.

181 The figure of 16.4 per cent excludes PICS funding.

182 For instance, delays can arise in the submission of invoices by the settings to the colleges.

183 $19.89 million of this amount were STP funds, with the other $4 million relating to the Emergency Department Workforce funds.

184 A number of Health’s assessments of college Tasmanian project reports noted the relative lack of expended funds and that the option for withholding part payment of the next scheduled payment (due in December 2014) would be considered at that time.

185 This discrepancy was subsequently resolved through a revised report submitted by the relevant college, but this was after the department had made the progress payment based on the original report.

186 Department of Finance, Commonwealth Grants Rules and Guidelines, 2014, p. 25.

187 Australian National Audit Office, Implementing Better Practice Grants Administration, December 2013, p. 16.

188 ANAO Audit Report No.5 2011–12, Development and Implementation of Key Performance Indicators to Support the Outcomes and Programs Framework, September 2011, p. 13.

189 ‘Mason Review’, unpublished working papers.

190 The Australasian College for Emergency Medicine, which administers training positions funded through the Emergency Medicine initiative, is not required to report against the KPIs as part of its periodic reporting obligations.

191 In this table Health refers to training positions as posts.

192 As at the conclusion of audit fieldwork in late October 2014.

193 At the February 2014 inter-college forum, Health agreed to review some of the KPIs and associated reporting. During the course of the audit, Health advised the ANAO that ‘in light of workload [issues], including new Government Budget measures and the ANAO performance audit, a review of the KPIs has not yet occurred. It is our intention to review the KPIs and related specialist medical college reports during 2016.’

194 Percentages for individual colleges ranged from 57 per cent to 100 per cent.

195 The objective of these projects is to strengthen training networks involving STP training positions, with a particular focus on rural and regional training.

196 Settings advised the ANAO that STP specialist trainees provide significant services to patients, albeit under the general direction of qualified specialists.

197 Percentages for individual colleges ranged from zero to 100 per cent.

198 Health Workforce Australia, Health Workforce 2025, 2012, <https://www.hwa.gov.au/our-work/build-capacity/australias-future-health-workforce-hw2025>. The report contains detailed modelling on workforce supply, demand and training and projects the numbers required between 2012 and 2025 for professional entry students and postgraduate and specialist training.

199 The department advised that the National Medical Training Advisory Network is developing medical training plans, including in relation to specialists, to inform government, health and education sectors. Health expects the first specialist training plan (for psychiatrists) to be provided to Health Ministers for consideration around February or March 2015.

200 Health refers to training positions as posts.

201Review of Australian Government Health Workforce Programs, op. cit., recommendation 3.10.

202 T Plibersek (Minister for Health), ‘Independent Review of Health Workforce Programs Released’, Canberra, 24 May 2013.

203 A number of colleges also noted at the inter–college forum that they also undertook a range of monitoring activities in respect of STP training positions, including through trainee and supervisor surveys.

204Review of Australian Government Health Workforce Programs, 2013, op. cit.

205 T Plibersek (Minister for Health), ‘Independent Review of Health Workforce Programs Released’, Canberra, 24 May 2013.