report

Delivering Western Australia’s Ambulance Services – Follow-up Audit

Audit finding – WA’s ambulance service is more efficient, but the DoH has made limited progress improving the contract and measuring performance

SJA’s ambulance service has expanded since 2013 and become more efficient in order to meet rapidly increasing demand

Growth in the number of cases attended by SJA ambulances has exceeded the increase in resources deployed by SJA. SJA has managed this by becoming more efficient. It has continued to achieve targets for the time it takes to respond to emergencies despite the average number of cases per ambulance increasing by 27% from 552 in 2012-13 to 699 in 2017-18. However, SJA has not always been able to maintain response targets for lower priority cases.

According to its annual report, at 30 June 2018, SJA had 1,578 paid full time equivalent staff and 8,489 volunteers. In 2017-18, 522 ambulances across 30 metropolitan and 114 country sub-centres responded to 335,609 cases. This is an increase of 56 ambulances and 292 paid staff since 2013 to respond to an extra 83,442 cases. Around a third of cases in metropolitan areas and half those in country areas are emergencies assigned the highest priority.

This is a 12% increase in ambulance numbers and 23% increase in paid staff to handle a 33% increase in cases. Since 2014-15, case numbers have continued to rise while ambulance numbers have not to the same degree (Figure 1).

Source: OAG from SJA data

Figure 1: Number of ambulance in WA (left axis) compared with number of cases (right axis), 2012-13 to 2017-18

The Productivity Commission reported on 30 January 2019 that SJA’s ambulance service is the lowest cost per person in Australia. The report defines ambulance cost per person as total ambulance service organisation expenditure per person in the population. The Productivity Commission reports both the total cost of ambulance services and the cost to government of funding them, because patient/health fund revenue from transport fees is significant for a number of jurisdictions. The report warns, however, that the data should be interpreted with caution. Specifically, it notes that differences in geographic size, terrain, climate, and population dispersal may affect costs of infrastructure and numbers of service delivery locations per person.

The relatively low cost per person in part reflects use of volunteers by country ambulance services. WACHS and the DoH have highlighted the difficulty of comparing costs with other services given their geographical and organisational differences. WACHS has also expressed concern about the adequacy of country services given their reliance on volunteers and resulting lack of guarantees about service levels in some country areas. WACHS has sought to address this concern in its September 2018 Country Ambulance Strategy public consultation document and is currently considering feedback.

Source: OAG

Figure 2: SJA WA is contracted by the DoH to provide ambulances for service delivery throughout WA

The DoH has made limited progress in implementing a new funding model and a contract focused on standards, performance and risk

The DoH and SJA agreed on a new contract for 2015-18 that improved on the contract we reviewed in 2013. This contract improved on the previous one by including standard terms and conditions used in WA public sector procurement but did not address some of our key concerns. Our 2013 audit recommended developing a new funding model focusing on standards, performance and risk but changes to the funding model in the new contract were minor. SJA is motivated to perform well by its mission and culture but weaknesses in the contract cause problems for the DoH, which does not have a clear view of how its funding is being used and if it is getting value for money. The current contract was extended in 2018 to 2020 while a new contract is negotiated.

Funding under the contract has increased annually by a total of $7.6 million since 2012-13 based on projected population increases and indexation, reaching $100.8 million this year. The funding will increase by 3.65% next year based on a projected population increase of 2.12% and indexation of 1.53%. The increase will happen without any link to performance or transparency of how SJA applies the funds or cost-effectively meets demand for services. Without links between funding, demand for services and performance, there are no financial incentives in the contract for SJA to improve performance and no consequences for missing targets. Nor does the contract provide more funds to cope with the impact of ramping or increases in demand that may occur outside the forecast increases due to population growth. This means that SJA does not carry any risk to its funding if the ambulance service misses its targets.

For 2017-18, contract funding was $97.7 million, making up one-third of revenue from ambulance services. SJA’s revenue from fees paid by ambulance patients and health funds was $116.0 million that year. Revenue from IHPT was $23.6 million, with $12.0 million coming from country IHPT. Revenues from country IHPT, which is an SJA monopoly, helps to pay for country ambulance services. The DoH also subsidised transporting patients over 65 years old in the amount of $45.7 million.

While the contract includes key performance indicators (KPIs) and targets, they are separate from the rest of the contract and do not cover all services and service locations. KPIs focus on how quickly ambulances respond to a call, rather than outcomes such as the wellbeing of patients. Consequently, they measure activity but not its value. In addition, the contract does not require, and SJA has not adopted, financial reporting that would provide more transparency and allow assessment of value for money.

SJA uses statistical analysis of past performance to estimate how many ambulances it needs to maintain response times, which depend on the number of ambulances free to respond during busy periods and levels of surplus capacity in the system. The number of ambulances available to respond is being reduced by ramping, but SJA has been able to manage this while meeting its emergency response time targets.

The DoH has not independently assessed what resources SJA would need for a given level of service and what that would cost. Without this kind of insight into the funding model, the DoH cannot effectively assess if it is getting value for money and may not be alert to any reductions in ambulance reserve capacity that may be occurring over time but are not yet visible in response times.

Another concern for the DoH and WACHS is the funding model for IHPT. The contract gives SJA an IHPT monopoly in country WA and charges country hospitals for the service. It uses its income from this activity to subsidise its emergency response services in country areas. The extent of this cross-subsidy is not visible to the DoH or WACHS, making it hard for them to assess if it is an efficient way to fund country ambulance services.

The contract sets target times for ambulances reaching patients but they don’t apply everywhere and it is unclear how meaningful they are for less urgent cases

The time it takes for ambulances to reach patients is important for emergency cases but there is little clinical evidence to support the response time targets for less serious cases. Targets do not apply in country areas more than 10 kilometres from the town centres of the 15 country towns where career paramedics are based. Analysis of ambulance data together with patient data from hospitals could provide evidence for response time targets but this has not been done. In the absence of clinical evidence, response times may not be the most useful measure of paramedic or volunteer performance for ambulance activity that is not emergency response. However, there is no widely accepted alternative.

Ambulance response time is the main performance measure for ambulance services in Australia and internationally and the contract sets targets for this measure for SJA. The targets apply throughout the metropolitan service area and in country WA locations within 10 kilometres of the 15 major country towns where the ambulance sub-centre staff includes at least 1 career, that is paid, paramedic.

Outside these areas, ambulance crews are required to use their best endeavours but their performance is not assessed against targets. The DoH and WACHS need to determine what levels of service they require in country WA and work with SJA on the cost of delivering them. They also need to develop better indicators of country ambulance performance. The DoH informed us that it is preparing a state-wide policy on ambulance services to address these issues.

Country areas with performance targets, together with the metropolitan area, cover around 90% of the WA population. This leaves around 270,000 people not covered by response time targets. SJA data shows that responses to this population make up 6-7% of country ambulance responses. SJA uses ‘best endeavours’ to get to these people but response times can vary widely. Available data does not show if patient outcomes are worse overall for this group of people as might be expected because of longer response times caused by greater distances.

Response time is important for emergency cases such as out-of-hospital cardiac arrest where research shows it is critical for patient outcomes. However, there is little evidence that response time targets are a meaningful measure of ambulance performance in less serious cases. Our previous audit concluded such targets were arbitrary.

Analysis of ambulance data together with patient data from hospitals could provide evidence for response time targets. To do this, data security and integrity would need to be guaranteed and data from different sources, including clinical as well as response time data, would need to be linked. Response time targets are independently audited and reported to the DoH, but SJA does not currently share its clinical data because the contract does not require it.

Response time targets range from 15 minutes 90% of the time for emergencies in the metropolitan area to 60 minutes 90% of the time for non-urgent, pre-booked patient transfers in metropolitan and country locations (Appendix 2). SJA consistently meets its targets for Priority 1 cases (Figure 3). It also often misses targets for less urgent cases (Figure 4). Definitions of priority levels are at Appendix 3.

Source: OAG from SJA data

Figure 3: Monthly ambulance response times (solid line) against target (broken line) in the most urgent cases (Priority 1) May 2017 – May 2019                                             

 

Source: OAG from SJA data

Figure 4: Monthly ambulance response times (solid line) against target (broken line) in second most urgent (Priority 2) cases May 2017 – May 2019                       

SJA meets its contractual requirements to report to the DoH monthly, quarterly and annually. Response times are reported monthly and annually. Reporting also includes activity, training, aged pensioners and senior service, IHPT, emergency management capacity and complaints.

SJA has improved its clinical quality controls but the contract gives the DoH no visibility of cases that do not meet guidelines or ways to assess their significance

SJA has substantially improved its internal clinical quality controls and in 2017-18 achieved 95% against its target of 90% compliance with the clinical guidelines agreed with the DoH. However, how individual cases comply with guidelines is not independently audited and the contract does not give the DoH any visibility of cases that did not comply other than their total number. This contrasts with other jurisdictions where ambulance services are part of the government health system and a single authority holds the data. Patients can complain to the Health and Disability Services Complaints Office and the DoH has access to detailed clinical data for critical incidents. But its lack of broader access makes it harder for the DoH to assess the continuous improvement in patient care over time that is required by the contract.

SJA conducted 7,386 clinical audits in the paramedic sub-centres, and 11,580 clinical audits altogether in 2017-18 out of a total of 335,609 cases. This sample size should be sufficient to give a high level of confidence that it is representative. However, the audits are not independent of SJA and are also potentially skewed by audits being requested by paramedics and country volunteers.

With a target compliance rate of 90%, this system allows potentially 34,000 non-compliant cases a year without SJA missing the target. Actual performance is better than that at 95% in 2017-18, but this still implies around 17,000 non-compliant cases during the year. Higher levels of compliance or more information on non-compliant cases would give the DoH greater assurance that SJA is delivering good patient care.

The contract requires SJA to engage in continuous improvement and use evidence based data to review all policies and procedures. It must also assess its own performance against benchmarks for best practice. More generally, the contract requires SJA to aim for ‘optimal patient outcomes’ but these are undefined and KPIs do not deal with paramedic performance specifically.

The DoH and SJA collect patient data but neither has access to data covering the whole patient journey that they could use to improve services

SJA and the DoH keep patient records, and patient data is included in the clinical handover from ambulance to ED. However, SJA and the DoH do not combine their data to analyse all aspects of the patient journey from the call for an ambulance to when they leave hospital. This limits the ability of both the DoH and SJA to use patient-centred data to improve services.

SJA reports to the DoH include:

  • response times
  • ramping times
  • results of SJA’s clinical audits of paramedics and volunteers reported monthly as the percentage of cases that comply with clinical governance guidelines
  • events leading to adverse patient outcomes, including fatalities
  • aggregated results of patient satisfaction surveys
  • revenue from ambulance services as a quarterly list of patient invoices billed to the DoH.

SJA reports other revenues annually. These are aggregated and not detailed enough to show costs of specific activities.

Curtin University independently analyses some of this (anonymised) data for academic research purposes and publishes the results, but this is confined to specific topics of defined duration. However, it shows that analysis of linked up data is possible and potentially valuable.

SJA has a comprehensive complaints process but does not share this information with the DoH. SJA’s position is it is legally bound to protect the privacy of this information. This means the DoH does not have access to information that could improve feedback on SJA performance and assist with improving the patient journey.

Back to Top