The past five years have been the warmest in recorded history1 and the rise in global temperatures is exposing populations to record-breaking temperatures and more frequent heatwaves.2, 3 In Australia, the frequency of extreme heat events has increased approximately fivefold since the 1950s.4 This heat exposure can have severe health consequences, particularly among vulnerable groups such as the elderly, outdoor workers and those with pre-existing illness or socioeconomic disadvantage.5-7 Over the past decade, state-based heat health warning systems (HHWSs) have been developed to integrate health system preparedness and emergency responses, and to provide early public warnings, timely advice and targeted support for vulnerable groups.8

As HHWSs become more widely adopted there is an increasing focus on evaluating the effectiveness of these systems.9-11 Studies from Europe,10 the US,12 Korea13 and Australia14, 15 have reported that HHWSs can lead to reductions in adverse health outcomes during heat events. Conversely, studies in some locations have not shown health benefits.10 As HHWSs are operationalised in ‘real world’ settings, often with complex multi-sectoral collaborations, this presents challenges for evaluation.10 The absence of a counterfactual scenario, or what would have happened without the intervention, represents a key challenge.11, 16 Nonetheless, research is needed to assess the benefits of HHWSs, to identify factors that contribute to effectiveness and to provide evidence to direct future improvements.17

To date, there has been limited attention to HHWS costs in relation to potential health benefits. A case study in Philadelphia, US, suggested that the HHWS costs were small when compared with estimated lives saved.12 Similarly, Hunt et al.18 concluded that HHWSs are likely to be a cost-effective option to reduce heat-related risks in several European cities. A case study in Madrid also showed a positive benefit-cost ratio.19

The South Australian HHWS (SA HHWS) provides a useful case study for the evaluation of an Australian heatwave intervention.20 The system was developed after the extreme heatwave in Jan–Feb 2009 that resulted in significant mortality and morbidity across south eastern Australia.21, 22 The SA HHWS integrates public heat warnings, health advisories and targeted support for vulnerable groups including the elderly, those with mental health conditions and the homeless (see Box 1).23 Our previous studies have shown high public recall and positive views about heat warnings in SA,24 and a beneficial impact of the HHWS in relation to health outcomes.14 By comparing pre- and post-intervention heatwave events, we estimated a significant decrease in heat-related morbidity outcomes following the introduction of the HHWS, but no apparent reduction in mortality.14

In this study, we aim to extend the evaluation of the SA HHWS to consider the implementation costs in relation to the potential health cost benefits. The study provides the first descriptive cost benefit evaluation for an Australian heat-health intervention.

Methods Data collection

The SA HHWS is a multi-agency intervention led by the SA State Emergency Service (SASES) – the lead agency for extreme weather emergencies in the state. Other key agencies involved are listed in Table 1. We consulted with key informants in these agencies to obtain details about the types of activities, resources and costs associated with implementing the SA HHWS, with a focus on public health interventions. Potential informants were contacted by email to invite their contributions to the study. A snowball recruitment approach identified other relevant contacts. Information was obtained through interviews, written contributions and publicly available sources throughout the period December 2019 to December 2020. Interviews were conducted either face-to-face, or by phone or video conference. These were digitally recorded, or written notes were taken by the researcher. The informants were asked to describe their role within the agency; the role of the agency in implementing HHWS interventions; the types and level of resources required to fulfil these roles; and estimates for any direct costs associated with these activities. Written consent was provided from all interviewees.

Table 1. SA HHWS activities and estimated intervention costs for a seven-day activation scenario, by agency.

Agency

Pre-seasonal activities

Seasonal activities

HHWS Activation

Post-activation/end of season

Estimated resources/costs (AU$)a

Staff hours

Unit costs

Total costs

SA State Emergency Service (SES)

Planning, communications, printing brochures, social media, community engagement, consumables ($63,883)

IT Systems Monitoring ($71,500)

Support for the SEC including media briefings ($9207+$46,132=55,339)

190,722

Department of Health and Wellbeing (SA Health), including Mental Health (MH) Directorateb

Review of HHWS strategy / communications plan, data surveillance (n/a)

Production of public heat health resources ($5000)

Support for SEC by 24-hour on call duty officer (n/a)

5,000

SA Health MH Directorate

Review and update MH team procedures

Risk assessments for clients ($145,895)

Monitoring daily forecasts throughout summer ($1465)

Education and follow-up for clinicians ($36,506)

Calls to MH clients (Box1) ($8388); Staff communications & reporting ($26,040)

Internal reporting, liaison, and end of season debrief ($4071)

4,878

43.5 / 62

222,365

SA Department of Human Services / Red Cross

Program Management; Corporate costs, media, other ($22,000)

Telecross REDI calls to registered clients (Box 1)c ($67,621)

n/a

n/a

89,621

SA Housing Authority, Uniting Care and othersd

Co-ordination between agencies (n/a)

Code Red intervention for homeless population (Box 1) ($48,000)

n/a

n/a

48,000

Bureau of Meteorology (SA)

Liaise with state level services (2 people x 15 hours) ($2,700)

Activation of the national heatwave service and daily update by a senior forecaster (Oct to March) ($8,200)

Support to the SEC – typically 1 briefing ($100)

120

90

11,000c

SA Police

Activation of the SEC

210

115 / 153

26,430

TOTAL

AU593,000

Notes: a: Estimated resource use and unit costs provided where available, otherwise cumulative program costs are shown. Estimates for operational activities including a 1-week HHWS activation. b: Cost estimates do not include South Australian Ambulance Service c: SA Tenders and Contracts DCSI540 - Extreme Heat Response Service. d: Funds are distributed by Uniting Care to other non-government service providers providing homeless services n/a – costs not available Box 1: Targeted public health programs associated with the SA HHWS

SA Health heatwave intervention for mental health clients

The intervention provides support for SA Mental Health Directorate clients, who may be at higher risk due to the effects of psychotropic medications, behavioural factors, and/or limited resources or support. All mental health clients are assessed at the beginning of summer using a heat vulnerability assessment tool to identify those who may require closer monitoring during heatwaves. On activation of the SA HHWS, these clients receive daily welfare checks by phone or home-visits if needed.

The Telecross Redi program to support vulnerable and isolated people

The Telecross Redi program is a telephone support program conducted by the Red Cross and funded by the SA Department of Human Services. The program provides daily welfare check phone calls to registered vulnerable clients, principally the community dwelling elderly population. Individuals can self-register or are registered by family members, doctors, or support workers. In the event that an individual does not respond to a welfare call the case is escalated and SA Police will conduct a home welfare check.

https://www.redcross.org.au/get-help/community-services/telecross/telecross-redi

The Code Red program to support the homeless

The Code Red program is designed to provide those sleeping rough a place to stay cool and hydrated. The funding provided to this program supports various non-government organisations who provide regular services to the homeless, enabling greater respite options during heatwaves. It is supported financially by the SA Housing Authority.

https://www.housing.sa.gov.au/latest-news/code-red2

Examining HHWS intervention costs

Information for costs was collected according to the following inclusion criteria: (i) the activity was specific to the HHWS and distinct from other core activities of the agency, and (ii) the activity was undertaken systematically. While some ad-hoc activities may be undertaken during heatwaves, it was not possible to capture reliable information about the costs or effects. For the three targeted public health interventions (Box 1), we focused on the activities in the Adelaide metropolitan area where the majority of the SA population (76%) resides.25 This allowed for a direct comparison with the estimated health benefits for the Adelaide population.14 Information was based on 2019/20 activities and costs and was collated across the different agencies to estimate an overall cost for a probable scenario of a seven-day HHWS activation and interventions. Total cost estimates were rounded to the nearest AU$1,000.

Examining health cost savings attributable to the HHWS

We based our cost savings estimates on the estimated health benefits attributable to the HHWS, as reported by Nitschke et al.14 In that study, a case series design was used to compare health outcomes in the Adelaide population during pre- and post-intervention heatwave events, in 2009 and 2014, respectively. Measurable reductions in ambulance call-outs (297 fewer), renal hospital admissions (119), heat-related admissions (141) and emergency presentations (134 renal and 145 heat-related), were attributed to the introduction of the HHWS.14 The heat-related category incorporated admissions/presentations for dehydration, heat/sunstroke, and exposure to excessive heat.

We estimated the averted health costs for these reported morbidity reductions using publicly available information for ambulance and hospital costs. For ambulance call-out costs, we used the patient-borne cost for an emergency call-out from the SA Ambulance Service in 2020.26 Averaged hospital costs were based on the Australian Independent Hospital Pricing Authority (IHPA) National Efficient Price (2020–21) for an acute public hospital admission or Emergency Department (ED) presentation ($5,320 per weighted activity unit).27 Price weights were applied to adjust costs to the type of admission or ED presentation (activity unit). To estimate averted costs from reduced renal admissions, we applied the IHPA price weights for ‘Kidney and Urinary Tract Signs and Symptoms’ (L65A and L65B), representing major and minor complexity (1.4940 and 0.5368, respectively), to represent admissions for a range of potential severities. There was no comparable IHPA diagnostic category for ‘heat-related’ admissions, so we applied an intermediate level price weight of 1.0. To estimate averted costs from reduced ED presentations, we applied lower- and upper-level price weights, representing non-admitted low-tier presentations (0.066) to admitted high-tier (0.3415), to reflect a range of potential severities.

Ethics Approval

The study received approval from the Human Research Ethics Committees of the University of Adelaide (ID33179) and the SA Department of Health (HREC/18/SAH/34).

Results Examining SA HHWS activities and costs

Information about HHWS activities and costs was obtained through key informant interviews (n=11) and/or written responses (n=4), representing nine different agencies. These activities are briefly described in the following sections and the costs are outlined in Table 1.

Pre-seasonal HHWS planning

The broad pre-seasonal planning activities included reviewing heatwave strategies and/or communication plans (SASES, SA Health), inter-agency co-ordination, and seasonal activation of the Bureau of Meteorology (BoM) national heatwave service (Table 1). This service is updated daily throughout the heatwave season, from October to March, and provides the operational trigger for activation of the SA HHWS. Each summer, the SA BoM advises local emergency services and other agencies about likelihood of heatwaves in the coming season to ensure that services can be aligned.

For the SA Health community mental health (MH) program (outlined in Box 1), the MH teams review and update heat procedures each summer and undertake pre-seasonal assessments to identify clients who may need additional support during HHWS activations. For the Telecross REDi and Code Red programs (Box 1), pre-seasonal planning and co-ordination occurs across the relevant agencies (Table 1). While some informants also described pre-seasonal reviews of occupational health and safety policies and procedures, this was considered to be outside the remit of the HHWS.

Activation of the HHWS

The HHWS is activated based on the BoM prospective three-day heatwave forecast. This forecast uses the excess heat factor (EHF) metric – a measure of three-day heat load that indicates unusually hot conditions for a location relative to both recent and historical average temperatures.28 The EHF metric allows for harmonised operation of the HHWS in different climatic areas of the state.29, 30

Upon activation for a severe or extreme heatwave by the SASES, agencies implement their HHWS responses/interventions, and the State Emergency Centre (SEC) is convened to manage the state response. The SEC is led by the SA Commissioner of Police and provides a central point of co-ordination for emergency response and recovery operations. Representatives from the SASES, SA Health, and the BoM support the centre during severe or extreme heatwave events. It was noted that the resources to support the SEC depend on the severity of the heat warning, and whether there is a concurrent bushfire or other emergency in the state. Each event will vary in the duration and level of commitment required.

During HHWS activations the SASES and SA Health issue broad public heat warnings and health advice through print media, television, radio, social media, agency web sites and printed resources. In addition, the targeted intervention programs for vulnerable groups are implemented (Box 1). There are some additional levels of data reporting, including real-time surveillance of public hospital services. Functional supporting agencies, such as the SA Police and Local Government Association (LGA) submit daily activity reports to the SASES during HHWS activations, but this was described as a minor time commitment. Following each HHWS activation period, the SA Health MH teams conduct de-briefings and reporting, culminating with end-of-season reporting.

Estimated HHWS costs

The information provided by key informants comprised estimates for resource use and unit costs, and/or aggregated costs, and is presented in Table 1. Based on this information, we estimated an overall cost of $593,000 for implementing the HHWS and targeted interventions, using 2019/20 costings, and based on a seven-day activation for an extreme heatwave (Table 1). The total includes estimated staff costs and incidentals, with staff time representing the principal resource from an agency perspective.

Not all activities could be costed by key informants because there was often no dedicated budget or resource allocation. Similarly, not all costs could be provided for data capture/reporting activities. However, based on the information provided, the costs in Table 1 represent the substantive costs associated with activating the HHWS.

Estimating a benefit-cost ratio for the SA HHWS

The estimated health cost savings attributable to the HHWS are shown in Table 2. These were calculated by applying the costings described in Methods to the reported HHWS-attributable health benefits – reduced ambulance call-outs, renal hospital admissions, heat-related admissions and emergency presentations.14 These reported outcomes were associated with a nine-day HHWS activation during a past heatwave event (summer 2014). For a comparison of potential health cost savings (Table 2) with the estimated HHWS intervention costs (Table 1), we adjusted these to a common time period (nine days), to estimate a benefit-cost ratio of 2.0-3.3.

Table 2. Estimated HHWS-attributable health cost savings. Reported health benefits are from Nitschke et al. (2016).

HHWS-attributable health benefitsa

Approximation of cost savings per case

Total estimated cost savings(AU$)

Ambulance call-outs

(297 fewer cases)

$1044b

310,000

Renal admissions

(119 fewer cases)

$2856 (minor complexity)c

$7948 (major complexity)

(339,864-945,812)

643,000d

Heat-related admissions

(141 fewer cases)

$5320 (intermediate)c

750,000

ED presentations (279 fewer cases)

[Renal (134) and Heat-related (145)]

$351 (lower estimate)e

$1817 (upper estimate)

(97,929-506,943)

302,000f

Sub-total hospital admissions

1,393,000

Hospital admissions, EDs, ambulance

2,005,000

Total estimated range

1,498,000–2,513,000

Notes: a: Estimated in relation to a 9-day HHWS activation period in 2014 (Nitschke et al, 2016)14 b: The cost of transport for an emergency call-out starts at $1044. Source: SAAS26 c: National Efficient Price (NEP2020-21) is $5,320 per weighted activity unit. NEP2020-21 price weights for admitted acute ‘L65A and L65B Kidney and Urinary Tract Signs and Symptoms’: minor complexity 0.5368; major complexity 1.4940. [Cost range is $5320x0.5368=$2856 minor, to $5320x1.4940=$7948 major complexity] d: Average of the minor to major complexity range is $642,838 e: NEP2020-21 weighted prices for ED presentations range from: lower range estimate: [Non-Admitted_T5_All other MDB groups price weight=0.0660x5320=$351] to upper range [Admitted_T1 All other MDB groups price weight=0.3415x5,320=$1817]. f: Mid-range estimate=279x$1084=$302,436 Discussion

Although heat warning systems have been implemented widely there is limited evidence about their health benefits and costs. This paper presents a descriptive assessment of the cost benefits of the South Australian HHWS intervention. Building on our previous evaluation of health benefits,14 we estimated potential cost savings from reduced morbidity that offset the estimated HHWS implementation costs by at least two-fold. On this basis, the SA HHWS represents a ‘no regret’ heatwave adaptation strategy.

Cost-benefit assessments conducted in the US12 and Europe18, 19 using different study approaches have drawn similar conclusions. A HHWS in Philadelphia was attributed with a high monetary benefit because of the estimated reductions in mortality.12 The corresponding HHWS implementation costs were assumed to be negligible in comparison, but without detailed examination.12 Studies by Hunt et al.18 and Chiabai et al.19 used economic modelling for current and future climates to show positive benefit-cost ratios for HHWSs in several European cities. Both studies used simplified assumptions about HHWS costs and measures for effectiveness that were drawn from external populations. In contrast, we consulted with key informants to estimate intervention costs and used location-specific evidence for the HHWS health benefits.14 These aspects represent the strengths of this study. Through stakeholder consultation we examined the range of operational processes, interventions and costs, but the information collected was dependent on participant recall. The collection of cost data is not incorporated into the HHWS implementation, leading to some uncertainty in the cost estimates.

Other economic appraisals of HHWSs have assessed benefits based on mortality outcomes.12, 18, 19 As Chiabai et al19 have shown, these appraisals are highly dependent on the approaches to valuing mortality. In contrast, our estimates were based on reductions in morbidity outcomes because the SA HHWS was not associated with any decrease in mortality.14 Our estimates for hospital cost savings were based on the Australian Independent Hospital Pricing Authority national efficient price for health care services and were sensitive to the price weight.27 However, even the lower estimate for cost savings would offset the estimated HHWS costs by two-fold. For ambulance costs, we used the patient-borne cost for an emergency call-out because actual costs were not available from the SA Ambulance Service. It should be noted that the service is subsidised by state government funding and by using the patient-borne cost we have likely underestimated ambulance cost savings. We also note that our morbidity outcomes did not include any primary care services, such as visits to the local doctor.

While our results suggest an overall cost benefit from the SA HHWS, the contribution from each of the targeted interventions has not been evaluated. We estimated the highest costs for the MH support program. This program was developed because of the significant impact of heatwaves on MH morbidity in Adelaide,31 which contributes to substantial health service costs.32 Notably, there was no apparent decrease in MH hospitalisations following introduction of the SA HHWS (our unpublished results). This finding does not detract from the MH support program which may have prevented other heat illnesses within this target group. However, the effectiveness of telephone outreach for MH has been questioned33 after a Canadian study of heatwave deaths found that many decedents with mental illness had been contacted in the 24 hours before their deaths, by health providers, family or others.34

The Telecross REDi and Code Red programs are operated by non-government/non-profit organisations using existing outreach structures. To some extent, these programs rely on volunteers and/or lower paid workers, and the total cost may be more than the budget provided. Further impact evaluation could be undertaken to assess the effectiveness of these interventions; however, this may be constrained by the size of the respective target populations.

There are multiple assumptions and limitations inherent in this evaluation. Firstly, in order to minimise error due to information recall, we examined recent (2019/20) SA HHWS activities and costs. However, the published estimates for HHWS-attributable health benefits were derived from heatwave events occurring in 2009 (pre-) and 2014 (post-intervention).14 The inherent assumption is that the interventions and potential health benefits have not changed substantially in the intervening time period. While the Mental Health, Code Red and Telecross REDi interventions have been stable, there have been changes to the HHWS operational activities over this time, including the heat metric used for activation (now EHF). These changes are likely to have increased the current intervention costs and therefore would not undermine our overall conclusion.

We defined explicit inclusion criteria for HHWS activities and costs in this study. Firstly, we sought to distinguish HHWS-specific activities from the broader operations for each agency, in order to assign HHWS-specific costs. We did not include agency costs associated with staff welfare or designing modified work-rest regimes during heatwaves because we considered these to be seasonal occupational health and safety measures, and not HHWS-specific. Secondly, we only included HHWS activities that were undertaken systematically to ensure greater reliability of information and consistency over time. We cannot exclude the possibility that any other (ad hoc) activities may have contributed to the reported HHWS-attributable health benefits.14 For example, local councils sometimes extend the opening hours of public facilities to provide cool refuges. Prospective data collection within the relevant agencies would more accurately capture such activities and costs. Lastly, we have not included research and development costs related to the initial development of the HHWS. While these activities underpin the HHWS, they do not represent ongoing program costs.

The potential for confounding needs to be acknowledged in HHWS assessments.10 By attributing all estimated health benefits and cost savings to the SA HHWS, we are likely to be underestimating the impact of external factors, such as increases in household air-conditioning. There is also likely to be an ‘autonomous’ adaptation effect, with people changing their behaviours independently of the intervention.10 On the other hand, there is the potential for broader societal benefits from the HHWS that are not readily quantified or valued. For example, providing support for vulnerable groups during heatwaves may contribute to continuing positive effects on wellbeing, or even reduce heat-related aggression or violence.35 By increasing public awareness of heat risks, the benefits of the HHWS may extend beyond the heatwave period, leading to carry over effects. In this way, the benefits may be considerably underestimated. However, it should also be noted that HHWS activations may have opportunity costs, by diverting resources from other community supports or activities.

The assessment of cost benefits for HHWSs provides useful evidence for public health and other agencies that plan emergency responses for extreme heat. In South Australia, future planning will benefit from ongoing evaluations, including for each of the targeted support programs. Evidence for a reduction in mortality is lacking; this remains a key objective for the HHWS and would have significant cost benefit implications. As the climate continues to warm it will be important to re-assess the performance of the HHWS. It is expected that changes in heat exposure, population demographics and adaptation will result in changing vulnerabilities over time. Statistical approaches such as interrupted time series analysis may be best suited to measure the longer-term impacts on health outcomes and the effects of any system changes.36 Finally, future evaluations would benefit from prospective data collection for HHWS activities and costs within each of the relevant agencies.

Conclusions

This retrospective evaluation indicates an overall cost benefit from the South Australian Heat Health Warning System and supports this as a ‘no regret’ response to heatwaves. As temperatures rise, a dynamic system will be needed to respond to changing exposures and vulnerabilities, to drive heat adaptation, and to ensure that support is effective and appropriately targeted.

Acknowledgements

The authors acknowledge the funding support from the NHMRC (Project Grant APP1145239 to P. Bi) and the generous and valuable involvement of key informants from all agencies.