National Aged Care Mandatory Quality Indicator Program: July to September 2019.
These notes outline the steps undertaken by the Australian Institute of Health and Welfare (AIHW) to process the data, construct the quality indicators (QIs) and provide an initial assessment of the quality of the data.
Specifications for the QIs are published in the National Aged Care Mandatory Quality Indicator Program Manual 1.0. Users of the QI data are advised to refer to the Manual for details of the data elements that have been submitted by residential aged care services and for information about indicator construction. Some of this information is summarised below.
Data collection and transmission to AIHW
In accordance with the Manual, all Australian Government-subsidised residential aged care services are required to collect data at the service level and submit their data via the My Aged Care Provider Portal to the Department of Health, or with the prior agreement of the Department, through a commercial benchmarking company.
Data for the first quarter—1 July 2019 to 30 September 2019—were required to be submitted by 21 October 2019.
QI data were provided to the AIHW via secure data transfer from the Department on 1 November 2019 and 20 November 2019 in the case of the last file listed below, in 5 separate files:
- an extract submitted through the My Aged Care Provider Portal
- a file from Victorian Public Sector Residential Aged Care Services (PSRACS)
- two files submitted to the Department by independent benchmarking companies
- one file from a large provider operating multiple residential aged care services.
Supplementary material, including the Department’s responses to queries raised during initial processing (see below) and some newly submitted data, were provided during the period up to 20 November 2019.
An additional file was provided by the Department that contained ‘occupied bed day’ (OBD) data for the period 1 July to 30 September 2019, for input to the denominators for calculation of the QIs.
Examination and cleaning of the QI data
Stage 1—checking the validity of service identifiers
The unique identifier for a residential aged care service (RACS_ID) is a number of up to 4 digits. For 19 records this field contained invalid information. These were referred to the Department for advice, and resulted in 12 records not able to be identified as a residential aged care service and removed from the data set.
Stage 2—resolution of duplicate records on individual source files
Ten cases of multiple records submitted under an individual RACS_ID were identified, most appearing to be separate data relating to wards or other management units within a single service. One duplicate pair was resolved by combining data, advice from the Department assisted with correcting the RACS_ID of two others and the remainder were removed as unable to be identified as a residential aged care service.
Stage 3—creation of the QI data set
Data fields on files were reformatted as required to create consistent field names and data types. The 5 files were then appended to create a single data set with 2,754 records.
Stage 4—resolution of duplicate records in the amalgamated QI data set
The creation of the QI data set identified 562 potential duplicate records (i.e. 281 pairs with the same RACS_ID). The majority of these duplicates were the result of QI data being entered both into the My Aged Care Provider Portal and submitted through a commercial benchmarking company or the Victorian PSRACS. These duplicates were resolved in 3 steps:
- For 1 pair, an error in the RACS_ID was corrected.
- For 160 pairs of QI data that were complete duplicates, those submitted through the My Aged Care Provider Portal were retained, in accordance with advice from the Department,
- For the remaining 120 pairs, it was identified that the records submitted through the My Aged Care Provider Portal contained more complete QI data, and in accordance with advice from the Department, these records were retained.
The ‘de-duplicated’ data set contained 2,473 records.
Merging OBD and service level data
The Manual specifies that QIs are calculated using a common denominator—(QI measure) per 1,000 OBDs. The number of OBDs for each residential aged care service is defined in the Manual as ‘the number of days in care in the subsidy claiming system’.
To bring numerator and denominator data together for indicator compilation, the OBD file for individual residential aged care services provided by the Department was merged with the de-duplicated QI data set. Six QI records that failed to match on their RACS_ID were removed, leaving 2,467 records in the QI data set.
Other mismatches between the OBD file and the QI data set resulted from a greater number of services appearing in the OBD file than the QI file—there were 429 failed matches from the OBD file. Further exploration of the reasons for this mismatch will be undertaken in future data updates, with back-casting and updating of the July-September quarter results as necessary.
Service level data
The QI data set was merged with service level data from the NACDC, to add additional characteristics for analysis—for instance, state and territory and regional characteristics published in the AIHW’s QI release for July to September 2019. This merge was done through a concordance, supplied by the Department, between the RACS_ID and the National Accredited Provider Service (NAPS) number, the identifier used in the NACDC. In this step, 54 records failed to match with a service identified in the NACDC. This reduced the data set available for analysis of QIs with service-level data to 2,413 records
Service response levels
Response levels are considered good for the first quarter of data collection. The 2,413 records with QI data available for analysis represent 90% of the 2,688 residential aged care services identified in the NACDC.
The completeness of the data submitted by residential aged care services varied for the 3 QIs:
- 36 residential aged care services did not supply data on pressure injuries
- 27 residential aged care services did not supply data on use of physical restraints
- 201 residential aged care services did not supply data on unplanned weight loss.
These numbers are relatively low. Reporting against unplanned weight loss indicators was perhaps more complete than expected, as the collection of weight data for the prior March to June quarter (from which to calculate a loss over time) had not been made mandatory under QI Program legislation.
Apparent coverage of residents in QI reporting
AIHW estimated the number of residents in each residential aged care service by dividing the number of OBDs by 92 (the number of days in the quarter). Against this computed number, the number of residents assessed for pressure injuries and monitored for unplanned weight loss, as reported by residential aged care services, provides an estimate of the proportion of residents of a residential aged care service covered in the QI data. These proportions had mean values of just over 100% for pressure injuries and just under 90% for unplanned weight loss indicators. However, proportions for individual services showed wide variation, with some values being implausibly high and others very low or zero. The AIHW could reach no firm conclusion about the reasons for this variation, based on the information it has. However, it did note a degree of commonality amongst services showing apparent high coverage levels across these indicators. This suggests that there may be common issues with alignment across the separately supplied OBD data with QI data submitted by residential aged care services. These issues will be further explored in future data updates, with back-casting and updating of the July-September quarter results as necessary.
As the number of residents observed for use of physical restraint was not reported (the Manual does not specify it), no corresponding statistics for apparent coverage for restraint use can be derived.
Outliers in calculated QIs
The AIHW has no firm basis for determining that an apparent ‘outlier’ in the distribution of QIs across residential aged care services represents an incorrect data point. While reporting of residential aged care (or long-term care) quality indicators has occurred in a number of overseas countries, indicator specifications and data collection protocols in use elsewhere do not match those prescribed for Australian residential aged care services. However, a QI program using the same indicators as those defined in the Manual has been operating in Victorian PSRACS for some years. Some regard could be paid here to reference ranges for aged care quality indicators published by the Victorian Department of Health (Campbell Research 2011).
A preliminary analysis of the distribution of the data for each QI across reporting units (residential aged care services) shows a wide dispersion. While median values and the inter-quartile ranges for calculated QIs at service level generally fall below the upper limits recommended for Victorian PSRACS for pressure injuries and unplanned weight loss QIs, some implausibly high indicator values are recorded for individual services, in both indicator domains. Similar comparisons for use of physical restraint indicators could not be made in the same manner with upper limits for reference ranges, as Victorian PSRACS recommend zero tolerance for these indicators.
No conclusion could be drawn about the possible presence of outliers that may represent implausibly low values. These are inter-mixed with valid low QI values.
Good levels of reporting are evident in this first release of data from the National Aged Care Mandatory Quality Indicator Program. Some issues with the dispersion of proportions of residents covered at service level and of the calculated QIs have been identified. Given these data quality issues, caution should be exercised in interpreting the indicators in this first quarterly report.
This release is the beginning of a process for residential aged care services, the Department and the AIHW. Future updates will provide further opportunity to assess and improve the quality of the data.
Campbell Research and Consulting 2011. Development of reference ranges for aged care quality indicators: Prepared for the Department of Health. Melbourne: Department of Health.
Department of Health 2019. National Aged Care Mandatory Quality Indicator Program Manual 1.0. Canberra: Department of Health.