9
The Audit and Best Practice for Chronic Disease CQI research program
A brief history
The Audit and Best Practice for Chronic Disease (ABCD) National Research Partnership originated in 2002 in the Top End of Australia’s Northern Territory. It built on substantial prior research and evaluation of CQI methods, using participatory action research to explore the feasibility and acceptability of CQI tools and processes in Aboriginal and Torres Strait Islander PHC. From 2010, with growth and support from service providers and researchers around Australia, the ABCD partnership explored variation in clinical performance, examined strategies for improving primary care, and worked with health service staff, management and policymakers to implement successful improvement strategies.1 By the end of 2014, the ABCD partnership had generated the largest and most comprehensive dataset on quality of care in Aboriginal and Torres Strait Islander PHC settings.
The work was extended through the Centre for Research Excellence in Integrated Quality Improvement. The 2015–19 phase of research resulted in further evidence on the effectiveness of CQI, factors that support its use by PHC teams and services, priorities for achieving large-scale health improvement, and the importance of Aboriginal and Torres Strait Islander leadership and participation in PHC services and research in this sector. From 2020, an Indigenous-led collaboration, the Centre for Research Excellence: Strengthening Systems for Indigenous Health Care Equity (CRE-STRIDE), focused on embedding CQI into PHC practice and expanding its use to tackle the social, cultural and environmental determinants of health.2 Ongoing research is based on growing evidence of the importance of community-driven, culture-strengthening interventions in Indigenous PHC settings.
Table 9.1 sets out the evolution of the ABCD CQI research program between 2002 and 2024, listing the key focus and aims of each phase of research.
| 2002–2004 | 2005–2009 | 2010–2014 | |
|---|---|---|---|
| ABCD project | ABCD Extension | ABCD National Research Partnership | |
| Focus | Exploring feasibility and acceptability of CQI tools and processes | Exploring scalability and expansion of CQI. Developing a CQI cycle for PHC | Supporting wide-scale implementation and developing a model for CQI |
| Aim | Explore whether a CQI approach was feasible and effective in Indigenous PHC3 | Identify support requirements for large-scale implementation of the ABCD model4 | Understand variation in quality of care and strategies for improvement5 |
| (2010-2016) | 2015–2019 | 2020–2024 | |
|---|---|---|---|
| One21seventy (service support, 2010–2016) | Centre for Research Excellence in Integrated Quality Improvement | CRE-STRIDE | |
| Focus | Supporting wide-scale implementation and developing a model for CQI | Embedding CQI approaches in systems | Strengthening leadership and engagement in system-wide CQI |
| Aim | Provide PHC services with training, tools and support for CQI, including a web-based data analysis and reporting system6 | Accelerate and strengthen large-scale CQI efforts7 | Strengthen Aboriginal and Torres Strait Islander research leadership for CQI. Apply Indigenous knowledge and methodologies. Extend CQI processes and tools to enhance community linkages and tackle social and cultural determinants of health.8 |
| Source: Adapted with permission from Bailie, Potts et al. 2021. | |||
Table 9.1 ABCD CQI research program 2002–24 – focus and aims of each phase of research.
This CQI research in Aboriginal and Torres Strait Islander PHC has made a large contribution to original research on CQI in PHC internationally, with over 180 papers published in the peer-reviewed literature between 2002 and mid-2023.
The ABCD participatory action research program produced evidence-based CQI tools and resources for PHC teams. These included a clinical systems assessment tool and eight clinical audit tools. The findings presented in the following 10 chapters resulted from the use of these tools. An audit and systems assessment tool for health promotion, a tobacco control audit tool and a food systems assessment tool were also developed (see Chapter 5).
Clinical audit tools based on clinical practice guidelines
As explained in Part II, the ABCD clinical audit tools and associated protocols are based on evidence and the service items listed in the relevant Australian and international guidelines for recommended care. Like the clinical guidelines, the ABCD audit tools were developed in consultation with expert reference groups. These groups comprised clinical experts, PHC practitioners with experience working in Aboriginal and Torres Strait Islander PHC and others with understanding of quality improvement and experience in using audit tools. The ABCD audit tools and processes for each area of clinical care (for example, child health) are designed to help PHC teams to identify items of care (for example, growth monitoring) that are being delivered well and less well, and priorities for improvement.
Systems assessment tool based on primary healthcare systems research
The ABCD systems assessment tool is designed to be used in conjunction with each clinical audit tool. The tool was also developed collaboratively and is based on internationally accepted models that reflect the way health systems work and interact at organisational, practice, client and community levels: namely the Chronic Care Model and associated Assessment of Chronic Illness Care tool developed in the United States, and the Innovative Care for Chronic Conditions Framework from the World Health Organization (WHO).
The resulting CQI tools are based on evidence, nationally applicable and realistic for use in a variety of PHC settings. Box 9.1 describes how PHC teams use the tools.
Box 9.1 How PHC teams use clinical audit and systems assessment tools
The ABCD clinical audit tools and processes cover major chronic conditions (diabetes, hypertension, coronary heart disease and renal disease), maternal health care, child health care, preventive services, mental health, sexual health, rheumatic heart disease and youth health. The clinical audit tools are used to retrospectively measure the quality of care documented in client records: they gather data on clinical performance to identify strengths in service delivery and important evidence-to-practice gaps. The ABCD system assessment tool enables PHC teams to undertake a structured assessment of the strengths and weaknesses of the organisational systems that support client care in each of these clinical areas. Each tool has a detailed protocol, which guides staff in using the tool and supports the generation of consistent data for comparison across consecutive audit cycles.
Within health services, PHC teams have used the tools for annual cycles of assessment and feedback. Data about “quality gaps” in clinical care and in health service systems have been used in conjunction with knowledge about the health service infrastructure, resources, staffing, community context and client population to inform goal setting and to plan and implement improvement. The effectiveness of improvement strategies is measured in the following audit cycle.
The ABCD tools provide PHC teams with data that can be presented in different formats to aid analysis and interpretation.
Using aggregated data for system improvement
Between 2005 and 2014, over 250 health services across five Australian jurisdictions used the same CQI tools to audit client records and assess system function. With consent from 175 of these services, de-identified data were brought together to provide information about the quality of care provided across a wider Aboriginal and Torres Strait Islander population living in urban, regional and remote areas. By aggregating and analysing these CQI data, it has been possible to establish patterns, variations, gaps and improvements in key areas of clinical PHC. Many of the examples in this part of the book report these results.
Aggregated CQI data can be used at different health system levels. Individual local PHC services and teams can use the data to benchmark the quality of care they provide against regional or national standards. Policymakers, government departments and large health organisations can use aggregated CQI data to compare service delivery between local health services or regions, to explore factors that contribute to variations in care, and to establish baselines for measuring the effect of their improvement policies and strategies. At all levels, aggregated CQI data can help to identify shared priorities for strengthening systems and policies to improve care. There has been very little CQI data available to date for these purposes. The data on clinical performance in PHC arising from the ABCD program is a unique resource in terms of its scale (number and geographic spread of PHC services), and its scope (range of important aspects of PHC care and service and system development). The data has ongoing relevance for meeting challenges in PHC equity.
International relevance of the ABCD research program
The knowledge gained from this work in Aboriginal and Torres Strait Islander PHC settings has wider relevance for implementing CQI. Although population needs and characteristics, health service structures and resources, PHC teams, settings and policy contexts differ, similar PHC improvement challenges prevail across settings and countries. In addition, these internationally accepted principles for the effective implementation of CQI underpin our approach and are reflected in:
- the use of CQI tools that are based on the best available evidence about what constitutes best practice
- the use of accurate, context-relevant data
- the participation of people in a range of roles relevant to care and service delivery
- the importance of contextual knowledge in data interpretation and intervention planning.
In the following chapters, we describe the implementation of CQI in 10 key areas of clinical PHC. These areas are considered key because they are important for the delivery of comprehensive PHC across the life span; they reflect Australian and international priorities for improving primary care for First Nations peoples; and they are important for improving health equity. In Australia, Aboriginal and Torres Strait Islander peoples continue to experience unequal access to health care, shorter life expectancy and an unequal burden of ill health compared with the general population. Many people experience health conditions associated with low-resource settings, including otitis media in children, trachoma and rheumatic heart disease. The topics of the next 10 chapters reflect these PHC challenges and CQI priorities.
Presentation of research findings
For each chapter topic, we provide background information, key aspects of recommended care, data and findings from our CQI research in Aboriginal and Torres Strait Islander PHC. Drawing on these findings, and research and experience in broader PHC contexts, we offer general messages for improving PHC delivery relevant to that condition or aspect of clinical care. Specific considerations for improving care for Aboriginal and Torres Strait Islander peoples are also included. As colonisation is a fundamental determinant of Indigenous peoples’ health worldwide, these considerations have relevance for improving care for other First Nations populations.
How to interpret the graphs in these chapters
The results of CQI research are presented using bar graphs and box and whisker plots. Here are some tips for interpreting the box and whisker plots in these chapters (Figure 9.1).
Box and whisker plots are used to show variation in PHC delivery between health centres, such as these examples:
- health centres with the minimum and maximum mean percentage in recorded delivery of care in accordance with best-practice guidelines (ends of whiskers show highest value if no outliers)
- outliers – health centres that are far away from most others in the dataset (or a distance that is greater than 1.5 times the length of the box)
- the level of variation between health centres in recorded delivery of care by dividing scores into quarters:
- the box represents the middle 50 per cent of health centres, and the line within the box represents the median (or middle health centre)
- the “whisker” at the top of the box (and outliers if present) represents the top 25 per cent of health centres
- the “whisker” at the bottom of the box (and outliers if present) represents the bottom 25 per cent of health centres
- the longer the box plot, the greater the range of care delivery (or variation) between health centres.
Figure 9.1 How to interpret box and whisker plots.
In assessing data trends, it is helpful to focus on:
- the trend for the mean (average) and median (middle) values for health centres – in particular, whether the mean and median are increasing, staying steady or decreasing
- the trend in the variation between health centres – in particular, whether the variation is getting less (shorter boxes, shorter whiskers) and, importantly, whether there is an improvement in the values for the health centres at the lower end of the range (higher level for the bottom end of whiskers under boxes).
References
Bailie, J., A. Laycock, D. Peiris, R. Bainbridge, V. Matthews, F. Cunningham et al. (2020). Using developmental evaluation to enhance continuous reflection, learning and adaptation of an innovation platform in Australian Indigenous primary health care. Health Research Policy and Systems 18(1): 45. DOI: 10.1186/s12961-020-00562-4.
Bailie, R., J. Bailie, S. Larkins and E. Broughton (2017). Editorial: Continuous quality improvement (CQI)—advancing understanding of design, application, impact, and evaluation of CQI approaches. Frontiers in Public Health 5(306). DOI: 10.3389/fpubh.2017.00306.
Bailie, R., D. Si, C. Connors, T. Weeramanthri, L. Clark, M. Dowden et al. (2008). Study protocol: Audit and Best Practice for Chronic Disease Extension (ABCDE) Project. BMC Health Services Research 8: 184. DOI: 10.1186/1472-6963-8-184.
Bailie, R., D. Si, C. Shannon, J. Semmens, K. Rowley, D.J. Scrimgeour et al. (2010). Study protocol: national research partnership to improve primary health care performance and outcomes for Indigenous peoples. BMC Health Services Research 10(129): 1–11. DOI: 10.1186/1472-6963-10-129.
Bailie, R., D. Si, S. Togni, G. Robinson and P. d’Abbbs (2004). A multifaceted health-service intervention in remote Aboriginal communities: 3-year follow-up of the impact on diabetes care. Medical Journal of Australia 181(4): 195–200. DOI: 10.5694/j.1326-5377.2004.tb06235.x.
CRE-STRIDE (n.d.). Strengthening systems for Indigenous health care equity. http://cre-stride.org.
Cunningham, F., S. Ferguson-Hill, V. Matthews and R. Bailie (2016). Leveraging quality improvement through use of the Systems Assessment Tool in Indigenous primary health care services: a mixed methods study. BMC Health Services Research 16(1): 583. DOI: 10.1186/s12913-016-1810-y.
Laycock, A., K. Conte, K. Harkin, J. Bailie, V. Matthews, F. Cunningham et al. (2020). Improving the quality of primary health care for Aboriginal and Torres Strait Islander Australians: Messages for Action, Impact and Research. Centre for Research Excellence in Integrated Quality Improvement 2015–2019. Lismore, NSW: University Centre for Rural Health, University of Sydney.