Pursuing Continuous Quality Improvement as a Uniqueness Factor is Sound Because It

• Journal List
• Cochrane Database Syst Rev
• PMC6486276

Cochrane Database Syst Rev. 2017 Nov; 2017(11): CD003319.

Continuous quality improvement: effects on professional practice and healthcare outcomes in primary care

Sue Brennan, Joanne E McKenzie, Paula Whitty, Heather Buchan, Sally Green, Marije Bosch, Michelle Fiander, Natalie A Strobel, Emma Donoghue, Matthew J Page, Jennifer Yost, and Daniela C Gonçalves‐Bradley

Sue Brennan

School of Public Health & Preventive Medicine, Cochrane Australia, Monash University553 St Kilda Road, MelbourneAustralia, 3004

Joanne E McKenzie

Monash University, School of Public Health & Preventive Medicine, 553 St Kilda Road, MelbourneAustralia, 3004

Paula Whitty

Newcastle University, Institute of Health and Society, NewcastleUK,

Heather Buchan

National Health and Medical Research Council, National Institute of Clinical Studies (NICS), GPO Box 4530, MelbourneAustralia, 3001

Sally Green

School of Public Health & Preventive Medicine, Cochrane Australia, Monash University553 St Kilda Road, MelbourneAustralia, 3004

Marije Bosch

Monash University, National Trauma Research Institute/Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Level 6The Alfred Centre, 99 Commercial Road, MelbourneAustralia, 3004

Michelle Fiander

Information Specialist, Consultant, OttawaCanada,

Natalie A Strobel

The University of Western Australia, School of Biomedical Sciences, 35 Stirling Highway, CrawleyAustralia, 6009

Emma Donoghue

Monash University, Australian & New Zealand Intensive Care Research Centre (ANZIC‐RC), School of Preventive Medicine and Public Health, MelbourneAustralia, 3004

Matthew J Page

Monash University, School of Public Health and Preventive Medicine, 553 St Kilda Road, MelbourneAustralia, 3004

Jennifer Yost

McMaster University, School of Nursing, 1280 Main Street West2J24H, HamiltonUSA, L8S 4L8

Daniela C Gonçalves‐Bradley

University of Oxford, Nuffield Department of Population Health, OxfordUK,

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

Primary aims

Is CQI effective for improving professional practice and healthcare outcomes compared to no intervention and is the magnitude of effect modified by the following factors?

• Intensity of the CQI implementation strategy.

• Organisation‐wide CQI strategy or support for CQI.

• Team performance and participation.

• Complexity of the targeted change.

• Type of healthcare setting.

• Healthcare professional performance at baseline.

Secondary aims:

Is CQI effective for improving professional practice and healthcare outcomes compared to:

• single interventions to change health professionals' behaviour (classified according to the Cochrane Effective Practice and Organisation of Care (EPOC) Group taxonomy); or

• multifaceted interventions to change health professionals' behaviour (classified according to the Cochrane EPOC Group taxonomy); or

• tailored interventions to overcome identified barriers to change; or

• other organisational change interventions with a team‐based component (e.g. Business Process Reengineering)?

Background

Healthcare providers, governments, and professional bodies have invested heavily in strategies to improve the quality and safety of health care. One of the most significant investments has been in strategies that focus on improving the processes and systems used to deliver care. Continuous Quality Improvement (CQI) is one of these strategies. CQI is one of several quality management approaches originating in the manufacturing sector to gain popularity in health care. Since its introduction to health care in the late 1980's, it has been widely adopted in hospitals and large healthcare organisations, particularly in the United States (Berwick 1989; Laffel 1989; Barsness 1993; Ferlie 2001). Use of CQI principles and practices continues to be a requirement of many accreditation bodies (Ferlie 2001) and persistent reports of systemic quality and safety problems continue to fuel interest in its use in all healthcare sectors (Institute of Medicine 2001; Ferlie 2001; Grol 2008).

CQI comprises an overarching theory of quality management, a set of management practices, and practical methods and tools used to operationalise the theory (McLaughlin 2006). Definitions, descriptions and actual implementation of CQI can be extremely varied, resulting in debate over the range of approaches that should be labelled as CQI. Boerstler and colleagues (Boerstler 1996) summarise the main components of CQI, in a definition that reflects the components of a CQI strategy most commonly cited in the healthcare literature (Shortell 1998): "1) a philosophy of continuous improvement of quality through improvement of organisational processes; 2) use of structured problem‐solving processes incorporating statistical methods and measurement to diagnose problems and monitor progress; 3) use of teams including employees from multiple departments and from different organisational levels as a major mechanism for introducing improvements in organisational processes; 4) empowering employees to identify quality problems and improvement opportunities and to take action on these problems and opportunities; and 5) an explicit focus on "customers"‐ both external and internal." (Boerstler 1996, p144).

The components of CQI mirror the underlying theory. Central to this theory is the concept that quality problems result from poor processes and systems, rather than from individual performance. Processes rely on the interdependent actions of individuals who share the knowledge essential for process improvement. This knowledge is captured by involving staff in cross‐disciplinary process improvement teams. Models and tools for problem solving, group decision making, and planning and testing small‐scale changes provide a method for teams to make changes that address local causes of undesirable variation in care. When embedded in every‐day work, this approach allows new evidence, technologies, and models of care to be incorporated into work processes, enhancing an organisation's ability to respond to the ever‐changing needs of its customers (Berwick 1989; Laffel 1989; Berwick 1990; Batalden 1993; McLaughlin 2006).

The definition of CQI we will use in this review is based on the core components of CQI, with slight modification to encompass variants used in smaller healthcare settings and those used independently of organisation‐wide strategies. Such variants focus on team‐based CQI, which is the level at which CQI methods are operationalised and the focus of this review. CQI teams may be pre‐existing clinical care teams, naturally occurring work groups such as in primary care settings, or they may be temporary teams formed specifically to undertake process improvement. Focussing on team‐based CQI facilitates comparison across studies using an organisational unit common to all settings.

The CQI methods operationalised at team level are typical of those used in primary care practices (Solberg 1998; Geboers 1999) and in quality improvement collaboratives (Kilo 1998; Ovretveit 2002). A widely recognised example is The Model for Improvement, developed by Moen and Nolan (Moen 1987) and promoted by the Institute for Healthcare Improvement. The model guides teams through the steps of CQI: identifying areas for improvement; setting measurable aims; identifying changes to the process of care that are likely to result in improvement; and using repeated Plan‐Do‐Study‐Act (PDSA) cycles and data analysis to plan and test changes (Institute for Healthcare Improvement 2008). Other examples include the FOCUS‐PDCA^® cycle developed by the Hospital Corporation of America (McEachern 1992) and the team quality improvement sequence (TQIS) used in the Norwegian health system (Ovretveit 1999).

Identifying CQI interventions can be complex because of unclear boundaries with other approaches to quality management and inconsistent labelling (Boaden 2008). In health care, the term Total Quality Management (TQM) is used interchangeably with CQI (McLaughlin 2006), although the two are not always synonymous. The terms quality management and quality improvement are also used to label CQI interventions. These terms are more commonly used as umbrella terms, covering an array of approaches of which CQI is one. Quality assurance and quality control are sometimes used interchangeably with CQI; however these approaches usually focus on ensuring compliance with performance indicators and include few if any CQI components (McLaughlin 2006). A number of interventions share the core components of CQI and will be considered for inclusion in this review. Quality improvement collaboratives, Six Sigma, and Lean are notable examples because of current investment in their use. Quality improvement collaboratives differ from CQI in that they bring together teams from multiple organisations to work on the same topic over a defined period of time (Kilo 1998;Ovretveit 2002; Wilson 2003). This intensive, facilitated approach to using CQI methods aims to accelerate the rate of process improvement. Six Sigma is an improvement methodology that aims to eliminate defects by reducing variation in processes. Its name represents the limit of acceptable variation in a process defined by its originators at Motorola. Lean is a management philosophy that focuses on system‐wide evaluation of work processes to optimise steps that add value and improve flow, and eliminate those that create waste. Both approaches apply quality improvement principles, methods, and tools similar to CQI (Boaden 2008).

CQI requires a supportive organisational context and a careful implementation strategy to facilitate adoption, ongoing use, and achievement of successful outcomes. O'Brien and Shortell's (O'Brien 1995) CQI model has four dimensions of capability thought necessary for successful implementation: 1) cultural ‐ views, norms, beliefs, and behaviours in an organisation that support the principles and practice of CQI; 2) technical ‐ competency in CQI methods and tools; 3) strategic ‐ alignment of CQI activities with the organisation's priorities; 4) structural ‐ management structures and systems that support CQI, including appropriate data and analysis systems. Measures of organisational support for CQI covering one or more of these dimensions may be predictive of CQI success (Shortell 1998). These include measures of organisation‐wide implementation of CQI, commitment to quality improvement, and organisational culture (Shortell 2000; Meurer 2004; Mannion 2008). Leadership support for CQI at all levels (top management, team, external profession, respected peers) is also believed to be critical (Berwick 1990).

At team level, ability to work as a team (team performance), team member participation, and presence of a champion supporting the CQI efforts may be important effect modifiers (Lemieux‐Charles 2002;Wilson 2003; Shortell 2004; Solberg 2007). Physician support and participation on CQI teams is thought to be particularly important (Weiner 1996; Shortell 1998). Team members' technical competence with CQI is widely regarded as critical to CQI success. Training in the theory, methods, and tools of CQI is used to achieve competency and increase motivation to participate in quality improvement (Boonyasai 2007). Competence can be bolstered with support to facilitate implementation and use. The nature and complexity of the targeted change has also been suggested to modify outcomes (Shortell 1998; Solberg 2007).

More than twenty years since CQI was first used in health care, there remains a firm belief that the methods offer great promise for improving the quality of care (Ferlie 2001). Despite an increasing body of research on CQI and its variants there is still uncertainty about its effects and the factors that might enhance effect (Shortell 1998; Iles 2001, Wagner 2001; Ovretveit 2003; Wensing 2006; Schouten 2008). However, there has been widespread debate about methods for evaluating complex quality improvement interventions such as CQI, particularly around the use of randomised trials (Mittman 2004; Auerbach 2007; Berwick 2008; Stevens 2008). While we agree that much can be learned from the experiences of those applying these methods in practice, we contend that well‐designed experimental studies, and synthesis of these studies, provide the most robust evidence for addressing questions of effectiveness and mechanism of effect. If team‐based CQI is effective, it should be possible to measure improved professional practice and healthcare outcomes in studies using robust designs. This review aims to identify and synthesise such studies.

Objectives

The overall objective of this review is to determine if CQI is effective for improving professional practice and ultimately healthcare outcomes. We plan to do this by conducting a systematic review of existing evidence to address the following questions:

Primary aims

Is CQI effective for improving professional practice and healthcare outcomes compared to no intervention and is the magnitude of effect modified by the following factors?

• Intensity of the CQI implementation strategy.

• Organisation‐wide CQI strategy or support for CQI.

• Team performance and participation.

• Complexity of the targeted change.

• Type of healthcare setting.

• Healthcare professional performance at baseline.

Secondary aims:

Is CQI effective for improving professional practice and healthcare outcomes compared to:

• single interventions to change health professionals' behaviour (classified according to the Cochrane Effective Practice and Organisation of Care (EPOC) Group taxonomy); or

• multifaceted interventions to change health professionals' behaviour (classified according to the Cochrane EPOC Group taxonomy); or

• tailored interventions to overcome identified barriers to change; or

• other organisational change interventions with a team‐based component (e.g. Business Process Reengineering)?

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs); controlled clinical trials (CCTs); controlled before and after studies (CBAs) with at least two intervention sites and two appropriate control sites and contemporaneous data collection; and interrupted time series analyses (ITS) with a clearly defined time point when the intervention occurred and at least three data points before and three after the intervention. We expect that these designs will generally involve clustering since CQI strategies are practice‐ or group‐level interventions.

Types of participants

Health professionals responsible for patient care in any healthcare setting. CQI teams may include other members, such as non‐clinical staff involved in the process of care, but interventions involving teams that do not include health professionals will be excluded.  Interventions conducted in non‐healthcare settings or only involving students will be excluded.

This review is limited to primary care settings. We used Starfield's definition of the features of primary care to guide inclusion decisions: "first‐contact access for each new need; long‐term person‐ (not disease) focused care; comprehensive care for most health needs; and coordinated care when it must be sought elsewhere" (p458) (Starfield, Shi et al. 2005). Eligible settings included general practice (including family practice and primary care physicians), community health facilities, dentistry practice, and primary contact allied health.

Types of interventions

To be considered for inclusion in this review an intervention must:

1. involve healthcare teams (including professionals and others involved in the process of care) undertaking a structured, iterative process to review and improve one or more of the processes or systems they use to deliver clinical care; and

2. include the following CQI methods:

   1. review and analysis of a process or system used to deliver clinical care to identify sources of variation and areas for improvement;

   2. a structured process improvement method or problem solving approach that is used to plan and test small‐scale changes to the work process (e.g. The Model for Improvement; PDSA cycles); and

   3. use of measurement and data analysis to assess and review the effect of changes.

These criteria encompass the components of CQI common to team‐based, organisation‐wide, and inter‐organisation approaches and are the minimum criteria for inclusion of studies in the review. Interventions may include additional components, as described under 'Characteristics of the CQI intervention'.

Interventions targeting improvement of administrative, management, or other processes not directly related to clinical care will be excluded.

Types of outcome measures

To be considered for inclusion in the review, studies must include one or more of the primary or secondary outcomes.

Primary outcomes

Any objective measure of:

1. healthcare professional performance (e.g. adherence to recommended practice or process of care).

Secondary outcomes

Any objective measure of:

1. patient outcome (e.g. pain, health‐related quality of life, function, mortality); or

2. surrogate patient outcome (e.g. patient experience of care, patient evaluation of care co‐ordination, readmission, length of stay).

Outcomes assessed by questionnaire will be categorised as objective if an instrument with established reliability and validity was used.

Other outcomes

While the primary aim of this review is to determine the effect of team‐based CQI on healthcare provider performance and patient outcomes, CQI strategies have the broad goal of improving organisational effectiveness across all dimensions of quality (effectiveness, safety, efficiency, equity, patient centeredness, timeliness (Institute of Medicine 2001)). Hence, we will extract data on any objective measure of the following CQI‐relevant outcomes.

1. Organisational changes to systems or processes used to deliver care (e.g. implementation of the Chronic Care Model, adoption of new systems for care co‐ordination).

2. Organisational performance indicators (e.g. access, equity, wait lists).

3. Team performance or participation (e.g. perceived team effectiveness, team climate).

4. Resource use (cost of implementing and sustaining the intervention, and subsequent use of healthcare resources).

5. Sustainability of effects over time.

Search methods for identification of studies

Electronic searches

We will search the following electronic databases for primary studies.

1. The Cochrane EPOC Group Specialised Register and the database of studies awaiting assessment (see 'Specialised register' under About Cochrane, Cochrane Groups, Effective Practice and Organisation of Care).

2. The Cochrane Central Register of Controlled Trials (CENTRAL).

3. Bibliographic databases: MEDLINE, EMBASE, CINAHL, PsycINFO, and Scopus.

We will also search the Database of Abstracts of Reviews of Effects (DARE) to identify potentially relevant reviews.

We will use the following strategy to search MEDLINE and will adapt the strategy for other databases, using the most appropriate controlled vocabulary for each. The strategy combines text words and controlled vocabulary for describing CQI with the Cochrane EPOC Group methodological filter. We will not apply any language restrictions.

Ovid MEDLINE search strategy

1. (quality and ((continuous$ or total) adj5 (manag$ or improv$))).tw.

2. ((continuous$ or total) and (quality adj3 (manag$ or improv$))).tw.

3. (CQI or TQM).tw.

4. total quality management/

5. quality manag$.tw.

6. ((process or processes or system or systems) adj3 (improving or improvement or improve or redesign$)).tw.

7. model for improvement.tw.

8. ((improvement or QI or quality assurance or QA) adj5 (team? or microsystem? or cycle?)).tw.

9. (PDSA or PDCA or TQIS or plan do study or plan do check).tw.

10. ((shewhart or shewart or deming) adj3 (cycle or method).tw.

11. rapid cycle.tw.

12. (quality adj2 collaborative?).tw.

13. (breakthrough adj3 (series or project or collaborative?)).tw.

14. (lean adj (approach or management or method? or methodology or thinking or enterpri#e or practice or philosophy or principles)).tw.

15. six sigma.tw.

16. or/1‐15

17. randomized controlled trial.pt.

18. random$.tw.

19. control$.tw.

20. intervention$.tw.

21. evaluat$.tw.

22. or/17‐21

23. animal/

24. human/

25. 23 not (23 and 24)

26. 24 not 25

27. 16 and 26

Searching other resources

For all included studies, we will conduct author and citation searches in Science Citation Index database.

Grey literature

We will search the Healthcare Management Information Consortium (HMIC) database for evaluations published in government reports and other grey literature. We will also search websites of relevant organisations for unpublished and ongoing studies. Examples include: the Institute for Healthcare Improvement (IHI) (www.ihi.org); the Agency for Healthcare Research and Quality (AHRQ) grants on‐line database (www.gold.ahrq.gov); the Robert Wood Johnson Foundation (www.rwjf.org); the National Institute for Health Research (NIHR) which includes the Service Delivery and Organisation Programme (SDO) (www.nihr.ac.uk and www.sdo.nihr.ac.uk ).

International trials registers

We will search the following registry search platforms to identify unpublished and ongoing studies: Current Controlled Trials metaRegister of Controlled Trials (www.controlled‐trials.com) and the International Clinical Trials Registry Platform Search Portal (www.who.int/trialsearch).

Reference lists

We will screen the reference lists of all included studies and any relevant reviews identified from the search.

Correspondence

We will write to the corresponding authors of all included studies and relevant reviews to assist with identification of unpublished and ongoing studies.

Data collection and analysis

Selection of studies

To ensure consistent application of inclusion criteria, all review authors independently screened a randomly selected subset of studies using a pre‐determined form and guidance. We discussed discrepancies (all authors) and, where required, revised the form and guidance to optimise consistency of screening decisions. Two authors (SB and MB) independently screened all titles and abstracts to identify potentially relevant papers. We retrieve full text copies of all potentially relevant papers, including those where the description of the intervention, study design, participants, or outcomes is insufficient to make a decision about inclusion. One review author collated retrieved papers (SB). One author applied inclusion criteria to the full text of potentially eligible studies (SB). A second author (either MB or JM) independently screened all included studies and those where there was potential uncertainty over screening decisions (i.e., those listed in the table of excluded studies). Disagreements were resolved by discussion between the two screening review authors and, where necessary, an arbiter (HB, MB, SG, JM or PW).

Data extraction and management

At least two review authors (SB and JM or MB) independently extracted data for each included study using a modified Cochrane EPOC Group data collection checklist that was pre‐determined and piloted by all review authors. Discrepancies in data extraction were resolved by discussion between the two review authors extracting data and, where necessary, an arbiter (HB, MB, SG, JM, or PW).

Characteristics of the CQI intervention

We will extract data on the following characteristics of the CQI intervention: scope of the CQI intervention; components of the CQI intervention (planned and actually implemented); components of the CQI intervention implementation strategy (planned and actually implemented). Where details are not included in the published study, or are unclear, we will contact authors requesting further information.

Scope of the CQI intervention

We will extract data on the scope of the CQI intervention, specifically whether the intervention was targeted at a single team per study site; multiple teams per study site but without an organisation‐wide intervention component; multiple teams per study site with an organisation‐wide intervention component; a single team per study site with an inter‐organisation intervention component (such as a quality improvement collaborative); or multiple teams per study site with an inter‐organisation intervention component.

Components of team‐based CQI

We will extract data on the intervention components using a framework for team‐based CQI to guide data extraction. The framework is adapted from the main elements of The Model for Improvement (Moen 1987; Institute for Healthcare Improvement 2008), FOCUS‐PDCA^® (McEachern 1992), and the team quality improvement sequence (TQIS) (Ovretveit 1999).

The framework includes the following intervention components.

1. Improvement team established, with health professional and non‐clinical representation from areas that contribute to delivery of the care process, to review the work process and plan and test changes.

2. Steps, methods, and tools

   1. Setting time‐specific, measurable aims for process improvement.

   2. Undertaking a process to map the steps in the work process targeted for change (e.g. using flow charts to pictorially represent the steps in a process).

   3. Identifying sources of variation in the work process, using data collection and analysis to identify the most important causes of variation (e.g. using Pareto diagrams to identify the major sources of variation in a process).

   4. Identifying the changes that are most likely to result in improvement, using knowledge of the process and sources of variation.

   5. Planning and testing changes to the process (e.g. using repeated cycles of a structured problem solving approach like the Plan–Do–Study–Act (PDSA) cycle, the Plan–Do–Check–Act (PDCA) cycle, or equivalent).

   6. Collecting and analysing process or outcome data at all points of the problem solving cycle to assess and review the effect of the changes (e.g. using control charts to monitor the performance of a process).

   7. Implementing process or system changes that lead to improvement.

We will extract data on specific tools used in the CQI process, categorising them as either tools to collect and manage data (e.g. cause and effect diagrams, control charts, Pareto charts), or tools to facilitate group processes and decision making (e.g. brainstorming; structured problem solving models; nominal group technique) (Gilman 1995; Ovretveit 2005).

Other components of team‐based CQI interventions

Additional intervention components will be classified using the Cochrane EPOC Group taxonomy. The additional components will be categorised as: 1) components delivered by the investigator that are considered to be part of the CQI intervention based on their definition of CQI, 2) additional interventions delivered by the investigator to supplement the CQI intervention, or 3) secondary interventions arising from the CQI process that are implemented by the improvement team to 'improve' the clinical care process.

Components of organisation wide CQI strategies

For studies evaluating organisation‐wide CQI interventions, we will treat the organisation level components as a potential effect modifier of team‐based CQI. We will extract descriptions of the organisation‐wide CQI components using a framework based on Lin and colleagues' commitment to quality improvement scale (Lin 2005). The scale includes four dimensions: leadership (articulated values, behaviours and abilities that support quality improvement, alignment of quality improvement with organisational goals); employee involvement in quality planning; human resources utilisation (training, rewards and recognition for quality improvement); customer satisfaction (assessment of customer needs and expectations and use of this data for improvement).

Components of inter‐organisation CQI strategies

We expect that most inter‐organisation strategies will fit the quality improvement collaborative model (Kilo 1998). We will use the criteria developed by Schouten and colleagues (Schouten 2008) to extract data on intervention components specific to quality improvement collaboratives. For reporting and analysis purposes, we will treat quality improvement collaboratives as a separate subgroup of team‐based CQI interventions (see 'Data synthesis' and 'Subgroup analysis and investigation of heterogeneity').

Implementation of the CQI intervention

We will extract descriptions of the strategy and methods used to implement team‐based CQI (planned and actual). The intensity of the implementation strategy will be treated as an effect modifier. We hypothesise that intensive training and support for use of CQI methods, such as that used in quality improvement collaboratives, will improve competency in the use of CQI methods and increase actual use of the methods, resulting in improved outcomes (Boonyasai 2007).

The intensity of implementation will be categorised independently, in a subjective manner by two review authors (SB and HB or PW) as high or moderate‐to‐low. Disagreements will be resolved by discussion between all review authors. Criteria are based on training and support provided in quality improvement collaboratives. All five of the following criteria must be met for implementation strategies to be rated as high intensity.

• Number of PDSA cycles or equivalent during intervention period (three or more cycles in 12 months or equivalent).

• Extent of training (four or more full‐day workshops in 12 months).

• Method of training (interactive educational meetings or workshops).

• Training content (covers quality improvement theory, methods, and tools including: the use of The Model for Improvement or equivalent; teamwork skills; measurement and analysis for improvement).

• Support throughout implementation process by experts in quality improvement or trained facilitators.

We will also extract data on the intervention duration, the size and composition of CQI teams, and frequency of CQI meetings.

Organisational context in which the intervention is used

We will extract descriptions of the healthcare setting and any measures of organisational support for quality improvement. Descriptions of the type and characteristics of the healthcare setting may include information about size, complexity, and disadvantage such as socioeconomic characteristics of the setting. In our analysis, we will include type of setting as a potential effect modifier, applying a simple categorisation of settings as 'hospital' or 'other' (see 'Subgroup analysis and investigation of heterogeneity'). While this does not reflect the diversity of healthcare settings, introducing change in complex, differentiated hospital settings poses particular challenges that may modify the effect of CQI. Organisational support for quality improvement will also be considered as a potential effect modifier. We will explore heterogeneity relating to organisational support across studies reporting a reliable and valid measure of organisational commitment to quality improvement (e.g. Shortell 2004), organisation‐wide implementation of quality improvement (e.g. Alexander 2007), organisational culture for quality improvement (e.g. Shortell 2004), or an equivalent instrument.

Complexity of targeted change

For each study we will report the purpose of the targeted change (e.g. appropriate management based on evidence‐based clinical practice guidelines, cost containment) and the nature of the desired change (e.g. reduction, increase, cessation).

The complexity of the targeted change will be categorised independently, in a subjective manner by two review authors (SB and HB or PW) as high, moderate or low. Disagreements will be resolved by discussion between all review authors. The categories will be based on:

• the number of changes required;

• the extent to which complex judgements or skills are necessary;

• the number of staff and professions involved in the change;

• the number of facilities or departments involved in the change.

Assessment of risk of bias in included studies

Two review authors (SB, JM) will independently assess the risk of bias for each included study, using the 'Risk of bias' tool described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008a) and additional criteria developed by the Cochrane EPOC Group (EPOC 2009). Any disagreements will be resolved by discussion involving a third review author (SG). Risk of bias will be considered in the analysis (see 'Data synthesis' and 'Sensitivity analysis') and fully described in the table of included studies.

For RCTs and CCTs we will assess risk of bias associated with the following six domains from the 'Risk of bias' tool: sequence generation; allocation concealment; blinding of participants, personnel, and outcome assessors; incomplete outcome data; selective outcome reporting; and other potential threats to validity (Higgins 2008a). We will include three additional domains that assess design‐specific threats to validity covered by the Cochrane EPOC group: imbalance of outcome measures at baseline; comparability of intervention and control group characteristics at baseline; and protection against contamination (EPOC 2009). Finally, for clustered study designs, we will assess the risk of bias associated with an additional domain: selective recruitment of participants. In studies eligible for inclusion in this review, the term 'participant' may refer to CQI teams, team members, and patients. Selective recruitment can occur at any of these levels when those responsible for recruitment have knowledge of the group allocation. We have written criteria for assessing selective recruitment in clustered designs (see 'Additional tables', Table 1), drawing on guidance from the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008a) and other sources.

Table 1

Criteria for judging risk of bias in recruitment of 'participants' in cluster designs

RECRUITMENT OF 'PARTICIPANTS'1 IN CLUSTER DESIGNS2
Was selective recruitment of CQI teams, team members, and patients, adequately prevented during the study?
Criteria for a judgement of 'YES' (i.e. low risk of bias).	'Participants' and those involved in the identification and/or recruitment of the 'participants' did not have knowledge of the group allocation because one of the following, or an equivalent, method was employed:
	'Participant' recruitment was completed prior to group assignment and the same 'participants' were followed up over time 'Participant' recruited post group assignment but: Carried out by a person who was blinded to the group allocation. Carried out by a person who is unaware of characteristics of the 'participants' (for example, effectiveness of teams, clinical characteristics of patients). Eligibility criteria were such that it was unlikely to be subverted by knowledge of group allocation (for example, all health professionals within a cluster included; all patients attending a hospital within a specified period of time included). Invited by mail questionnaire with identical information to 'participants' in the intervention and control arms.
Criteria for a 'NO' (i.e. high risk of bias).	Those involved in the identification and/or recruitment of the 'participants' may have had knowledge of the group allocation:
	'Participant' identification and/or recruitment undertaken post group allocation by a person who was unblinded and who may have had knowledge of characteristics of the of the 'participants'.
Criteria for a judgement of 'UNCLEAR' (uncertain risk of bias).	Insufficient information to permit judgement of 'Yes' or 'No'.

We will assess risk of bias in CBA studies using the same domains and criteria applied to RCTs (EPOC 2009).

For ITS studies, we will assess risk of bias associated with the following seven domains: intervention independent of other changes; shape of intervention effect pre‐specified; intervention unlikely to affect data collection; blinding of outcome assessors to intervention allocation; incomplete outcome data; selective outcome reporting; and other sources of bias (EPOC 2009).

We will use the criteria specified in the Cochrane Handbook for Systematic Reviews of Interventions and Cochrane EPOC Group guidance to judge whether a study has a low, high, or unclear risk of bias for each domain. For each included study, we will report our assessment of risk of bias for each domain together with a descriptive summary of the information that influenced our judgment. The overall risk of bias for each study will be judged according to the criteria described under 'Data synthesis'.

Measures of treatment effect

Outcomes

For each outcome category (dichotomous healthcare professional performance, continuous healthcare professional performance, dichotomous patient outcome, continuous patient outcome), we will include the outcome which has been identified as the primary outcome by the publication authors. When multiple primary outcomes are identified, we will rank the intervention effect estimates of these outcomes, as reported in the publication, and select the outcome with the median effect estimate. For RCTs, where possible, we will verify that the specified primary outcomes in the publications are consistent with those identified in the trial protocols and/or trial registry entries. When no primary outcome is specified, we will use that specified in the sample size calculation. If there are no sample size calculations, we will rank the intervention effect estimates, as reported in the publication, and select the median effect estimate. When there is an even number of outcomes, we will include the outcome whose effect estimate is ranked n/2, where n is the number of outcomes.

In results tables we will report whether we have used the primary outcome or the outcome with the median effect estimate.

All outcomes described by authors will be collected along with how they were measured (self‐report, chart abstraction, other objective, major or minor outcome).

Measures of treatment effect for RCTs, CCTs, and CBAs

For included outcomes, we will extract the intervention effect estimate reported in the publications along with its P value and confidence interval, and the method of statistical analysis used to calculate it. If an inappropriate statistical method has been used, we will not present the P value or confidence interval except if we are able to re‐analyse the data. In this circumstance, the P value will be annotated with the word 're‐analysed' in the results tables.

To make comparisons between studies, where possible, we will calculate the effect estimates listed in Table 2 and Table 3 (see 'Additional tables'). For binary outcomes, our primary effect estimate will be the relative risk and for continuous outcomes, our primary effect estimate will be the standardised mean difference. P values for these effect estimates will be calculated adjusting appropriately for the design where possible. The effect estimates will be standardised so that ratios greater than one, and differences between the intervention and comparator groups greater than zero, represent benefit for the intervention group.

Table 2

Effect estimators for dichotomous outcomes from RCTs, cluster‐RCTs, CCTs, and CBAs

Dichotomous outcomes
Relative risk
	RR = RPost,I/RPost,C where RPost,I=risk of the outcome in the intervention group post intervention, and RPost,C=risk of the outcome in the comparator group post intervention.
Relative risk adjusted for baseline
	RRAdj=(RPost,I/RPost,C)/(RPre,I/RPre,C) where RPost,I and RPost,C are defined above, and RPre,I and RPre,C represent, respectively, the risk of the outcome in the intervention and comparator groups pre intervention.
Risk difference
	RD=RPost,I – RPost,C where RPost,I and RPost,C are as defined above.

Table 3

Effect estimators for continuous outcomes from RCTs, cluster‐RCTs, CCTs, and CBAs

Continuous Outcomes
Standardised mean difference
	SMD=(MeanPost,I ‐ MeanPost,C)/SDPre,pooled where MeanPost,I and MeanPost,C are respectively the means in the intervention and comparator groups post intervention, and SDPre,pooled is the pooled standard deviation of the two groups pre intervention.
Relative percentage change post intervention
	RPC=((MeanPost,I ‐ MeanPost,C)/MeanPost,C)x100 where MeanPost,I and Mean Post,C are as defined above.
Difference in means post intervention
	DM=(MeanPost,I ‐ MeanPost,C) where MeanPost,I and MeanPost,C are as defined above.
Difference in mean change
	DMC=(MeanPost,I ‐ MeanPost,C) ‐ (MeanPre,I ‐ MeanPre,C) where MeanPost,I and MeanPost,C are as defined above, and MeanPre,I and MeanPre,C are respectively the means in the intervention and comparator groups pre intervention.

Measures of treatment effect for ITS and repeated measures ITS

For ITS studies we plan to report the following estimates, and their P values, from regression analyses which adjust for autocorrelation: (i) change in level of the outcome at the first point after the introduction of the intervention (immediate effect of the intervention), (ii) the post‐intervention slope minus the pre‐intervention slope (long term effect of the intervention).

Unit of analysis issues

Clustering

Studies where clusters of individuals are randomised (cluster‐RCTs, CCTs) or allocated (CBAs) to intervention groups, but where inference is intended at the level of the individual, need to be analysed appropriately to account for correlation of observations within clusters. Standard statistical methods assume independence of observations, and their use in these types of studies will generally result in artificially small P values and overly narrow confidence intervals for the effect estimates (Ukoumunne 1999). These studies can generally be re‐analysed by making assumptions about the intra‐cluster correlation (ICC). Estimates of ICC will be obtained from contacting authors, or imputed, using external estimates from similar studies (Ukoumunne 1999; Health Services Research Unit) or using general recommendations from empirical research (Campbell 2000). If this is not possible we will report the effect estimate and annotate the phrase 'unit of analysis error'.

ITS studies

ITS studies have been shown to be frequently incorrectly analysed (Ramsay 2003) with statistical methods which do not account for the autocorrelation of data points. We plan to re‐analyse results from such studies where we are able to obtain the data from the authors, or from data presented in graphs or tables in the publication. Time series regression analysis accounting for first order autocorrelation will be used to analyse the data and estimate a change in level of the outcome at the first point after the introduction of the intervention, and the post intervention slope minus the pre‐intervention slope (Ramsay 2003; Austvoll‐Dahlgren 2008). Confidence intervals will be calculated for these effect estimates.

Dealing with missing data

Attrition rates will be presented for the selected outcomes of the included studies. We plan to extract data which allows for at least an available case analysis from the included studies. We do not plan to undertake any imputation for missing outcome data.

Some studies may have missing summary data, for example, ICCs or standard deviations. We will impute these summary data where possible and report the assumptions we have made in the results tables. The affect of our choice of ICCs on the pooled effect estimate in any meta‐analysis will be investigated through sensitivity analyses.

Assessment of heterogeneity

We will make an assessment of whether to pool relative risks measuring the effectiveness of CQI compared to no intervention on healthcare professional performance by assessing if the interventions in the included trials within the two main categories of CQI (quality improvement collaboratives and other forms of team‐based CQI) are similar enough. We will assess statistical heterogeneity in any meta‐analyses undertaken by visually inspecting the scatter of effect estimates on the forest plots and by the I² statistic (Higgins 2003).

Assessment of reporting biases

In addition to undertaking an extensive search of the literature, we plan to search the International Clinical Trials Registry Platform Search Portal and Current Controlled TrialsmetaRegister of Controlled Trials to identify trials in addition to those found through the search. Investigators of these trials will be contacted for further information.

We plan to investigate if there is evidence of small study effects, if there are sufficient RCTs (at least ten), using funnel plots and formal statistical tests for funnel plot asymmetry. It is anticipated that there may be heterogeneity in effect estimates because of variability in the intervention. We therefore plan to use the statistical test proposed by Rücker et al (Rücker 2008), which can be used when there is substantial between‐study heterogeneity. While this test has not been evaluated for relative risks, there are currently no firm guidelines recommending a preferred test for this measure of treatment effect (Sterne 2008).

Data synthesis

For each comparison, we will report tables of summary statistics for each of the included studies (RCTs, CCTs, CBAs). The tables will include baseline and follow‐up summary statistics, effect estimates and their statistical significance, and information on effect modifiers and study design. We will summarise the effect estimates for the dichotomous healthcare performance outcome within comparison, type of intervention (quality improvement collaborative versus other team‐based CQI), and study design. This will include the presentation of the median effect estimate, inter‐quartile range, and the range. Graphs, such as box‐plots, will be used to graphically display this data. Only summaries of the dichotomous healthcare performance outcome will be made since this outcome will generally be a measure of adherence and can therefore be more consistently interpreted across studies compared to outcomes collected at the patient level. We have selected the dichotomous outcome in preference to the continuous outcome since dichotomous healthcare performance outcomes appear to be reported more frequently (Grimshaw 2004 (appendix 1); Jamtvedt 2006) and data used in the calculation of the SMD may be more difficult to obtain.

If possible, we plan to use meta‐analytical methods to pool relative risks measuring the effectiveness of CQI compared to no intervention on healthcare performance. Only RCTs which are judged as being at a low risk of bias, using the criteria below, will be included in these analyses. Studies will be pooled in subgroups defined by type of intervention (quality improvement collaborative versus other team‐based CQI) but will not be pooled across these subgroups, since we believe the variability of implementation intensity will be too large. We will make an assessment of the clinical and methodological diversity before deciding whether to undertake meta‐analyses (Assessment of heterogeneity). Random‐effects meta‐analysis (DerSimonian 1986) will be used to pool intervention effects because of anticipated clinical and methodological diversity. We plan to report an approximate 95% range of underlying effect estimates, based on the between‐study variance estimate, to provide some information on the spread of effect estimates (Higgins 2009). We will prepare a 'Summary of findings' table from the results of the meta‐analysis using the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Schünemann 2008).

RCTs will be judged as being at a low risk of bias if they meet all of the following criteria: a low risk of bias for sequence generation, a low or unclear risk of bias for allocation concealment, a low risk of bias for blinding, a low risk of bias for incomplete outcome data, a low or unclear risk of bias for selective outcome reporting, and a low risk of bias for 'participant' recruitment. Our reasons for including RCTs with unclear allocation concealment and selective outcome reporting are as follows. We anticipate that all included RCTs will be cluster‐RCTs because CQI interventions are targeted at groups rather than individuals. Allocation concealment is not usually a source of bias in cluster‐RCTs because the clusters are often randomised at once. Although empirical methodological research has demonstrated that selective outcome reporting is an important form of bias (Dwan 2008), in this review we expect it will be difficult to make a definitive judgment (either low or high risk of bias) for this domain since CQI will be evaluated across diverse clinical conditions, resulting in large variability in the outcomes collected. This will make it difficult, for example, to make comparisons between RCTs to identify outcomes that may have been selectively reported.

Results from ITS studies will be presented in tables for each comparison with summary statistics for each of the included studies, change in level of the outcome at the first point after the introduction of the intervention, post intervention slope minus the pre‐intervention slope, and information on effect modifiers. This will also be presented graphically using, for example, scatter plots of change in level versus change in slope with combinations of statistical significance denoted by different symbols.

Subgroup analysis and investigation of heterogeneity

We plan to investigate if the effect of CQI versus no intervention is modified by the type of intervention (quality improvement collaborative versus other team‐based CQI), intensity of the intervention implementation (high intensity versus low intensity), organisational support for CQI (supportive versus non‐supportive), team performance (high performing teams versus low or moderately performing teams), and setting (hospital versus other). This will be investigated visually (for example using box‐plots and bubble plots) and formally through subgroup analyses, or if there are enough trials, using random‐effects meta‐regression. Meta‐regression is the preferred approach since we are able to estimate the relative change in the intervention effect (and 95% CI) for each subgroup compared to a reference subgroup. Investigation of the latter four effect modifiers will be undertaken separately within the type of intervention. The categorisation of the levels of the effect modifiers 'organisational support for CQI' and 'team performance' can not be defined a priori since it is unknown what measurement tools will be used to assess these factors in the included RCTs. If possible, we will use information about the measurement tools to classify the effect modifiers. If this is not possible, we plan to use reported summary statistics and information about the scales to create a standardised score for each RCT. These scores will then be split into two groups based on the median standardised score.

A Bayesian method for performing random‐effects meta‐analysis will be used to investigate if baseline healthcare performance is predictive of the effectiveness of CQI on healthcare performance, if there are an adequate number of RCTs (Thompson 1997). This will be undertaken separately within type of intervention. This approach to investigating underlying healthcare performance as a source of heterogeneity corrects for the bias occurring from regression to the mean (Thompson 1997).

Sensitivity analysis

For the primary meta‐analysis comparing the effectiveness of CQI to no intervention on healthcare performance, we will undertake a sensitivity analysis to investigate how the pooled intervention effect is affected by the inclusion of RCTs at an unclear or high risk of bias. In addition, we will investigate if the pooled intervention effect is robust to our assumptions of ICCs.

Acknowledgements

We are very grateful to the Cochrane EPOC Review Group editorial team for their support, advice and assistance throughout the conception and preparation of this protocol, in particular Jeremy Grimshaw, Emma Tavender and Alain Mayhew. We acknowledge Steve Rogers' work conceiving and writing an earlier protocol for a Cochrane Review of CQI, which was published in 2001. We thank Lana Kluchareff for retrieving full text papers.

Notes

Withdrawn from publication for reasons stated in the review

What's new

Date	Event	Description
3 November 2017	Amended	Protocol for a Cochrane Review withdrawn from publication

History

Protocol first published: Issue 4, 2001

Date	Event	Description
15 May 2009	New citation required and major changes	Complete rewrite of protocol involving change to conceptualisation and design of the review.
20 August 2008	Amended	Converted to new review format.

Contributions of authors

SB wrote the first and subsequent drafts of the protocol, excluding the analysis plan which was written by JM. All authors contributed to conceptualising and designing the review, and provided comments on drafts of the protocol. PW and HB provided content advice in relation to CQI interventions.

Sources of support

Internal sources

• Departmental Doctoral Scholarship (SB), Australasian Cochrane Centre, Institute of Health Services Research, Monash University, Australia.

External sources

• No sources of support supplied

Declarations of interest

PW's post is funded by the North East Strategic Health Authority, UK, which is currently leading a major initiative to introduce the Toyota Production System into north east health care. HB is employed by the National Institute of Clinical Studies, National Health and Medical Research Council, Australia, which has been involved in funding and running quality improvement collaboratives. SB, SG, and JM have no known personal, professional or financial interests or affiliations that might lead to conflict of interest in the conduct of this review. SG's declarations of interest are described in full in The Cochrane Library (see About Cochrane, Cochrane Groups, Australasian Cochrane Centre, 'Declarations of interest')

Notes

The planned review outlined in this protocol has not been successfully converted into a full Cochrane Review within established timelines and for this reason has been withdrawn from the CDSR.

References

Additional references

• Alexander JA, Weiner BJ, Shortell SM, Baker LC. Does quality improvement implementation affect hospital quality of care?. Hospital Topics2007; Vol. 85, issue 2:3‐12. [PubMed]
• Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do It. New England Journal of Medicine2007; Vol. 357, issue 6:608‐13. [PubMed]
• Austvoll‐Dahlgren A, Aaserud M, Vist G, Ramsay C, Oxman AD, Sturm H, et al. Pharmaceutical policies: effects of cap and co‐payment on rational drug use. Cochrane Database of Systematic Reviews2008, issue Issue 1:Art. No.: CD007017. DOI:10.1002/14651858.CD007017. [PubMed]
• Barsness ZI, Shortell SM, Gillies EFX, Hughes JL, O'Brien D. The quality march: National survey profiles hospital quality improvement activities. Hospital and Health Networks1993; Vol. 67, issue 23:52‐5.
• Batalden PB, Stoltz PK. A framework for the continual improvement of health care: building and applying professional and improvement knowledge to test changes in daily work. Joint Commission Journal on Quality Improvement1993; Vol. 19, issue 10:424‐47. [PubMed]
• Berwick D. Continuous improvement as an ideal in health care. New England Journal of Medicine1989; Vol. 320, issue 1:53‐6. [PubMed]
• Berwick D, Godfrey BA, Roessner J. Curing health care: new strategies for quality improvement: a report on the National Demonstration Project on Quality Improvement in Health Care. 1st Edition. San Francisco: Jossey‐Bass, 1990:xxv, 287. [Google Scholar]
• Berwick DM. The Science of Improvement. JAMA2008; Vol. 299, issue 10:1182‐4. [PubMed]
• Boaden R, Harvey G, Moxham C, Proudlove N. Quality Improvement: theory and practice in healthcare. NHS Institute for Innovation and Improvement2008.
• Boerstler H, Foster RW, O'Connor EJ, O'Brien JL, Shortell SM, Carman JM, et al. Implementation of total quality management: conventional wisdom versus reality. Hospital and Health Services Administration1996; Vol. 41, issue 2:143‐59. [PubMed]
• Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA2007; Vol. 298, issue 9:1023‐37. [PubMed]
• Campbell M, Grimshaw J, Steen N. Sample size calculations for cluster randomised trials. Changing Professional Practice in Europe Group (EU BIOMED II Concerted Action). Journal of Health Services Research and Policy2000; Vol. 5, issue 1:12‐6. [PubMed]
• DerSimonian R, Laird N. Meta‐analysis in clinical trials. Controlled Clinical Trials1986; Vol. 7, issue 3:177‐88. [PubMed]
• Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE. 2008/09/05 2008; Vol. 3, issue 8:e3081. [PMC free article] [PubMed]
• Cochrane Effective Practice and Organisation of Care (EPOC) Group. Draft EPOC 'Risk of Bias' guideline, 13 February 2009. http://www.epoc.cochrane.org/en/handsearchers.html (accessed 27 April 2009).
• Ferlie E, Shortell SM. Improving the quality of health care in the United Kingdom and United States: A framework for change. The Milbank Quarterly2001; Vol. 79, issue 2:281‐315. [PMC free article] [PubMed]
• Geboers H, Grol R, Bosch W, Hoogen H, Mokkink H, Montfort P, et al. A model for continuous quality improvement in small scale practices. Quality in Health Care1999; Vol. 8, issue 1:43‐8. [PMC free article] [PubMed]
• Gilman SC, Lammers JC. Tool use and team success in continuous quality improvement: are all tools created equal?. Quality Management in Health Care1995; Vol. 4, issue 1:56‐61. [PubMed]
• Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technology Assessment2004; Vol. 8, issue 6:iii‐iv, 1‐72. [PubMed]
• Grol R, Berwick DM, Wensing M. On the trail of quality and safety in health care. BMJ2008; Vol. 336, issue 7635:74‐6. [PMC free article] [PubMed]
• Health Services Research Unit. Database of ICCs: Spreadsheet (Empirical estimates of ICCs from changing professional practice studies). http://www.abdn.ac.uk/hsru/epp/cluster.shtml (accessed ).
• Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ2003; Vol. 327, issue 7414:557‐60. [PMC free article] [PubMed]
• Higgins J, Altman D. Chapter 8: Assessing risk of bias in included studies. In: Higgins J, Green S editor(s). Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. [Google Scholar]
• Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. [Google Scholar]
• Higgins JPT, Thompson SG, Spiegelhalter DJ. A re‐evaluation of random‐effects meta‐analysis. Journal of the Royal Statistical Society Series A ‐ Statistics in Society 2009;172:137‐159. [PMC free article] [PubMed] [Google Scholar]
• Iles V, Sutherland K. Managing change in the NHS. Organisational change: a review for health care managers, professionals and researchers. London: National Co‐ordinating Centre for NHS Service Delivery and Organisation R & D; 2001 May. London: NHS SDO Programme.
• Institute for Healthcare Improvement. Improvement methods. www.ihi.org (accessed 3 September 2008).
• Institute of Medicine (U.S.) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press, 2001:xx, 337. [Google Scholar]
• Jamtvedt G, Young J, Kristoffersen D, O'Brien M, Oxman A. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews2006, issue Issue 2:Art. No.: CD000259. DOI: 10.1002/14651858.CD000259.pub2. [PubMed]
• Kilo CM. A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement's Breakthrough Series. Quality Management in Health Care1998; Vol. 6, issue 4:1‐13. [PubMed]
• Laffel G, Blumenthal D. The case for using industrial quality management science in health care organizations. JAMA1989; Vol. 262, issue 20:2869‐73. [PubMed]
• Lemieux‐Charles L, Murray M, Baker G, Barnsley J, Tasa K, Ibrahim S. The effects of quality improvement practices on team effectiveness: a mediational model. Journal of Organizational Behavior2002; Vol. 23, issue 5:533‐53.
• Lin MK, Marsteller JA, Shortell SM, Mendel P, Pearson M, Rosen M, et al. Motivation to change chronic illness care: results from a national evaluation of quality improvement collaboratives. Health Care Management Review2005; Vol. 30, issue 2:139‐56. [PubMed]
• Mannion R, Davies H, Scott T, Jung T, Bower P, Whalley D, et al. Measuring and assessing organisational culture in the NHS (OC1). London: National Co‐ordinating Centre for the National Institute for Health Research Service Delivery and Organisation Programme (NCCSDO); 2008 June.
• McEachern E, Schiff L, Cogan O. How to start a direct patient care team. Quality Review Bulletin1992:103‐22. [PubMed]
• McLaughlin CP, Kaluzny AD, editors. Continuous quality improvement in health care: theory, implementation and applications. 3rd Edition. Gaithersburg (MD): Aspen Publishers Inc., 2006:xii, 467. [Google Scholar]
• Meurer SJ, Counte MA, Rubio DM, Arrington B. Measuring a hospital's ability to improve. American Journal of Medical Quality2004; Vol. 19, issue 5:214‐22. [PubMed]
• Mittman BS. Creating the evidence base for quality improvement collaboratives. Annals of Internal Medicine2004; Vol. 140, issue 11:897‐901. [PubMed]
• Moen R, Nolan T. Process improvement. Quality Progress1987; Vol. 20, issue 9:62‐8.
• O'Brien JL, Shortell SM, Hughes EF, Foster RW, Carman JM, Boerstler H, et al. An integrative model for organization‐wide quality improvement: lessons from the field. Quality Management in Health Care1995; Vol. 3, issue 4:19‐30. [PubMed]
• Ovretveit J. A team quality improvement sequence for complex problems. Quality in Health Care1999; Vol. 8, issue 4:239‐46. [PMC free article] [PubMed]
• Ovretveit J, Bate P, Cleary P, Cretin S, Gustafson D, McInnes K, et al. Quality collaboratives: lessons from research. Quality and Safety in Health Care2002; Vol. 11, issue 4:345‐51. [PMC free article] [PubMed]
• Ovretveit J. What are the best strategies for ensuring quality in hospitals? Health Evidence Network synthesis report. Copenhagen: World Health Organisation; November 2003. Copenhagen: World Health Organisation.
• Ovretveit J. What are the advantages and limitations of different quality and safety tools for health care? Health Evidence Network synthesis report. Copenhagen: World Health Organisation; October 2005. Copenhagen.
• Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. International Journal of Technology Assessment in Health Care2003; Vol. 19, issue 4:613‐23. [PubMed]
• Rücker G, Schwarzer G, Carpenter J. Arcsine test for publication bias in meta‐analyses with binary outcomes. Statistics in Medicine2008; Vol. 27, issue 5:746‐63. [PubMed]
• Schouten LMT, Hulscher MEJL, Everdingen JJE, Huijsman R, Grol RPTM. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ2008; Vol. 336, issue 7659:1491‐4. [PMC free article] [PubMed]
• Schünemann HJ, Oxman AD, Higgins JPT, Vist GE, Glasziou P, Guyatt GH. Chapter 11: Presenting results and 'Summary of findings' tables. In: Higgins JPT, Green S editor(s). Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. [Google Scholar]
• Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: What it will take to accelerate progress?. The Milbank Quarterly. Blackwell Publishing Limited, 1998; Vol. 76, issue 4:593‐624. [PMC free article] [PubMed]
• Shortell S, Jones R, Rademaker A, Gillies R, Dranove D, Hughes E, et al. Assessing the impact of total quality management and organizational culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Medical Care2000; Vol. 38, issue 2:207‐17. [PubMed]
• Shortell SM, Marsteller JA, Lin M, Pearson ML, Wu SY, Mendel P, et al. The role of perceived team effectiveness in improving chronic illness care. Medical Care2004; Vol. 42, issue 11:1040‐8. [PubMed]
• Solberg LI, Brekke ML, Kottke TE, Steel RP. Continuous quality improvement in primary care: what's happening?. Medical Care1998; Vol. 36, issue 5:625‐35. [PubMed]
• Solberg LI. Improving medical practice: a conceptual framework. Annals of Family Medicine2007; Vol. 5, issue 3:251‐6. [PMC free article] [PubMed]
• Sterne JAC, Egger M, Moher D. Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S editor(s). Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. [Google Scholar]
• Stevens DP. How do we know?. Quality and Safety in Health Care2008; Vol. 17, issue 3:154‐5. [PubMed]
• Thompson SG, Smith TC, Sharp SJ. Investigating underlying risk as a source of heterogeneity in meta‐analysis. Statistics in Medicine. 1998/01/09 1997; Vol. 16, issue 23:2741‐58. [PubMed]
• Ukoumunne OC, Gulliford MC, Chinn S, Sterne JA, Burney PG. Methods for evaluating area‐wide and organisation‐based interventions in health and health care: a systematic review. Health Technology Assessment1999; Vol. 3, issue 5:iii‐92. [PubMed]
• Wagner C, Wal G, Groenewegen PP, Bakker DH. The effectiveness of quality systems in nursing homes: a review. Quality in Health Care2001; Vol. 10, issue 4:211‐7. [PMC free article] [PubMed]
• Weiner BJ, Alexander JA, Shortell SM. Leadership for quality improvement in health care: empirical evidence on hospital boards, managers, and physicians. Medical Care Research and Review1996; Vol. 53, issue 4:397‐416. [PubMed]
• Wensing M, Wollersheim H, Grol R. Organizational interventions to implement improvements in patient care: a structured review of reviews. Implementation Science2006; Vol. 1, issue 2. [PMC free article] [PubMed]
• Wilson T, Berwick DM, Cleary PD. What do collaborative improvement projects do? Experience from seven countries. Joint Commission Journal on Quality and Safety2003; Vol. 29, issue 2:85‐93. [PubMed]

---

Articles from The Cochrane Database of Systematic Reviews are provided here courtesy of Wiley

---

andersenyouripasted1985.blogspot.com

Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6486276/

Share this post