- Open Access
- Open Peer Review
Identifying models of delivery, care domains and quality indicators relevant to palliative day services: a scoping review protocol
Systematic Reviews volume 6, Article number: 100 (2017)
With an ageing population and increasing numbers of people with life-limiting illness, there is a growing demand for palliative day services. There is a need to measure and demonstrate the quality of these services, but there is currently little agreement on which aspects of care should be used to do this. The aim of the scoping review will be to map the extent, range and nature of the evidence around models of delivery, care domains and existing quality indicators used to evaluate palliative day services.
Electronic databases (MEDLINE, EMBASE, CINAHL, PsycINFO, Cochrane Central Register of Controlled Trials) will be searched for evidence using consensus development methods; randomised or quasi-randomised controlled trials; mixed methods; and prospective, longitudinal or retrospective case-control studies to develop or test quality indicators for evaluating palliative care within non-residential settings, including day hospices and community or primary care settings. At least two researchers will independently conduct all searches, study selection and data abstraction procedures. Meta-analyses and statistical methods of synthesis are not planned as part of the review. Results will be reported using numerical counts, including number of indicators in each care domain and by using qualitative approach to describe important indicator characteristics. A conceptual model will also be developed to summarise the impact of different aspects of quality in a palliative day service context. Methodological quality relating to indicator development will be assessed using the Appraisal of Indicators through Research and Evaluation (AIRE) tool. Overall strength of evidence will be assessed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system. Final decisions on quality assessment will be made via consensus between review authors.
Identifying, developing and implementing evidence-based quality indicators is critical to the evaluation and continued improvement of palliative care. Review findings will be used to support clinicians and policymakers make decisions on which quality indicators are most appropriate for evaluating day services at the patient and service level, and to identify areas for further research.
With an ageing population and increasing numbers of people with life-limiting illness, there is a growing demand for palliative care day services. Consequently, there is a need to measure the quality and value of these services, but there is little agreement on which aspects of care should be used to do this. Previous studies have indicated that day services may be associated with high patient satisfaction, but there is currently limited evidence demonstrating that services lead to significant improvements in symptom control or quality of life [1, 2]. Palliative day services have also been shown to differ substantially across settings, with wide variations in staffing structures, core components of care and models of delivery [3, 4]. Evaluating quality of palliative day services is essential for assessing care across diverse settings and for monitoring quality improvement approaches. Development and implementation of quality indicators for palliative day services is therefore a key priority.
Recent studies have used established methods to develop quality indicators in general palliative care [5, 6], but there is evidence to suggest that separate indicators may be required in different contexts, such as non-residential settings including day hospice, community or primary care settings . Since quality of care is a multidimensional concept, its assessment can require different measures. Indicators are not a direct measure of quality but are centred on evidence-based standards of care. In this manner, indicators can be used as quantitative measures to assess care processes, structures or outcomes and to monitor, evaluate and improve the quality of important clinical and support functions, as well as governance and management factors that can influence patient outcomes . Consequently, they allow for comparisons to be made over time and across settings and to set priorities and support patient choice. It is important to recognise that process and structural indicators can only be considered as valid when it can be demonstrated through high-quality evidence that they result in improved outcomes. These associations can derive from primary research, as well as reviews of existing evidence and expert consensus methods.
This planned review will inform the initial phases of the UK Consensus Project on Quality in Palliative Care Day Services, a project which aims to develop an evidence-based quality indicator set for day services using the multiphase RAND/UCLA appropriateness method .
The aim of this scoping review will be to map the extent, range and nature of the existing evidence around models of delivery, care domains and quality indicators used to evaluate palliative day services. In doing so, the review will also seek to detect gaps in this evidence and identify future directions for research in the area. Specific objectives will be (1) to identify existing quality indicators relevant to palliative day services and to evaluate the quality of their development and (2) to categorise existing quality indicators by care domain and model of delivery.
A scoping review methodology will be employed as these approaches take account of a broader range of study designs and methods than systematic reviews, which typically address research questions around the effectiveness of a specific intervention. Established methodological frameworks and recommendations will be used to guide the review process [10,11,12]. The review will include the following components or phases: (1) formulation of research questions, (2) identification and selection of relevant evidence, (3) extraction of potential quality indicators from existing evidence, (4) charting and synthesis of results including development of a conceptual model and (5) engagement and consultation with key stakeholders to ensure relevance of the conceptual model to current practice and to facilitate knowledge exchange processes. In addition to the use of this methodological framework, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P) checklist  will be followed to ensure preparation of a robust protocol (see Additional file 1).
A systematic search strategy will be carried out by combining key Medical Subject Heading (MeSH) terms including “Quality indicator”, “Patient-centered care”, “Quality criteria” and “Quality improvement”. The proposed MEDLINE search strategy is shown in Table 1. Key search terms will be identified though discussion between project team members and with other experts and stakeholders in the area. The fully developed systematic search strategy will be peer reviewed using the Peer Review of Electronic Search Strategies (PRESS) tool . Five electronic databases (Ovid MEDLINE via Ovid, EMBASE via Ovid, CINAHL via EBSCO, PsycINFO and the Cochrane Central Register of Controlled Trials) will be searched from January 2000 to the present day. Only evidence published since 2000 will be included to ensure relevance to current palliative care settings and day service delivery models. No language restrictions will be applied during searches. Hand searches of the reference lists from included articles and relevant systematic reviews will also be carried out. The search strategy will also use previous recommendations, which are based on a feasible and robust method of applying a systematic search to identify grey literature . Grey literature searches will be conducted using http://www.opengrey.eu/ and Google Scholar to identify projects in progress, practice guidelines and policy documents or reports not indexed in the electronic databases. Google Trends (https://www.google.com/trends/), a publically accessible resource providing data on geospatial and temporal patterns for user-specified terms, will be used to identify closely matching terms commonly used to locate information around quality of care. Given the broad scope of the review, only the first 50 returns from the website searches will be extracted. The websites of ten key palliative care and general healthcare organisations will also be searched.
Study screening and inclusion
Two researchers (SOC, MD) will independently conduct the database searches and import all potential evidence sources into an EndNote referencing software database (EndNote X7.3, Thomson Reuters, Ontario, Canada). The same researchers will then screen the title and abstract of records identified during the searches to assess relevance of the content in comparison with the above eligibility criteria. Non-relevant titles will be removed at this stage. Full-text copies of potentially relevant articles will be obtained and reviewed to determine inclusion using the same criteria. Agreement between reviewers at this stage will be determined using the κ statistic with an accepted cut-off point of above 0.60 used to indicate adequate agreement . A third reviewer will be used to resolve any disagreement or uncertainty around final inclusion (NMcC). All evidence excluded during screening will be recorded, along with the reason for exclusion. Reviewers will not be masked to author name or publication type during the screening process.
Study eligibility criteria
To broaden the scope of the review and reflect the diverse nature of day services, the review will include all evidence identified which involved the development or evaluation of quality indicators to assess palliative care within any non-residential setting, including day hospice and community or primary care settings. The review will focus on quality indicators for adults receiving care for any life-limiting conditions including cancer or uncontrolled symptoms due to cancer treatment, heart failure, pulmonary disease, dementia, end stage liver or renal disease and neurologic conditions including multiple sclerosis, motor neurone disease or Parkinson’s disease. We will not include indicators developed to assess care in patients undergoing surgical procedures, as measures of quality and care pathways are distinct in this group of patients. The following types of evidence will be considered: consensus development methods; randomised or quasi-randomised controlled trials; mixed methods; prospective, longitudinal or retrospective case-control studies. Clinical practice guidelines, consensus statements, systematic reviews and relevant reports or policy documents will also be included. Where indicators are identified from these latter sources, the review will attempt to ascertain the original development research. All included evidence will be required to identify or propose at least one potential quality indicator related to structures, processes or outcomes at the individual patient or service level of care. Quality indicators will be defined as any measure, able to be expressed as a fraction, which compares actual care against an ideal criterion [17, 18]. The review will therefore include not only evidence directly identifying candidate indicators but also evidence that discusses the outcome or event of interest without proposing the event as a quality indicator.
Two researchers (SOC, MD) will independently use a standard data abstraction form to chart the following information from the included evidence: author and date, country, document type (research paper, guideline, policy document etc.), target population/setting, indicator sets (domains), methods used in development, proposed methods of measurement and evidence of testing or implementation. Specific information on numerator and denominator descriptions, definitions and type of quality indicators will also be extracted and classified as structural, process or outcome indicators using the Donabedian Framework . According to this model, structural indicators include factors that affect the context in which care is delivered (i.e. organisational characteristics such as availability of equipment and services or staffing). Process indicators relate to the delivery of care (i.e. assessment and interventions), while outcome indicators are concerned with the effectiveness of care in terms of change in patient important outcomes (i.e. health status, satisfaction with care and overall quality of life). Indicators removed at each phase will also be verified by all authors.
The data abstraction form will be piloted using a sample of ten non-relevant quality indicator articles in order to standardise the abstraction process and make any required modifications. The quality of each indicator will be assessed by all reviewers using the Appraisal of Indicators through Research and Evaluation (AIRE) tool, which uses a combined score based on 20 items across four domains: purpose, relevance and organisational context; stakeholder involvement; scientific evidence, and additional evidence, formulation and usage . Each item will be rated on a 4-point Likert scale ranging from “1” (strongly disagree: confident that the criterion has not been fulfilled or no information was available) to “4” (strongly agree: confident that the criterion has been fulfilled). The mean score, reported as a percentage, will be determined for each domain, as well as for the overall score assigned. Indicators will be considered to be of good quality if they score 50% or more in all four domains. The quality of overall evidence will be assessed initially by two reviewers (SOC, MD) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system . This system appraises and summarises the quality and strength of recommendations across important outcomes. Overall evidence for each outcome will be rated as very low, low, moderate or high quality, based on study design and limitations, consistency of effect and the directness or generalisability of evidence . Final decisions on methodological quality will be reached through consensus between review authors. While appraisal of methodological quality is not considered as essential during conduct of a scoping review , the assessments planned as part of this review will provide valuable information for the evidence summary tables that will be produced for the initial phases of the RAND/UCLA appropriateness method .
All quality indicators extracted from the evidence sources, including their original descriptions, will be combined in a single excel file (Microsoft, Washington, USA). This list will then be scrutinised and non-relevant or duplicate indicators will be removed. The original and revised file will be reviewed and cross checked by a second reviewer, with any disagreement or uncertainty resolved via discussion with a third reviewer. The final list of quality indicators will be agreed by consensus among the reviewers. The indicators will then be grouped according to their proposed quality domain. These domains will be determined based on those used in previous quality indicator development projects identified during the review and following discussion with the review team (see Tables 2 and 3).
Due to the predicted breadth and heterogeneity of the identified evidence, meta-analyses and statistical methods of synthesis are not planned as part of the review. Quantitative results will be reported using numerical counts, including year of publication, country, evidence type and the number of indicators in each care domain. A qualitative approach will be used to describe indicator characteristics in terms of their method of reporting, evidence of reliability testing and the methodological quality of their development. All data processing, including calculation of the κ statistic to determine reliability of the search strategy, will be handled using SPSS (Version 21.0). Extracted information will be used to develop plain language evidence summary tables, with quantitative data displayed in graphical form. These will be used to guide the expert rating phase using the RAND/UCLA appropriateness method . To further aid the expert rating process, a concise and standardised list of indicator definitions will be developed by using a priori criteria to standardise the wording of the indicators in the evidence summary tables. For example, where indicator definitions include any IF and THEN statements, or reference to a specified condition, these will be removed. Any mention of exact timing (e.g. how many days before an intervention is started after referral) will also be removed as it is expected that this will differ for day services in comparison to other care settings. Since it is anticipated that some indicators will be generally comparable but may be measured in a number of ways or used in different care settings, the extracted information will be also used to develop a conceptual model, based on the Donabedian framework , and a modified version of the framework used in the OECD Health Care Quality Indicators Project . The model will summarise the impact of different aspects of quality in a palliative day care context and will help to identify areas of discussion among the expert panel during the consensus meeting phase of the RAND/UCLA appropriateness process . This model will also be used to identify gaps in the evidence or areas where indicators are not currently available to evaluate existing services. Particular focus will be placed on the context in which certain indicators have been developed or implemented to ensure relevance to current practice (see Additional file 2).
Identifying, developing and implementing evidence-based quality indicators is critical to the evaluation and continued improvement of palliative care. The planned scoping review will form part of a larger research process aimed at developing capacity in the evaluation and implementation of quality indicators in palliative day services. The review is intended to reflect and effectively summarise the diverse nature of palliative care delivered in day hospice, community or primary care settings. It is anticipated that a substantial number of indicators, distributed across different areas and models of care will be identified. Key stakeholders involved in the overall project (including hospice and day service leads, medical and nursing staff, healthcare professionals (physiotherapists, occupational therapists, complementary therapists, psychologists), volunteer staff and service users) will also be included in the summarising and reporting phase of the research process. Feedback provided by relevant stakeholders at this stage will be used to ensure relevance to current practice and to promote knowledge exchange and translation processes. The review findings will be used to identify gaps in the current evidence and to inform a planned systematic review. Findings will also be used to assist clinicians and policymakers ascertain whether existing indicators are appropriate for evaluating palliative day services at the patient and service level and to categorise how indicator measurement and quality improvement approaches fit within existing models of care.
Appraisal of Indicators through Research and Evaluation tool
Grading of Recommendations, Assessment, Development, and Evaluation
Medical Subject Heading
Peer Review of Electronic Search Strategies
Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols
Sandsdalen T, Grøndahl VA, Hov R, et al. Patients’ perceptions of palliative care quality in hospice inpatient care, hospice day care, palliative units in nursing homes, and home care: a cross-sectional study. BMC Palliat Care. 2016;15(1):79.
Goodwin DM, Higginson IJ, Myers K, et al. Effectiveness of palliative day care in improving pain, symptom control, and quality of life. J Pain Symptom Manage. 2003;25(3):202–12.
Davies E, Higginson IJ. Systematic review of specialist palliative day-care for adults with cancer. Support Care Cancer. 2005;13(8):607–27.
Goodwin DM, Higginson IJ, Myers K, et al. What is palliative day care? A patient perspective of five UK services. Support Care Cancer. 2002;10(7):556–62.
Claessen SJ, Francke AL, Belarbi HE, et al. A new set of quality indicators for palliative care: process and results of the development trajectory. J Pain Symptom Manage. 2011;42:169–82.
Leemans K, Cohen J, Francke AL, et al. Towards a standardized method of developing quality indicators for palliative care: protocol of the Quality indicators for Palliative Care (Q-PAC) study. BMC Palliat Care. 2013;12:6.
Iliffe S, Davies N, Manthorpe J, et al. Improving palliative care in selected settings in England using quality indicators: a realist evaluation. BMC Palliat Care. 2016;15:69.
Farquhar M. AHRQ quality indicators. In: Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses, vol. 45. Rockville: Agency for Healthcare Research and Quality (US); 2008.
Fitch K, Bernstein SJ, Aguilar MD, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND Corporation; 2001. http://www.rand.org/pubs/monograph_reports/MR1269.
Colquhoun HL, Levac D, O'Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4.
Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.
Pham MT Rajić A, Greig JD, et al. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.
Moher D, Shamseer L, Clarke M, PRISMA-P Group, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
McGowan J, Sampson M, Salzwedel DM, et al. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.
Boeker M, Vach W, Motschall E. Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough. BMC Med Res Methodol. 2013;13:131.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–82.
Dy SM, Kiley KB, Ast K, et al. Measuring what matters: top-ranked quality indicators for hospice and palliative care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association. J Pain Symptom Manage. 2015;49(4):773–81.
Lind S, Adolfsson J, Axelsson B, et al. Quality indicators for palliative and end of life care: a review of Swedish policy documents. BMJ Support Palliat Care. 2013;3(2):174–80.
Donabedian A. The quality of care: how can it be assessed? JAMA. 1998;121(11):1145–50.
Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.
Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. BMJ. 2008;336(7652):1049–51.
Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.
Arah OA, Westert GP, Hurst J, et al. A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care. 2006;18 Suppl 1:5–13.
We acknowledge members of The UK Consensus Project on Quality in Palliative Day Services team: Dr Kathy Armour, Marie Curie Hospice, West Midlands; Professor Joanna Coast, University of Bristol; Professor Joachim Cohen, Vrije Universiteit, Brussels; Professor Michael Donnelly; Queen’s University Belfast; Dr Anne Finucane, Marie Curie Hospice, Edinburgh; Dr Joan Fyvie, Marie Curie Hospice, Belfast; Dr Louise Jones, Royal Free & University College Medical School; Professor George Kernohan, Ulster University; Dr Kathleen Leemans, Vrije Universiteit, Brussels; Professor David Oxenham, County Durham and Darlington Foundation NHS Trust; Dr Paul Perkins, Sue Ryder, Leckhampton Court Hospice, Cheltenham.
The project is funded by Marie Curie.
Availability of data and materials
SOC and MD drafted the initial protocol. MD and NMcC reviewed and amended the protocol to develop the final version. All authors contributed to manuscript revisions and approved the final version.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRISMA-P checklist. Checklist including list of recommended items to include in a systematic review protocol. (29.7 Kb)
Proposed conceptual model mapping quality indicators by stage of care. A proposed conceptual model based on the Donabedian framework and a modified version of the framework used in the OECD Health Care Quality Indicators Project. (21.7 Kb)
About this article
Cite this article
O’Connor, S.R., Dempster, M. & McCorry, N.K. Identifying models of delivery, care domains and quality indicators relevant to palliative day services: a scoping review protocol. Syst Rev 6, 100 (2017) doi:10.1186/s13643-017-0489-4
- Palliative care
- Quality indicators
- Quality improvement
- Systematic review