This paper is in the following e-collection/theme issue:

Decision Support for Health Professionals (1086) Clinical Information and Decision Making (1294) Adoption and Change Management of eHealth Systems (654)
Adoption of Clinical Decision Support in Multimorbidity: A Systematic Review

Adoption of Clinical Decision Support in Multimorbidity: A Systematic Review

Adoption of Clinical Decision Support in Multimorbidity: A Systematic Review

Authors of this article:

Paolo Fraccaro1, 2, 3 Author Orcid Image ;   Mercedes Arguello Casteleiro4 Author Orcid Image ;   John Ainsworth1, 3 Author Orcid Image ;   Iain Buchan1, 2, 3 Author Orcid Image
Article Authors Cited by (70) Tweetations (13) Metrics

    Review

    1NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Institute of Population Health, The University of Manchester, Manchester, United Kingdom

    2Centre for Health Informatics, Institute of Population Health, The University of Manchester, Manchester, United Kingdom

    3Health eResearch Centre, The University of Manchester, Manchester, United Kingdom

    4Bio-Health Informatics Group, School of Computer Science, The University of Manchester, Manchester, United Kingdom

    Corresponding Author:

    Paolo Fraccaro, MEng

    Centre for Health Informatics

    Institute of Population Health

    The University of Manchester

    Vaughan House

    Portsmouth St

    Manchester, M13 9GB

    United Kingdom

    Phone: 44 161 275 1132

    Fax:44 161 275 1132

    Email: paolo.fraccaro@postgrad.manchester.ac.uk


    Background: Patients with multiple conditions have complex needs and are increasing in number as populations age. This multimorbidity is one of the greatest challenges facing health care. Having more than 1 condition generates (1) interactions between pathologies, (2) duplication of tests, (3) difficulties in adhering to often conflicting clinical practice guidelines, (4) obstacles in the continuity of care, (5) confusing self-management information, and (6) medication errors. In this context, clinical decision support (CDS) systems need to be able to handle realistic complexity and minimize iatrogenic risks.

    Objective: The aim of this review was to identify to what extent CDS is adopted in multimorbidity.

    Methods: This review followed PRISMA guidance and adopted a multidisciplinary approach. Scopus and PubMed searches were performed by combining terms from 3 different thesauri containing synonyms for (1) multimorbidity and comorbidity, (2) polypharmacy, and (3) CDS. The relevant articles were identified by examining the titles and abstracts. The full text of selected/relevant articles was analyzed in-depth. For articles appropriate for this review, data were collected on clinical tasks, diseases, decision maker, methods, data input context, user interface considerations, and evaluation of effectiveness.

    Results: A total of 50 articles were selected for the full in-depth analysis and 20 studies were included in the final review. Medication (n=10) and clinical guidance (n=8) were the predominant clinical tasks. Four studies focused on merging concurrent clinical practice guidelines. A total of 17 articles reported their CDS systems were knowledge-based. Most articles reviewed considered patients’ clinical records (n=19), clinical practice guidelines (n=12), and clinicians’ knowledge (n=10) as contextual input data. The most frequent diseases mentioned were cardiovascular (n=9) and diabetes mellitus (n=5). In all, 12 articles mentioned generalist doctor(s) as the decision maker(s). For articles reviewed, there were no studies referring to the active involvement of the patient in the decision-making process or to patient self-management. None of the articles reviewed adopted mobile technologies. There were no rigorous evaluations of usability or effectiveness of the CDS systems reported.

    Conclusions: This review shows that multimorbidity is underinvestigated in the informatics of supporting clinical decisions. CDS interventions that systematize clinical practice guidelines without considering the interactions of different conditions and care processes may lead to unhelpful or harmful clinical actions. To improve patient safety in multimorbidity, there is a need for more evidence about how both conditions and care processes interact. The data needed to build this evidence base exist in many electronic health record systems and are underused.

    JMIR Med Inform 2015;3(1):e4

    doi:10.2196/medinform.3503

    Keywords

    decision support systems, management; systematic review; multiple chronic diseases; multiple pathologies; multiple medications 


    Background

    Patients affected by multiple diseases are acknowledged to be one of the greatest challenges for modern health care, especially as populations age [1]. Different terms have been used in the medical literature to refer to coexistent pathologies; the most accepted are [2] comorbidity, defined in 1970 as “any distinct additional clinical entity that has existed or may occur during the clinical course of a patient who has the index disease under study” [3], and multimorbidity, later defined as “the coexistence of 2 or more chronic conditions, where 1 is not necessarily more central than others” [4]. In this review, we look at the presence of simultaneous medical conditions as the decision-making context without emphasizing the prominence of any 1 condition, and we follow the European General Practice Research Network, which defines multimorbidity as “any combination of chronic disease with at least 1 other disease (acute or chronic) or bio-psychosocial factor (associated or not) or somatic risk factor” [5]. Here we use multimorbidity in a broad sense to infer comorbidity as well.

    Impact of Multimorbidity on Public Health

    Estimates of the prevalence of multimorbidity emanate from countries with detailed primary care records. A national population study carried out in the Netherlands estimated an overall prevalence of 29.7%, ranging from 10% in those younger than 20 years to 78% in those older than 80 years [6]. Another population study in Scotland found an overall prevalence of 23.2% [7]. The prevalence of multimorbidity in a population increases with age [8]. Thus, a growing proportion of the population is affected by multimorbidity as populations age [9], particularly in countries with demographic patterns like the United Kingdom [10]. Previous studies [11-13] most commonly reported the following disease groups as likely to concur: cardiovascular diseases, diabetes mellitus, chronic kidney disease, chronic musculoskeletal disorders, chronic lung disorders, and mental ill health (particularly dementia and depression). There is also a greater burden of multimorbidity at younger ages (younger than 65 years) in deprived areas [7]. Thus, the public health and economic impact of multimorbidity is large [14]. In the United States, 84% of total health expenditure involves patients with more than 1 condition [15], whereas multimorbid patients in England account for the majority of primary care encounters [16] and this is expected to rise [15].

    Patient-Centered Care and Iatrogenic Risks

    The model of care in multimorbidity is changing, from a disease- and organization-centered approach [3] to patient-centered holistic care [17]. Patient-centeredness considers psychological and physiological needs, the patient’s concerns and priorities for care, self-care, and coordination between different professions and organizations, with primary care as an integrator [17]. Although patient-centered care is ideal for managing complex, chronic conditions, it is challenging to implement [5]; therefore, at present, patients with multimorbidity are commonly underserved by poorly integrated care systems [18,19]. This fragmentation reduces the safety, effectiveness, and efficiency of care [1]. A previous study reported that 10% to 20% of unscheduled care among older multimorbid adults is iatrogenic (eg, medication-related harm) [20].

    Self-Management and Continuity of Care

    The presence of simultaneous care plans for multiple conditions leads to confusion and, in turn, generates safety hazards. Clear care plans, blending clinical care with self-management, are essential in multimorbidity [21]. Such plans need to incorporate not only biomedical but also psychosocial factors, such as mood, informal care network, and patient income/finances [21]. Communication between patients/carers and health professionals over complex care plans can be challenging; therefore, self-care may be unreliable [21,22]. For example, it was estimated an average Medicare patient in the United States with 1 chronic condition sees 4 different health care professionals in 1 year and this number increases to 14 in the presence of 5 different chronic conditions [22]. Increasing the number of health professionals involved creates a combinatorial explosion of communication interfaces and, for the patient, greater difficulty in understanding, remembering, and recalling guidance [22]. The most common problems arising from this miscommunication are duplication of tests and harmful decisions made on the basis of incomplete or incorrect information [23,24]. Primary care and general practitioners, in particular, are seen as a nexus of coordination for complex care such as this [24]. However, general practitioner workload is increasing beyond its capacity with the rising prevalence of chronic diseases and multimorbidity [25].

    Clinical Practice Guidelines and Polypharmacy

    Clinical research processes tend to focus narrowly on a single disease, mechanism, or treatment. This parsimony is reflected in the production of clinical practice guidelines; therefore, interactions between diseases are barely touched upon in care pathways (even if they are referred to as “integrated”) [26]. More recently, organizations such as the National Institute for Health and Care Excellence (NICE) have started to address multimorbidity explicitly [27] and a framework of principles for system-wide action to deal with comorbidities has been developed in England by the Department of Health and the National Health Service (NHS) [28]. Most current guidelines, however, do not consider interactions between diseases or between treatments [29]. Therefore, potential synergies or conflicts between different care pathways operating for the same patient may be missed [30]. For example, Boyd et al [29] applied clinical practice guidelines to a hypothetical case of a 79-year-old woman with multiple moderately severe chronic conditions (osteoporosis, osteoarthritis, diabetes mellitus, hypertension, and chronic obstructive pulmonary diseases). The guideline-derived treatment regimen was extremely complex and potentially harmful—comprising 14 nonpharmacologic treatments (ie, self-monitoring, diet, exercise, health care visits, and laboratory testing) and 12 unique medications with 19 doses of medication per day [29]. Even in simpler cases, such as the presence of 2 diseases and 2 related treatments, researchers report 16 possible exposure patterns (half relevant for clinical practice guidelines) and 4 possible interaction combinations [26]. The 2 previous examples precipitate a “prescribing cascade” whereby drugs are prescribed to treat the adverse effects of other drugs, which is common in polypharmacy (the use of multiple medications) [31].

    Even the most primary care-focused of health care systems, such as the NHS [27], do not deal safely, effectively, or efficiently with multimorbidity and polypharmacy [32]. In the future, with an aging population, most health care system resources will be stretched by the care needs of multimorbid patients [33].

    Informatics Implications

    Multimorbid health care requires complex communication, analysis, summarization, and presentation of heterogeneous clinical information from multiple sources. It is acknowledged that electronic health records (EHRs), especially in primary care, require enhanced functionality to support decisions in these complex care processes [34]. A clinical decision support (CDS) system provides “clinicians, patients, or individuals with knowledge and person-specific or population information, intelligently filtered or presented at appropriate times, to foster better health processes, better individual patient care, and better population health.” [35]. Despite notable failures [36], CDS systems have the potential to improve clinical outcomes [37,38]. Indeed, multimorbidity was defined as one of the “grand challenges in clinical decision support” by Sittig et al [39]; however, this area remains underinvestigated [40,41], with concerns raised over the unmet needs in primary care [40]. Some of the current challenges are lack of provision of integrated clinical practice guidelines, disease-centered rather than patient-centered approaches, difficulties in embedding CDS into clinical systems, and lack of training to make best use of CDS [40]. EHRs and computerized physician order entry systems include rules that deal with drug-drug interactions; however, the whole patient context is not considered and the system may “overalert” physicians [42]. The overalert is another main risk in multimorbidity, which is known as alert fatigue: “the mental state provoked by managing too many irrelevant alerts from the system, which consume physical and psychological energies and lead the user to ignore also the relevant alerts resulting in potential harm for the patient” [43]. Prescribing alerts are especially important in polypharmacy, which has well-established risks of harm [44]. However, in some situations, multiple prescriptions are valid [30] and should not be dissuaded by inappropriate alerts. Context awareness, such as an “application’s ability to adapt to changing circumstances and respond according to the context of use” [45], is crucial in decision support interventions [46], especially for multimorbidity where many variables are in place. However, a greater understanding of which information and sociotechnical factors of the context have to be taken into account in health care has still to be established [47].

    Previous reviews have investigated specific aspects of CDS in multimorbidity; for example, prescribing in the elderly [48] and chronic disease management [49]. We could find no satisfactory review of CDS in multimorbidity from a technical/methodological perspective to guide the engineering of future systems. This interdisciplinary review plugs that gap.

    Aim and Objectives

    The aim was to review the current state of the art of CDS in multimorbidity. The objectives were to review the aspects of decision support target, contextual information about patients/practitioners/services, decision support technology, user interface considerations, decision maker(s), diseases, and evaluation. These aspects were analyzed to identify what works and what does not in CDS for multimorbidity, why systems failed to produce the expected outcomes, and what solutions might be adopted to address the problems.


    Overview

    This review follows the guidelines from Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework [50]. PRISMA consists of a list of 27 items and a 4-phase flow diagram to complete that was identified as the optimal way to perform and report systematic reviews and meta-analyses about health care interventions by an experienced group of researchers and methodologists [50].

    Eligibility

    Inclusion Criteria

    Studies that linked the concepts of multimorbidity, comorbidity, or polypharmacy to the concept of CDS, referring to the definitions provided previously, were selected from the literature.

    The studies included in this literature review are articles about CDS systems that (1) address general issues about the multimorbid population, (2) support care for a particular subpopulation of multimorbid patients, (3) manage comorbidities related to a main disease, (4) deal with multiple concurrent medications in multimorbid population, and (5) describe statistical or machine-learning methods for clinical prediction in which the multimorbid patients’ data feed the modeling/learning and an holistic approach is adopted.

    Exclusion Criteria

    Studies excluded from this literature review were about (1) CDS characteristics in general, without describing a CDS system in detail; (2) economic evaluations of CDS; (3) CDS systems in which multimorbidity was not a key feature; (4) social and operational research into CDS with no reference to clinical outcomes; (5) statistical or machine-learning approaches in which comorbidities were part of the model, but the patient-centered approach was not considered; and (6) systems that checked drug-drug interactions by means of simple rules, without taking into account multimorbidity or comorbidities.

    Information Sources

    MEDLINE and Scopus [51] were selected as the source indexes because they conform to the Cochrane requirement [52] of being “searched electronically both for words in the title or abstract and by using the standardized indexing terms, or controlled vocabulary, assigned to each record.” We used the PubMed [53] interface to MEDLINE, which also includes up-to-date citations not yet indexed in MEDLINE [52]. In addition, Scopus can use Medical Subject Headings (MeSH) terms for structured queries [54].

    Some target studies could only be found in the grey literature, such as theses and conference proceedings. Scopus allows search restrictions to some categories of grey literature, such as conference proceedings. This wider searching aimed to reduce publication bias.

    The searches were performed in December 2013 and January 2014 without any restriction in the publication date.

    Search

    For the search, we followed 3 key points from the Cochrane Handbook [52]:

    1. Searches should seek high sensitivity—this may result in poor precision.
    2. Too many different search concepts should be avoided, but a wide variety of search terms should be combined with “or” within each concept.
    3. Both free-text and subject headings should be used (eg, MeSH) [55].

    The focused clinical question that drove this systematic review was: What is the current level of adoption of CDS in multimorbidity? To answer this question, 3 different search concepts were selected:

    1. Decision support: it has many related MeSH descriptors, such as “decision support systems, management” or “decision support techniques.” Examples of individual hyponyms manually selected are “clinical decision support system,” “decision support software,” and “decision support tool.”
    2. Multimorbidity: it has zero related MeSH descriptors. Semantically, the closed concept comorbidity has 1 MeSH descriptor. Examples of synonyms manually selected are “concurrent conditions,” “multiple chronic diseases,” and “multiple pathologies.”
    3. Polypharmacy: it has just 1 MeSH descriptor and it should not be confused with the concept polypharmacology. Examples of synonyms manually selected are “several prescriptions,” “poly-prescriptions,” and “multiple medications.”

    In essence, the search created for the focused clinical question that drove this systematic review was based on 3 different search concepts and the hyponyms and synonym terms combined with “or.” Conceptually, our clinical query was the following: <decision support> AND (<multimorbidity> OR <polypharmacy>).

    In Scopus, the query created imposed that the relevant terms selected appear in the title, abstract, or keywords. The search yielded 954 articles (see Figure 1). Only literature from the social sciences, arts, and humanities was excluded from the search, and no restriction on the type of publication was imposed. Therefore, a wider selection of articles beyond the grey literature was retrieved.

    Because multimorbidity is underrepresented in MeSH (ie, no MeSH descriptor), we created a PubMed query that looked for the relevant terms selected in the title/abstract. The search created yielded 10,223 articles (ie, 10 times more document results than in Scopus). We investigated the origin of this high number by looking at the query as it appeared under search details when using the PubMed search engine. Some of the synonyms manually selected for multimorbidity were not recognized; thus, they were split up automatically by PubMed [56]. The query as executed in PubMed contained overly general terms, such as “conditions,” “diseases,” and “pathologies.” This severely affected the performance of the query. To further illustrate this, a subquery automatically generated by PubMed as part of the original query “decision support[Title/Abstract] AND conditions[Title/Abstract]” yielded 420 results. However, this subquery did not reflect our focused clinical question and it was very unlikely that it retrieve the articles that we were interested in. Because the quality of any search depends on all constituents, we recognized that our original query was unsuitable for the PubMed search engine. More importantly, we became aware of the difficulties of constructing a PubMed query tailored to the medical question being investigated. Next, we tried to create more focused queries for the PubMed search engine, such as “multimorbidity[Title/Abstract] AND decision support[Title/Abstract],” which yielded only 6 articles. The low number of articles retrieved made us suspect that a substantial amount of articles were missing.

    Knowing other researchers who were also conducting systematic reviews in the area of CDS, we thought of a search intended for a global evidence map [57] (ie, a search that sought to address broader questions about a particular area rather than focused clinical questions). Global evidence maps are similar to systematic reviews because they are both conducted in a formal process; however, the time taken for a global evidence map is longer (in excess of 2 years [58]). We were interested in decision support related to electronic clinical documentation systems and safety surveillance, so we created a new PubMed query to provide a better context of the area under study where our clinical query should focus on. The new query as it appeared under search details when using the PubMed search engine was “decision support[Title/Abstract] OR (safety[Title/Abstract] AND surveillance[Title/Abstract]) OR electronic health record[Title/Abstract] OR electronic medical record[Title/Abstract] OR electronic patient record[Title/Abstract].” This approach was adopted to guarantee the inclusion of all relevant articles even when CDS functionalities were described in studies about EHRs or safety surveillance systems without using CDS-related terms. To identify articles relevant to our focused clinical question, we used automatic annotation of all articles’ excerpts retrieved by the broader query using the hyponyms and synonym terms that appeared in the original clinical query for the 3 different search concepts originally selected. For details, see the next subsection.

    ‎
    Figure 1. Review flow diagram.
    View this figure

    Study Selection

    For the PubMed article excerpts retrieved out of the broad query, we modified the manual approach to screening citations for systematic reviews and adopted some automation. In the area of automated document classification, there is an emerging body of research that uses machine-learning methods to help with the process of citation screening (eg, [59]). We adopted a simpler, but well-founded, type of automation for prescreening PubMed article excerpts, which did not classify article excerpts as “relevant” or “irrelevant.” We used automatic annotation of text (title and abstracts) based on a controlled vocabulary known beforehand and tailored to our study. This method is analogous to the bioinformatics practice of relating genes that have been annotated using a common schema, such as an ontology [60,61], which is directly relevant to systematic reviews [62,63]. We note that the Cochrane Collaboration is considering ontologies to support evidence synthesis [62].

    The annotation was performed using a controlled vocabulary (ie, the list of the hyponyms and synonym terms manually created for our clinical query). This annotation can coexist with native annotations from PubMed article excerpts based on MeSH and/or authors’ keywords. The concrete details of the annotation process are out of the scope of this paper. Once the annotation was performed, a selection of articles were selected based on our clinical query “<decision support> AND (<multimorbidity> OR <polypharmacy>).” Thus, only article excerpts with at least 1 term in title/abstract related to decision support and at least 1 term in title/abstract related to multimorbidity or polypharmacy were identified as related to our clinical question.

    Articles obtained by the preceding procedure were combined with the ones from the Scopus search and, after removing duplicates, screened based on title and abstract. Relevant articles were assessed through full-text analysis to select the articles to be included in the systematic review.

    Data Collection Process and Data Items

    Overview

    A careful selection of relevant features was agreed on by the authors (PF, JA, and IB) and data on the following aspects were collected. A summary was generated for each data item and study.

    Decision Support Target

    This included clinical tasks supported by the CDS system: prevention, diagnosis, care pathway guidance (ie, management of patients according to clinical practice guidelines), medication (eg, prescription, medication review), patient education, patient self-management, and care continuity (supporting communication between health care professionals involved in multimorbid patients).

    Contextual Information

    Information was collected regarding the context processed or taken into account by the system to provide support: patient clinical notes (ie, demographics or family history), laboratory results, comorbidities, medications, clinical practice guidelines, and clinicians’ knowledge.

    Decision Support Technology

    These data included:

    1. Mode of delivery: type of technical solution used to deliver the system: desktop application, Web application, and mobile application.
    2. Methodology: methods used to perform the CDS intervention: data visualization techniques [64] (ie, providing intuitive interfaces to minimize errors); social network techniques; international communication and coding standard, such as Health Level Seven International (HL7) [65] to communicate information and Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) [66] or International Classification of Diseases (ICD9-10) [67], to store data; machine-learning techniques [68]; natural language processing [69]; knowledge-based systems [70] (ie, using rules or based on ontologies [71]); and mobile technologies.
    3. User interface considerations: reported considerations about techniques to enhance and make easier user utilization of the system: interactivity, user-centered design, summarization, and workflow graphs.
    4. Decision maker(s): user(s) of the CDS system: nurse, specialist doctor, pharmacist, generalist doctor (ie, general practitioner or family doctor), and patient.
    5. Diseases/conditions: CDS target conditions: obesity, diabetes mellitus, cardiovascular diseases, chronic respiratory diseases, chronic kidney disease, neurological conditions, mental health disorders, chronic musculoskeletal diseases, etc.
    6. Evaluation: type of evaluation of system’s effectiveness: uncontrolled impact studies (eg, surveys or health services measurements before/after CDS), controlled comparisons (eg, comparing new vs old/no CDS), and no evaluation.

    Synthesis of Results

    The results of the review were summarized in a table. The table was organized by the aspects of CDS defined previously and provides a qualitative summary for each included study. An additional quantitative summary to highlight general trends over time and patterns of evidence is also provided.


    Study Selection

    The search via Scopus retrieved 954 articles. We retrieved 17,145 articles via PubMed by using the broad search and annotation introduced previously; 79 results were recalled. After removing duplicates and screening the title and abstract, 50 articles were selected for in-depth analysis of the full text. A total of 20 studies were included in the review. The PRISMA process was followed and is reported in Figure 1.

    Study Characteristics

    Table 1 contains the summary of all data items collected for each study included in the review along with its reference.

    Table 1. Summary of collected items for included studies.
    AuthorsDecision support targetContextual informationDecision support methods/deliveryUser interface considerationsDecision maker/diseasesEvaluation
    Abidi [72]Pathways (merging clinical practice guidelines for different diseases into 1 personalized guideline)Patient clinical notes & clinical practice guidelinesKnowledge-based system (ontology) & international standards/———/——
    Abidi et al [73]Diagnosis & pathways (alignment of care pathways in a patient-specific comorbid combination) & patient educationPatient clinical notes & clinical practice guidelinesKnowledge-based system (ontology based)/desktop applicationInteractivity & summarizationGeneralist doctor/chronic cardiovascular diseasesControlled comparison-expert panel (revision by 2 generalist doctors and 1 specialist doctor)
    Bindoff et al [74]Medication (review)Patient clinical notes & medications & laboratory resultsKnowledge-based system (rule based)/ ——Pharmacists/—Controlled comparison-human vs system comparison (system identified more problems)
    Dassen et al [75]Medication (prescription)Patient clinical notes & medications & clinical practice guidelines & comorbidities & laboratory resultsKnowledge-based system (ontology based) & international standards/desktop applicationInteractivity & workflow graphsSpecialist doctor/cardiovascular diseases—
    de Wit et al [76]Medication (review)Patient clinical notes & clinical practice guidelines & clinician knowledge & laboratory resultsKnowledge-based system (rule based)/——Nurses/other (home care for the elderly)No evaluation
    Duke et al [77]Medication (review)Medications & clinician knowledgeKnowledge-based system & data visualization techniques & natural language processing/Web platformInteractivity & summarizationSpecialist doctor & generalist doctor/—Controlled comparison-new vs old system (same accuracy but decreasing in time of 60%)
    Farkas et al [78]Diagnosis (comorbidities)Patient clinical notesNatural language processing/———/obesityControlled comparison-simulations (Fβ=1 score of 97% for classification based on textual evidence and 96% for intuitive judgments; Fβ=1 score of 76% for classification based on textual evidence and 67% for intuitive judgments)
    Georg et al [79]Medication (prescription)Patient clinical notes & clinical practice guidelinesKnowledge-based system (rule based)/——Generalist doctor/cardiovascular diseases—
    Grando et al [80]Medication (prescription)Patient clinical notes & clinical practice guidelinesKnowledge-based system (ontology based)/——Generalist doctor/chronic respiratory diseases & diabetes & cardiovascular diseases & chronic musculoskeletal diseases & others—
    Jafarpour et al [81]Pathways (merging clinical practice guidelines for different diseases into 1 personalized guideline)Patient clinical notes & clinical practice guidelines & clinician knowledgeKnowledge-based system (ontology based)/——Generalist doctor/cardiovascular diseasesNo evaluation
    Martínez-García et al [82]Care continuity & pathwaysPatient clinical notes & clinical practice guidelines & clinician knowledgeInternational standards & social network techniques/Web application (linked to electronic health record)—Nurse, generalist doctor, specialist doctor/—Controlled comparison-survey (positively judged)
    Michel et al [83]Medication (prescription)Patient clinical notes & clinical practice guidelines & clinician knowledge & medications & laboratory results & comorbiditiesKnowledge-based system & data visualization techniques & international standards/desktop application (linked to electronic health record)SummarizationGeneralist doctor/chronic pain (opioid treated)—
    Naureckas et al [84]Diagnosis & pathwaysPatient clinical notes & clinical practice guidelinesKnowledge-based system & data visualization techniques/desktop application (linked to electronic health record)User-centered designGeneralist doctor/child obesity and related diseases (eg, diabetes, cardiovascular diseases, chronic kidney disease)Impact evaluation-service performance metrics & survey
    Riaño et al [85]Diagnosis & medication (prescription) & pathways (developing a personalized treatment) & preventionPatient clinical notes & clinical practice guidelines & clinician knowledgeKnowledge-based system (ontology based) & international standards/desktop application (linked to electronic health record)—Generalist doctor/home care in long-term conditions (eg, obesity, diabetes, cardiovascular diseases, chronic respiratory diseases, chronic kidney disease, neurological conditions, mental health disorders, chronic musculoskeletal diseases)Controlled comparison-survey (positively judged)
    Riaño et al [86]Medication (prescription)Patient clinical notes & clinician knowledgeKnowledge-based system (rule based)/——Generalist doctor/cardiovascular diseases & diabetesControlled comparison-expert panel (results validated by a generalist doctor)
    Suojanen et al [87]DiagnosisPatient clinical notes & clinician knowledgeMachine learning/——Specialist doctor/chronic neurological diseasesControlled comparison-simulation (out of 30 cases: false positive rate=19%; false negative rate=23%)
    Vallverdú et al [88]Medication (prescription)Patient clinical notes & clinician knowledgeKnowledge-based system (rule based)/desktop application—Generalist doctor/cardiovascular diseases & diabetesControlled comparison-expert panel (agreement with output from the system 100%-20/20)
    Wicht et al [89]Diagnosis (comorbidities)Patient clinical notes & clinician knowledgeKnowledge-based system + data visualization techniques/Web platformInteractivitySpecialist doctor/other (cancer)Controlled comparison-expert panel (agreement with output from the system 84%-26/31)
    Wilk et al [90]Pathways (merging clinical practice guidelines for different diseases into 1 personalized guideline)Patient clinical notes & clinical practice guidelines & clinician knowledgeKnowledge-based system (rule-based constraint logic programming)/—Workflow graphsGeneralist doctor/other (duodenal ulcer, transient ischemic attack)—
    Wilk et al [91]Pathways (alerting physicians about possible adverse interactions between 2 concurrent clinical practice guidelines)Patient clinical notes & clinical practice guidelinesKnowledge-based system (rule-based constraint logic programming [92])/——Specialist doctor & generalist doctor/chronic neurological & gastrointestinal diseases—

    Results of Individual Studies

    Table 2 shows the frequency distribution of the categories of aspects of CDS reported.

    Most articles reviewed focused on 1 of 3 clinical tasks: medication (n=10), clinical guidance (n=8), and diagnosis (n=6). From a methodological point of view, knowledge-based systems were the most frequently used (n=17). To further illustrate this, Riaño et al [85] described a CDS system that targets 3 decisions and uses knowledge-based systems. The authors developed a system that (1) provided patient-centered recommendations to better manage chronic diseases in the home setting and (2) used EHRs to refine an ontology, which described relevant concepts from clinical practice guidelines and the literature for 19 chronic diseases. The goal of this study was a patient-tailored ontology that contained patient-specific concepts that could be used to verify the diagnosis entered into the system. Starting from the personalized ontology, general treatment plans and patient management instructions could be combined into an individual plan. For multimorbid patients, a semiautomatic procedure was applied that involved the system’s end-user. The system was able also to identify preventive opportunities by looking for anomalous circumstances, such as diagnosis inconsistent with other information or information missing which should always be presented alongside other information.

    Abidi et al [73] presented a system that helped doctors in diagnosis and management of patients (2 decision support targets) and used an ontology (knowledge centric) that was able to align clinical pathways for the multimorbid patient.

    In the articles reviewed, medication was the main theme by far. This clinical task had the most contextualized input data and appeared as prescription (n=7) and medication review (n=3). Michel et al [83] developed a system that aimed to guide the generalist doctor through a summary of comprehensive relevant information (patient information, patient medication, patient laboratory results, and patient comorbidities) and suggested the optimal opioid treat for chronic pain. Dassen et al [75] developed a system, along the lines of Michel at al [68], considering comprehensive relevant information (patient information, patient medication, patient laboratory results, and patient comorbidities) and used an ontology to support cardiologists’ prescriptions according to clinical practice guidelines. The articles by de Wit et al [76] focused on medication review and their system was intended to support safer care for the elderly. The system was capable of processing extracts of clinical data from electronic prescribing systems and EHRs (containing patient medication, patient conditions, and patient laboratory results) and alerted nurses about potentially harmful situations.

    Another prevalent theme was the possible interaction between concurrent clinical practice guidelines for multimorbid patients. For example, Abidi et al [73] and Jafarpour et al [81] used ontologies to develop systems to merge 2 concurrent clinical practice guidelines into a comorbid personalized guideline. Jafarpour et al [81] carried out this task by creating an ontology that collected merging criteria obtained from clinical experts. Wilk et al [90,91] used constraint logic programming to identify and mitigate possible adverse interactions between clinical practice guidelines. One system alerted doctors about possible hazards and suggested how to mitigate them [91]. Martinez-Garcia et al [82] developed a system that improved clinical guidance by providing health care professionals with relevant information from clinical practice guidelines and also supported communication between health care professionals. Their system (1) was directly linked to the EHR through HL7, an international standard for interoperability in health care and (2) adopted social networking techniques to enhance the continuity of care through a Web platform—it provided relevant patient information and performed safety checks according to clinical practice guidelines.

    Some studies addressed the diagnosis of comorbidities for patients affected by an index condition/disease. For example, Farkas et al [78] used natural language processing applied to clinical notes to diagnose comorbidities in obese patients. Suojanen et al [87] used machine learning (causal Bayesian networks) for diagnosis of multiple concurrent neuropathies.

    Table 2. Synthesis of occurrences’ numbers and references for collected data items.
    Theme and categoryFrequencyReferences
    Decision support task


    		Prevention1[85]

    		Diagnosis6[73,78,84,85,87,89]

    		Pathway8[72,73,81,82,84,85,90,91]

    		Medication10[74-77,79,80,83,85,86,88]

    		Patient education1[73]

    		Continuity of care1[82]

    		Self-management0—
    Decision support technology


    		Data visualization techniques4[77,83,84,89]

    		Social network techniques1[82]

    		International standards5[73,75,82,83,85]

    		Machine learning1[87]

    		Natural language processing2[77,78]

    		Knowledge-based system17[72-77,79-81,83-86,88-91]

    		Mobile technologies0—
    Contextual information


    		Patient clinical notes19[72-76,78-91]

    		Laboratory results4[74-76,83]

    		Comorbidities2[75,83]

    		Medications4[74,75,77,83]

    		Clinician knowledge11[76,77,81-83,85-90]

    		Clinical practice guidelines13[72,73,75,76,79-85,90,91]
    Decision maker(s)


    		Nurse2[76,82]

    		Specialist doctor6[75,77,82,87,89,91]

    		Generalist doctor13[73,77,79-86,88,90,91]

    		Pharmacist1[74]

    		Patient0—

    		Not specified2[72,78]
    Diseases


    		Obesity3[78,84,85]

    		Diabetes5[80,84-86,88]

    		Cardiovascular diseases9[73,75,79-81,84-86,88]

    		Chronic respiratory diseases2[80,85]

    		Chronic kidney diseases2[84,85]

    		Chronic neurological conditions3[85,87,91]

    		Mental health disorders1[85]

    		Chronic musculoskeletal diseases2[80,85]

    		Other8[76,80,83-85,89,90]

    		Not specified4[72,74,77,82]

    User interface considerations


    		Interactivity4[73,75,77,89]

    		User-centered design1[84]

    		Summarization3[73,77,83]

    		Workflow graphs2[75,90]

    		Not specified13[73,74,76,78-82,85-88,91]
    Evaluation


    		Impact evaluation (service performance metrics)1[84]

    		Impact evaluation (survey)1[84]

    		Controlled comparison (expert panel)4[72,86,88,89]

    		Controlled comparison (survey)2[82,85]

    		Controlled comparison (simulation)2[78,87]

    		Controlled comparison (human vs system)1[74]

    		Controlled comparison (new vs old system)1[77]

    		No evaluation2[76,81]

    		Not specified7[72,75,79,80,83,90,91]

    For the decision makers, generalist doctors were the most cited users of the CDS systems (n=13) followed by specialist doctors (n=6). No articles reported the patient as the decision maker. The system that appeared to involve the largest number of decision makers was described by Martinez-Garcia et al [82], in which nurses, specialist doctors, and generalist doctors were end users.

    For disease, many articles considered multiple diseases (eg, [80,84-86,91]), with Riano et al [85] reporting 19 chronic conditions.

    For user interface considerations, most articles (n=13) did not provide details about the user interface. Where this information was provided, interactivity (n=4) [73,75,77,89] and summarization (n=3) [73,77,83] were the most cited features, whereas workflow graphs [75,90] were seldom mentioned. Only Naureckas et al [84] presented a CDS system that adopted a user-centered design with prompts and forms that helped generalist doctors to develop more effective behaviors for supporting diagnosis, management, and screening of comorbidities for children with obesity.

    Regarding type of evaluation, some articles reported effectiveness objectively, including controlled comparisons (n=9) or impact evaluations (n=1). The articles that conducted surveys about their systems achieved positive judgments about the outcome provided [82,85]. In terms of accuracy, many studies reported good performance [87-89]. Duke et al [77] compared UpToDate [93] with a new system that had the same accuracy, but improved (by 60%) timeliness of decision. Bindoff et al [74] compared a CDS system with expert pharmacists when performing a medication review; overall, the system identified more potential problems than the human experts did.


    Summary of Evidence

    Overview

    This literature review found a modest number of articles addressing CDS and multimorbidity—an evidence base disproportionately small in comparison to the need for decision support.

    The Lack of Patient-Centered Approaches

    Most articles dealt with CDS targets that (1) were narrowly defined in terms of comorbidities around an index condition or (2) considered patient comorbidities only during prescription for a specific condition. Thereby, only a few of the studies reviewed referred to multimorbidity using a patient-centered approach, which is the ideal [5]. Riano et al [85] adopted a comprehensive approach to integrated care; however, user intervention is necessary to personalize treatments when multimorbidity is present.

    Combination of Clinical Practice Guidelines

    An important challenge of multimorbidity in CDS is the combination of clinical practice guidelines in a nonharmful way [39]. We found some studies that addressed this explicitly. An interesting solution was introduced by Jafarpour et al [81], which created an ontology with merging criteria provided by experts. Although rigorous evidence is lacking, to exploit physicians’ “clinical mind-lines,” such as “tacit guidelines that are internalized and collectively reinforced from the experience and discussion with colleagues and patients to embody the complex and flexible knowledge needed in practice” [94], seems the only solution. However, all systems described in the articles reviewed tended to simplify the analysis by referring to only 2 concurrent clinical practice guidelines. This scenario is too simplistic for the current reality because multimorbid patients often face more than 2 simultaneous pathologies [29].

    Continuity of Care

    Discontinuity of care between different health professionals is an important source of safety problems, which is highly relevant to multimorbidity considering the large numbers of professionals involved. Yet only 1 article [82] considered this aspect. Prevalent technologies such as social media may foster communication across different clinical settings. There is a notable gap in the evidence base here.

    No Self-Management Interventions

    Self-management is key in multimorbidity [21]. In the articles reviewed, no CDS interventions for multimorbid patient self-management were found. Similarly, we noticed the absence of mobile technologies for CDS in multimorbidity.

    Methodological Considerations

    From a methodological point of view, knowledge-based systems were most commonly reported. Data-driven methods, such as machine-learning techniques, were barely used in the reviewed studies, with just 1 study adopting them [87].

    The Technological Interoperability Shortfalls

    Multimorbidity is composed of interacting variables; therefore, systems need to be aware of as many contextual factors as possible to deliver relevant support and information [95]. Emerging international standards, such as HL7, are supposed to enable interoperability in health care; however, only 1 article reviewed used HL7, the system developed by Martinez-Garcia et al [82].

    The Need for More Rigorous Evaluations

    Evaluations of usability and effectiveness of systems are key to avoiding patient harm and waste in health care systems [96]. The so called “e-iatrogenesis” [97] arising from information systems has more potential pitfalls when there are multiple conditions. Rigorous evaluations are needed to test systems before and after their deployment to guarantee patient safety [98,99]. We found a lack of rigorous evaluations of effectiveness and usability here, which is consistent with the overall state of CDS [36] research. Patient safety needs to be assured by rigorous evaluation, not only of the underlying software/technologies but also of their real-world interaction with users [100]. The expected approaches to evaluating human-computer interaction [101,102] were not found in the articles we examined.

    Limitations

    This review has several limitations. First, only Scopus and PubMed sources were searched—other relevant material may exist in the grey literature. Second, the titles and abstracts of the articles selected are anchored to the terms included in the 3 thesauri—some articles may have been missed if other synonyms were used. Third, it was not possible to find studies covering all aspects of CDS we considered—some aspects, such as the evaluation of the effectiveness and usability, were quite sparsely covered, but this is a general weakness of the CDS literature [36]. Finally, we did not follow the traditional systematic review process for all searches. However, we are confident that our strategy guaranteed the inclusion of all relevant articles about the topic. There is an ongoing discussion of what should and should not be automated in systematic reviews, particularly to strike the right balance between depth and timeliness [103]. Here we took the middle ground, using computational methods to make a more “concept-complete” search tractable. Therefore, this review may contribute to the ongoing discussion about semiautomated prescreening of medical literature while preserving rigorous methods of evidence synthesis.

    Implications for Future Research and Conclusions

    This review shows how multimorbidity is understudied in CDS, yet this is an area of public health and clinical importance that should be a prime target for CDS research.

    There are already many technologies in health care and industry relevant to dealing with the complexity of multimorbid decision support. Kawamoto et al [104] argue that wider adoption of international terminologies (eg, SNOMED CT) and electronic health record standards can lead to better CDS, tapping into the vast amount of data produced in routine clinical practice for multimorbid patients. Moreover, technical frameworks [105] were already proposed for a “shared and informed decision making” in industry that with appropriate adjustments could be used to enhance continuity of care in multimorbidity. In addition, the absence of any substantial articles dealing with self-care for people affected by multiple conditions was remarkable given the rapid growth in connected/consumer health and its inevitable influence on CDS in the future.

    Multimorbidity is a relatively new field of clinical research and more evidence is needed to support CDS in this area. This underpinning knowledge is, however, challenging. For example, patients with multiple conditions or on multiple medications are often excluded from clinical trials [106]. However, EHRs afford the possibility of observational studies important for understanding multimorbid disease risks, care processes, and care outcomes. Such observational data have established value in decreasing the prescribing cascade and other iatrogenesis [107]. Automation of care pathways/processes that are poorly understood, such as merging guidelines [30], may lead to unhelpful or harmful clinical actions. The informatics challenge herein is to build the evidence base about multimorbid care while engineering more supportive/directive clinical information systems incrementally. The clinical epidemiology and health services research must be interwoven with the systems development. Gathering more clinical evidence and getting more involvement from patients and health professionals is central to finding a technological approach to managing multimorbidity and enhancing patient safety. At the same time, rigorous evaluation of all sociotechnical and human-computer interaction aspects of produced CDS interventions is certainly a priority for the future.

    Patients with multiple conditions are one of the most important groups for health care systems to understand and evolve to serve [33]. There are multiple dynamics in which CDS and health informatics can contribute in meeting this challenge: (1) using EHR data to understand multimorbidity and plug a relatively sparse evidence base, (2) coproducing care decisions between patients and practitioners in the face of complexity and uncertainty, and (3) blending n-of-1 patient experiments/experience with evidence about the “average patient like Mrs X...” It is hard to conceive of such complexity being tamed by today’s EHR interfaces, punctuated by blizzards of alerts and dashboards. Future CDS may be part of an integrated health avatar [108]: “the electronic representation of an individual’s health as directly measured or inferred by statistical models or clinicians.” To achieve such integration, however, there is a pressing need for more realistically complex CDS research, particularly in multimorbidity.

    Acknowledgments

    Funded by the national Institute for Health Research Greater Manchester Primary Care Patient Safety Translational Research Centre (NIHR GM PSTRC) and the MRC Health eResearch Centre, Farr Institute, UK (MR/K006665/1). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.

    Conflicts of Interest

    None declared.

    1. Boyd CM, Fortin M. Future of multimorbidity research: How should understanding of multimorbidity inform health system design? Public Health Rev. Public Health Rev 2010;32:451-474 [FREE Full text]
    2. Almirall J, Fortin M. The coexistence of terms to describe the presence of multiple concurrent diseases. JOC 2013:4-9. [CrossRef]
    3. Feinstein AR. The pre-therapeutic classification of co-morbidity in chronic disease. Journal of Chronic Diseases 1970 Dec;23(7):455-468. [CrossRef]
    4. Fortin M, Soubhi H, Hudon C, Bayliss EA, van den Akker M. Multimorbidity's many challenges. BMJ 2007 May 19;334(7602):1016-1017 [FREE Full text] [CrossRef] [Medline]
    5. Le Reste JY, Nabbe P, Manceau B, Lygidakis C, Doerr C, Lingner H, et al. The European General Practice Research Network presents a comprehensive definition of multimorbidity in family medicine and long term care, following a systematic review of relevant literature. J Am Med Dir Assoc 2013 May;14(5):319-325. [CrossRef] [Medline]
    6. van den Akker M, Buntinx F, Metsemakers JF, Roos S, Knottnerus JA. Multimorbidity in general practice: prevalence, incidence, and determinants of co-occurring chronic and recurrent diseases. J Clin Epidemiol 1998 May;51(5):367-375. [Medline]
    7. Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. The Lancet 2012 Jul;380(9836):37-43. [CrossRef]
    8. Fortin M, Hudon C, Haggerty J, Akker M, Almirall J. Prevalence estimates of multimorbidity: a comparative study of two sources. BMC Health Serv Res 2010;10:111 [FREE Full text] [CrossRef] [Medline]
    9. Uijen AA, van de Lisdonk EH. Multimorbidity in primary care: prevalence and trend over the last 20 years. Eur J Gen Pract 2008;14 Suppl 1:28-32. [CrossRef] [Medline]
    10. Dunnell K. The changing demographic picture of the UK: national statistician's annual article on the population. Popul Trends 2007(130):9-21. [Medline]
    11. Zhang X, Decker FH, Luo H, Geiss LS, Pearson WS, Saaddine JB, et al. Trends in the prevalence and comorbidities of diabetes mellitus in nursing home residents in the United States: 1995-2004. J Am Geriatr Soc 2010 Apr;58(4):724-730. [CrossRef] [Medline]
    12. Stevens LA, Li S, Wang C, Huang C, Becker BN, Bomback AS, et al. Prevalence of CKD and comorbid illness in elderly patients in the United States: results from the Kidney Early Evaluation Program (KEEP). Am J Kidney Dis 2010 Mar;55(3 Suppl 2):S23-S33. [CrossRef] [Medline]
    13. Yaffe K, Ackerson L, Kurella Tamura M, Le Blanc P, Kusek JW, Sehgal AR, Chronic Renal Insufficiency Cohort Investigators. Chronic kidney disease and cognitive function in older adults: findings from the chronic renal insufficiency cohort cognitive study. J Am Geriatr Soc 2010 Feb;58(2):338-345 [FREE Full text] [CrossRef] [Medline]
    14. Taylor AW, Price K, Gill TK, Adams R, Pilkington R, Carrangis N, et al. Multimorbidity - not just an older person's issue. Results from an Australian biomedical study. BMC Public Health 2010;10:718 [FREE Full text] [CrossRef] [Medline]
    15. Anderson G. Chronic Care: Making the Case for Ongoing Care. Princeton, NJ: Robert Wood Johnson Foundation; 2010.   URL: http://www.rwjf.org/en/research-publications/find-rwjf-research/2010/01/chronic-care.html [accessed 2014-02-21] [WebCite Cache]
    16. Salisbury C, Johnson L, Purdy S, Valderas JM, Montgomery AA. Epidemiology and impact of multimorbidity in primary care: a retrospective cohort study. Br J Gen Pract 2011 Jan;61(582):e12-e21 [FREE Full text] [CrossRef] [Medline]
    17. Schofield T. Patient-centered medicine: rransforming the clinical method. BMJ 1995 Dec 09;311(7019):1580-1580. [CrossRef]
    18. Vogeli C, Shields AE, Lee TA, Gibson TB, Marder WD, Weiss KB, et al. Multiple chronic conditions: prevalence, health consequences, and implications for quality, care management, and costs. J Gen Intern Med 2007 Dec;22 Suppl 3:391-395 [FREE Full text] [CrossRef] [Medline]
    19. Zulman DM, Asch SM, Martins SB, Kerr EA, Hoffman BB, Goldstein MK. Quality of care for patients with multiple chronic conditions: the role of comorbidity interrelatedness. J Gen Intern Med 2014 Mar;29(3):529-537. [CrossRef] [Medline]
    20. Budnitz DS, Shehab N, Kegler SR, Richards CL. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med 2007 Dec 4;147(11):755-765. [Medline]
    21. Bayliss EA, Ellis JL, Steiner JF. Barriers to self-management and quality-of-life outcomes in seniors with multimorbidities. Ann Fam Med 2007;5(5):395-402 [FREE Full text] [CrossRef] [Medline]
    22. Eichner J, Blumenthal D. Medicare in the 21st century: building a better chronic care system. SSRN Journal 2003 Jan. [CrossRef]
    23. Kuperman GJ. Health-information exchange: why are we doing it, and what are we doing? J Am Med Inform Assoc 2011;18(5):678-682 [FREE Full text] [CrossRef] [Medline]
    24. Haggerty JL. Ordering the chaos for patients with multimorbidity. BMJ 2012;345:e5915. [Medline]
    25. Østbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M, Michener JL. Is there time for management of patients with chronic diseases in primary care? Ann Fam Med 2005;3(3):209-214 [FREE Full text] [CrossRef] [Medline]
    26. Weiss CO, Varadhan R, Puhan MA, Vickers A, Bandeen-Roche K, Boyd CM, et al. Multimorbidity and evidence generation. J Gen Intern Med 2014 Apr;29(4):653-660. [CrossRef] [Medline]
    27. National Institute for Health and Care Excellence. 2013 May 14. NICE a proven success story   URL: http://www.nice.org.uk/newsroom/news/NICEAProvenSuccessStory.jsp [accessed 2014-04-05] [WebCite Cache]
    28. Department of Health. A framework of principles for system-wide action. 2014.   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​307143/​Comorbidities_framework.​pdf [accessed 2014-05-01] [WebCite Cache]
    29. Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005 Aug 10;294(6):716-724. [CrossRef] [Medline]
    30. Guthrie B, Payne K, Alderson P, McMurdo ME, Mercer SW. Adapting clinical guidelines to take account of multimorbidity. BMJ 2012;345:e6341. [Medline]
    31. Koper D, Kamenski G, Flamm M, Böhmdorfer B, Sönnichsen A. Frequency of medication errors in primary care patients with polypharmacy. Fam Pract 2013 Jun;30(3):313-319. [CrossRef] [Medline]
    32. Gaal S, Verstappen W, Wolters R, Lankveld H, van Weel C, Wensing M. Prevalence and consequences of patient safety incidents in general practice in the Netherlands: a retrospective medical record review study. Implement Sci 2011;6:37 [FREE Full text] [CrossRef] [Medline]
    33. Dobrow M. International Journal of Integrated Care. 2009 Mar 19. Book review: Caring for people with chronic conditions: a health system perspective   URL: http://www.ijic.org/index.php/ijic/article/view/298/594 [accessed 2014-02-21] [WebCite Cache]
    34. Krist AH, Beasley JW, Crosson JC, Kibbe DC, Klinkman MS, Lehmann CU, et al. Electronic health record functionality needed to better support primary care. J Am Med Inform Assoc 2014 Jan;21(5):764-771. [CrossRef] [Medline]
    35. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007;14(2):141-145 [FREE Full text] [CrossRef] [Medline]
    36. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
    37. Cantrill S. Computers in patient care: the promise and the challenge. Queue 2010;8(8):20 [FREE Full text] [CrossRef]
    38. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012 Jul 3;157(1):29-43. [CrossRef] [Medline]
    39. Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al. Grand challenges in clinical decision support. J Biomed Inform 2008 Apr;41(2):387-392 [FREE Full text] [CrossRef] [Medline]
    40. Grace A, Mahony C, O’Donoghue J, Heffernan T, Molony D, Carroll T. Evaluating the effectiveness of clinical decision support systems: the case of multimorbidity care. Journal of Decision Systems 2013 Apr;22(2):97-108. [CrossRef]
    41. Smith SM, Soubhi H, Fortin M, Hudon C, O'Dowd T. Interventions for improving outcomes in patients with multimorbidity in primary care and community settings. Cochrane Database Syst Rev 2012;4:CD006560. [CrossRef] [Medline]
    42. Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14(1):29-40 [FREE Full text] [CrossRef] [Medline]
    43. van der Sijs H. Drug safety alerting in computerized physician order entry: unraveling and counteracting alert fatigue. 2009.   URL: http:/​/www.​researchgate.net/​publication/​241860650_Drug_safety_alerting_in_computerized_physician_order_entry_Unraveling_and_counteracting_alert_fatigue [accessed 2014-02-21] [WebCite Cache]
    44. O'Brien R, Wyke S, Guthrie B, Watt G, Mercer S. An 'endless struggle': a qualitative study of general practitioners' and practice nurses' experiences of managing multimorbidity in socio-economically deprived areas of Scotland. Chronic Illn 2011 Mar;7(1):45-59. [CrossRef] [Medline]
    45. Kjeldskov J, Skov M. Supporting work activities in healthcare by mobile electronic patient records. In: Computer Human Interaction SE-20. Berlin: Springer Berlin Heidelberg; 2004:191-200.
    46. Burstein F, Holsapple CW. Decision support systems in context. Inf Syst E-Bus Manage 2008 Feb 29;6(3):221-223. [CrossRef]
    47. Bricon-Souf N, Newman CR. Context awareness in health care: a review. Int J Med Inform 2007 Jan;76(1):2-12. [CrossRef] [Medline]
    48. Yourman L, Concato J, Agostini JV. Use of computer decision support interventions to improve medication prescribing in older adults: a systematic review. Am J Geriatr Pharmacother 2008 Jun;6(2):119-129. [CrossRef] [Medline]
    49. Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, CCDSS Systematic Review Team. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implement Sci 2011;6:92 [FREE Full text] [CrossRef] [Medline]
    50. Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6(7). [CrossRef]
    51. Scopus.   URL: http://www.scopus.com/ [accessed 2014-02-21] [WebCite Cache]
    52. Lefebvre C, Manheimer E, Glanville J. The Cochrane Collaborative. 2011. Cochrane Handbook for Systematic Reviews of Interventions   URL: http://www.cochrane.org/handbook [WebCite Cache]
    53. PubMed.   URL: http://www.ncbi.nlm.nih.gov/pubmed [accessed 2014-02-21] [WebCite Cache]
    54. Sullo E. Scopus. J Med Libr Assoc 2007 Jul;95(3):367-368. [CrossRef]
    55. Lowe HJ, Barnett GO. Understanding and using the medical subject headings (MeSH) vocabulary to perform literature searches. JAMA 1994 Apr 13;271(14):1103-1108. [Medline]
    56. Dorsch J, DeGroote S. An Introduction to PubMed. 2000 Jul.   URL: http://www.uic.edu/depts/lib/lhsp/temp/pubmed.htm [accessed 2014-09-24] [WebCite Cache]
    57. Bragge P, Clavisi O, Turner T, Tavender E, Collie A, Gruen RL. The Global Evidence Mapping Initiative: scoping research in broad topic areas. BMC Med Res Methodol 2011;11:92 [FREE Full text] [CrossRef] [Medline]
    58. Bero L, Busuttil G, Farquhar C, Koehlmoos TP, Moher D, Nylenna M, et al. Measuring the performance of the Cochrane library. Cochrane Database Syst Rev 2012;12:ED000048. [CrossRef] [Medline]
    59. Wallace C, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence-based practice center. In: Proceedings of the 2nd ACM SIGHIT symposium on International health informatics-IHI ’12. 2012 Presented at: 2nd ACM SIGHIT symposium on International health informatics-IHI ’12; Jan 20-23, 2012; New York p. 819-824. [CrossRef]
    60. Dennis G, Sherman BT, Hosack DA, Yang J, Gao W, Lane HC, et al. DAVID: Database for Annotation, Visualization, and Integrated Discovery. Genome Biol 2003;4(5):P3. [Medline]
    61. Huntley RP, Harris MA, Alam-Faruque Y, Blake JA, Carbon S, Dietze H, et al. A method for increasing expressivity of Gene Ontology annotations using a compositional approach. BMC Bioinformatics 2014;15:155 [FREE Full text] [CrossRef] [Medline]
    62. Mavergames C, Oliver S, Becker L. Systematic Reviews as an interface to the web of (trial) data: Using PICO as an ontology for knowledge synthesis in evidence-based healthcare research. 2013 Presented at: 3rd Workshop on Semantic Publishing (SePublica). 10th Extended Semantic Web Conference; May 26, 2013; Montpellier, France p. 22-26.
    63. Huang Z, ten Teije A, van Harmelen F. SemanticCT: a semantically-enabled system for clinical trials. Process Support and Knowledge Representation in Health Care SE-2 2013;8268:25. [CrossRef]
    64. Jacko JA. The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. Boca Raton, FL: Lawrence Erlbaum Assoc Inc; 2007.
    65. Health Level Seven International.   URL: http://www.hl7.org.uk/ [accessed 2014-02-21] [WebCite Cache]
    66. National Library of Medicine Online. SNOMED Clinical Terms (SNOMED CT)   URL: http://www.nlm.nih.gov/research/umls/Snomed/snomed_main.html [accessed 2014-02-21] [WebCite Cache]
    67. World Health Organization. International Classification of Diseases   URL: http://www.who.int/classifications/icd/en/ [accessed 2014-02-22] [WebCite Cache]
    68. Mehryar M, Afshin R, Ameet T. Foundations of Machine Learning (Adaptive Computation and Machine Learning series). Cambridge, MA: The MIT Press; 2012.
    69. Jurafsky D, Martin JH. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Upper Saddle River, NJ: Pearson Prentice Hall; 2009.
    70. Brachman RJ, Levesque HJ. Knowledge Representation and Reasoning. San Diego, CA: Elsevier/Morgan Kaufmann Publishers; 2004.
    71. Suárez-Figueroa C, Gómez-Pérez A, Motta E, Gangemi A. Ontology Engineering in a Networked World. Berlin: Springer; 2012.
    72. Abidi S. A conceptual framework for ontology based automating and merging of clinical pathways of comorbidities. In: Knowledge Management for Health Care Procedures. Berlin: Springer; 2009:55-66.
    73. Abidi S, Cox J, Shepherd M, Abidi SR. Using OWL ontologies for clinical guidelines based comorbid decision support. 2012 Presented at: 45th Hawaii International Conference on System Sciences; Jan 4-7, 2012; Maui, HI p. 3030-3038. [CrossRef]
    74. Bindoff I, Stafford A, Peterson G, Kang BH, Tenni P. The potential for intelligent decision support systems to improve the quality and consistency of medication reviews. J Clin Pharm Ther 2012 Aug;37(4):452-458. [CrossRef] [Medline]
    75. Dassen RM, Gorgels APM, Berendsen A, Dijk WA, de Clercq PA, Hasman A, et al. Guideline assessment and implementation in congestive heart failure. Computers in Cardiology 2003:331-334. [CrossRef]
    76. de Wit HA, Mestres Gonzalvo C, Hurkens KP, Mulder WJ, Janknegt R, Verhey FR, et al. Development of a computer system to support medication reviews in nursing homes. Int J Clin Pharm 2013 Oct;35(5):668-672. [CrossRef] [Medline]
    77. Duke JD, Li X, Grannis SJ. Data visualization speeds review of potential adverse drug events in patients on multiple medications. J Biomed Inform 2010 Apr;43(2):326-331 [FREE Full text] [CrossRef] [Medline]
    78. Farkas R, Szarvas G, Hegedus I, Almási A, Vincze V, Ormándi R, et al. Semi-automated construction of decision rules to predict morbidities from clinical texts. J Am Med Inform Assoc 2009 Apr;16(4):601-605 [FREE Full text] [CrossRef] [Medline]
    79. Georg G, Séroussi B, Bouaud J. Synthesis of elementary single-disease recommendations to support guideline-based therapeutic decision for complex polypathological patients. Stud Health Technol Inform 2004;107(Pt 1):38-42. [Medline]
    80. Grando A, Farrish S, Boyd C, Boxwala A. Ontological approach for safe and effective polypharmacy prescription. AMIA Annu Symp Proc 2012;2012:291-300 [FREE Full text] [Medline]
    81. Jafarpour B, Abidi S. Merging disease-specific clinical guidelines to handle comorbidities in a clinical decision support setting. Artificial Intelligence in Medicine 2013;7885:28-32. [CrossRef]
    82. Martínez-García A, Moreno-Conde A, Jódar-Sánchez F, Leal S, Parra C. Sharing clinical decisions for multimorbidity case management using social network and open-source tools. J Biomed Inform 2013 Dec;46(6):977-984. [CrossRef] [Medline]
    83. Michel M, Trafton J, Martins S, Wang D, Tu S, Johnson N, et al. , Improving Patient Safety Using ATHENA-Decision Support System Technology: The Opioid Therapy for Chronic Pain Experience. Adv Patient Saf New Dir Altern Approaches (Vol 4 Technol Medicat Safety) 2008;4. [Medline]
    84. Naureckas SM, Zweigoron R, Haverkamp KS, Kaleba EO, Pohl SJ, Ariza AJ. Developing an electronic clinical decision support system to promote guideline adherence for healthy weight management and cardiovascular risk reduction in children: a progress update. Transl Behav Med 2011 Mar;1(1):103-107 [FREE Full text] [CrossRef] [Medline]
    85. Riaño D, Real F, López-Vallverdú JA, Campana F, Ercolani S, Mecocci P, et al. An ontology-based personalization of health-care knowledge to support clinical decisions for chronically ill patients. J Biomed Inform 2012 Jun;45(3):429-446 [FREE Full text] [CrossRef] [Medline]
    86. Riano D, Collado A. Model-based combination of treatments for the management of chronic comorbid patients. Artificial Intelligence in Medicine 2013;7885:11-16. [CrossRef]
    87. Suojanen M, Andreassen S, Olesen KG. A method for diagnosing multiple diseases in MUNIN. IEEE Trans Biomed Eng 2001 May;48(5):522-532. [CrossRef] [Medline]
    88. López-Vallverdú JA, Riano D, Collado A. Rule-based combination of comorbid treatments for chronic diseases applied to hypertension, diabetes mellitus and heart failure. In: Process Support and Knowledge Representation in Health Care. Berlin: Springer; 2013:30-41.
    89. Wicht A, Wetter T, Klein U. A web-based system for clinical decision support and knowledge maintenance for deterioration monitoring of hemato-oncological patients. Comput Methods Programs Biomed 2013 Jul;111(1):26-32. [CrossRef] [Medline]
    90. Wilk S, Michalowski M, Michalowski W, Hing MM, Farion K. Reconciling pairs of concurrently used clinical practice guidelines using Constraint Logic Programming. AMIA Annu Symp Proc 2011;2011:944-953 [FREE Full text] [Medline]
    91. Wilk S, Michalowski W, Michalowski M, Farion K, Hing MM, Mohapatra S. Mitigation of adverse interactions in pairs of clinical practice guidelines using constraint logic programming. J Biomed Inform 2013 Apr;46(2):341-353 [FREE Full text] [CrossRef] [Medline]
    92. Balduccini M, Son TC. Logic Programming, Knowledge Representation, and Nonmonotonic Reasoning. Berlin: Springer Berlin Heidelberg; 2011.
    93. UpToDate. Evidence-based clinical decision support at the point of care   URL: http://www.uptodate.com/home [accessed 2014-02-21] [WebCite Cache]
    94. Gabbay J, le May A. Practice-Based Evidence for Healthcare: Clinical Mindlines. London: Taylor & Francis; 2010.
    95. Riedmann D, Jung M, Hackl WO, Stühlinger W, van der Sijs H, Ammenwerth E. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med Inform Decis Mak 2011;11:35 [FREE Full text] [CrossRef] [Medline]
    96. Magrabi F, Ong MS, Runciman W, Coiera E. Patient safety problems associated with heathcare information technology: an analysis of adverse events reported to the US Food and Drug Administration. AMIA Annu Symp Proc 2011;2011:853-857 [FREE Full text] [Medline]
    97. Weiner JP, Kfuri T, Chan K, Fowles JB. "e-Iatrogenesis": the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc 2007;14(3):387-8; discussion 389 [FREE Full text] [CrossRef] [Medline]
    98. Denham CR, Classen DC, Swenson SJ, Henderson MJ, Zeltner T, Bates DW. Safe use of electronic health records and health information technology systems: trust but verify. J Patient Saf 2013 Dec;9(4):177-189. [CrossRef] [Medline]
    99. Kilbridge PM, Welebob EM, Classen DC. Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems. Qual Saf Health Care 2006 Apr;15(2):81-84 [FREE Full text] [CrossRef] [Medline]
    100. Coiera E. Interaction design theory. Int J Med Inform 2003 Mar;69(2-3):205-222. [Medline]
    101. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012 Dec;45(6):1202-1216 [FREE Full text] [CrossRef] [Medline]
    102. Phansalkar S, Edworthy J, Hellier E, Seger DL, Schedlbauer A, Avery AJ, et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010 Sep;17(5):493-501 [FREE Full text] [CrossRef] [Medline]
    103. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev 2014;3:74 [FREE Full text] [CrossRef] [Medline]
    104. Kawamoto K, Del Fiol G, Lobach DF, Jenders RA. Standards for scalable clinical decision support: need, current and emerging standards, gaps, and proposal for progress. Open Med Inform J 2010;4:235-244 [FREE Full text] [CrossRef] [Medline]
    105. Karacapilidis N. An overview of future challenges of decision support technologies. In: Intelligent Decision-Making Support Systems SE-20. London: Springer; 2006:385-399.
    106. Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007 Mar 21;297(11):1233-1240. [CrossRef] [Medline]
    107. Vogt-Ferrier N. Older patients, multiple comorbidities, polymedication… should we treat everything? European Geriatric Medicine 2011 Feb;2(1):48-51. [CrossRef]
    108. Buchan I, Winn J, Bishop CM. A unified modeling approach to data-intensive healthcare. In: The Fourth Paradigm. Redmond, WA: Microsoft Research; 2009:91-97.

    ‎
    EHR: electronic health record
    CDS: clinical decision support
    NICE: National Institute for Health and Care Excellence

    Edited by G Eysenbach; submitted 01.05.14; peer-reviewed by E Pan, O Anya, A James; comments to author 20.08.14; revised version received 26.09.14; accepted 08.11.14; published 07.01.15

    Copyright

    ©Paolo Fraccaro, Mercedes Arguello Castelerio, John Ainsworth, Iain Buchan. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 07.01.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.