Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2019

Open Access 01-12-2019 | Type 2 Diabetes | Research article

What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project

Authors: Giuseppe Fico, Liss Hernanzez, Jorge Cancela, Arianna Dagliati, Lucia Sacchi, Antonio Martinez-Millana, Jorge Posada, Lidia Manero, Jose Verdú, Andrea Facchinetti, Manuel Ottaviano, Konstantia Zarkogianni, Konstantina Nikita, Leif Groop, Rafael Gabriel-Sanchez, Luca Chiovato, Vicente Traver, Juan Francisco Merino-Torres, Claudio Cobelli, Riccardo Bellazzi, Maria Teresa Arredondo

Published in: BMC Medical Informatics and Decision Making | Issue 1/2019

Login to get access

Abstract

Background

To understand user needs, system requirements and organizational conditions towards successful design and adoption of Clinical Decision Support Systems for Type 2 Diabetes (T2D) care built on top of computerized risk models.

Methods

The holistic and evidence-based CEHRES Roadmap, used to create eHealth solutions through participatory development approach, persuasive design techniques and business modelling, was adopted in the MOSAIC project to define the sequence of multidisciplinary methods organized in three phases, user needs, implementation and evaluation. The research was qualitative, the total number of participants was ninety, about five-seventeen involved in each round of experiment.

Results

Prediction models for the onset of T2D are built on clinical studies, while for T2D care are derived from healthcare registries. Accordingly, two set of DSSs were defined: the first, T2D Screening, introduces a novel routine; in the second case, T2D Care, DSSs can support managers at population level, and daily practitioners at individual level. In the user needs phase, T2D Screening and solution T2D Care at population level share similar priorities, as both deal with risk-stratification. End-users of T2D Screening and solution T2D Care at individual level prioritize easiness of use and satisfaction, while managers prefer the tools to be available every time and everywhere. In the implementation phase, three Use Cases were defined for T2D Screening, adapting the tool to different settings and granularity of information. Two Use Cases were defined around solutions T2D Care at population and T2D Care at individual, to be used in primary or secondary care. Suitable filtering options were equipped with “attractive” visual analytics to focus the attention of end-users on specific parameters and events. In the evaluation phase, good levels of user experience versus bad level of usability suggest that end-users of T2D Screening perceived the potential, but they are worried about complexity. Usability and user experience were above acceptable thresholds for T2D Care at population and T2D Care at individual.

Conclusions

By using a holistic approach, we have been able to understand user needs, behaviours and interactions and give new insights in the definition of effective Decision Support Systems to deal with the complexity of T2D care.
Appendix
Available only for authorised users
Literature
2.
go back to reference Kane R, Priester R, Totten A. Meeting the challenge of chronic illness. Baltimore: The Johns Hopkins University Press; 2005. Kane R, Priester R, Totten A. Meeting the challenge of chronic illness. Baltimore: The Johns Hopkins University Press; 2005.
5.
go back to reference American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.CrossRef American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.CrossRef
6.
go back to reference Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ. 2011;343:d7163.CrossRef Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ. 2011;343:d7163.CrossRef
7.
go back to reference Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AM, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900.CrossRef Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AM, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900.CrossRef
8.
go back to reference Zarkogianni K, Litsa E, Mitsis K, Wu P, Kaddi CD, Cheng C, Wang MD, Nikita KS. A review of emerging technologies for the management of diabetes mellitus. IEEE Trans Biomed Eng. 2015;62(12):2735–49.CrossRef Zarkogianni K, Litsa E, Mitsis K, Wu P, Kaddi CD, Cheng C, Wang MD, Nikita KS. A review of emerging technologies for the management of diabetes mellitus. IEEE Trans Biomed Eng. 2015;62(12):2735–49.CrossRef
9.
go back to reference Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.CrossRef Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.CrossRef
10.
go back to reference Roshanov PS, et al. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implement Sc. 2011;6(1):92.CrossRef Roshanov PS, et al. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implement Sc. 2011;6(1):92.CrossRef
11.
go back to reference Roshanov PS, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.CrossRef Roshanov PS, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.CrossRef
12.
go back to reference Miller A, Moon B, Anders S, Walden R, Brown S, Montella D. Integrating computerized clinical decision support systems into clinical work: a meta-synthesis of qualitative research. Int J Med Inform. 2015;84(12):1009–18.CrossRef Miller A, Moon B, Anders S, Walden R, Brown S, Montella D. Integrating computerized clinical decision support systems into clinical work: a meta-synthesis of qualitative research. Int J Med Inform. 2015;84(12):1009–18.CrossRef
13.
go back to reference Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform. 2015;53:3–14.CrossRef Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform. 2015;53:3–14.CrossRef
14.
go back to reference Zhang J. Human-centered computing in health information systems part 1: analysis and design. J Biomed Inform. 2005;38(1):1–3.CrossRef Zhang J. Human-centered computing in health information systems part 1: analysis and design. J Biomed Inform. 2005;38(1):1–3.CrossRef
15.
go back to reference Rinkus S, Walji M, Johnson-Throop KA, Malin M, Turley JP, Smith JW, Zhang J. Human-centered design of a distributed knowledge management system. J Biomed Inform. 2005;38:4–17.CrossRef Rinkus S, Walji M, Johnson-Throop KA, Malin M, Turley JP, Smith JW, Zhang J. Human-centered design of a distributed knowledge management system. J Biomed Inform. 2005;38:4–17.CrossRef
16.
go back to reference Nemeth CP, Nunnally M, O’Connor M, Klock PA, Cook R. Getting to the point: developing IT for the sharp end of healthcare. J Biomed Inform. 2005;38:18–25.CrossRef Nemeth CP, Nunnally M, O’Connor M, Klock PA, Cook R. Getting to the point: developing IT for the sharp end of healthcare. J Biomed Inform. 2005;38:18–25.CrossRef
17.
go back to reference Xiao Y. Artifacts and collaborative work in healthcare: methodological, theoretical and technological implications of the tangible. J Biomed Inform. 2005;38:26–33.CrossRef Xiao Y. Artifacts and collaborative work in healthcare: methodological, theoretical and technological implications of the tangible. J Biomed Inform. 2005;38:26–33.CrossRef
18.
go back to reference Malhotra S, Laxmisan A, Keselman A, Zhang J, Patel VL. Designing the design phase of critical care devices: a cognitive approach. J Biomed Inform. 2005;38:34–50.CrossRef Malhotra S, Laxmisan A, Keselman A, Zhang J, Patel VL. Designing the design phase of critical care devices: a cognitive approach. J Biomed Inform. 2005;38:34–50.CrossRef
19.
go back to reference Samaras GM, Horst RL. A systems engineering perspective on the human-centered design of health information systems. J Biomed Inform. 2005;38:61–74.CrossRef Samaras GM, Horst RL. A systems engineering perspective on the human-centered design of health information systems. J Biomed Inform. 2005;38:61–74.CrossRef
20.
go back to reference Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. 2005;38:75–87.CrossRef Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. 2005;38:75–87.CrossRef
21.
go back to reference Patterson ES, Boebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–99.CrossRef Patterson ES, Boebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–99.CrossRef
22.
go back to reference Laxmisan A, Malhotra S, Keselman A, Johnson TR, Patel VL. Decisions about critical events in device-related scenarios as a function of expertise. J Biomed Inform. 2005;38:200–12.CrossRef Laxmisan A, Malhotra S, Keselman A, Johnson TR, Patel VL. Decisions about critical events in device-related scenarios as a function of expertise. J Biomed Inform. 2005;38:200–12.CrossRef
23.
go back to reference Ginsburg GE. Human factors engineering: a tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213–9.CrossRef Ginsburg GE. Human factors engineering: a tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213–9.CrossRef
24.
go back to reference Reddy M, McDonald DW, Pratt W, Shabot MM. Technology, work, and information flows: lessons from the implementation of a wireless alert pager system. J Biomed Inform. 2005;38:229–38.CrossRef Reddy M, McDonald DW, Pratt W, Shabot MM. Technology, work, and information flows: lessons from the implementation of a wireless alert pager system. J Biomed Inform. 2005;38:229–38.CrossRef
25.
go back to reference Despont-Gros C, Mueller H, Lovis C. Evaluating user interactions with clinical information systems: a model based on human–computer interaction models. J Biomed Inform. 2005;38:244–55.CrossRef Despont-Gros C, Mueller H, Lovis C. Evaluating user interactions with clinical information systems: a model based on human–computer interaction models. J Biomed Inform. 2005;38:244–55.CrossRef
26.
go back to reference World Health Organization, 2009. Practical guidance for scaling up health service innovations. World Health Organization, 2009. Practical guidance for scaling up health service innovations.
27.
go back to reference European Commission, 2015. European scaling up strategy on active and healthy ageing. European Commission, 2015. European scaling up strategy on active and healthy ageing.
28.
go back to reference van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111.CrossRef van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111.CrossRef
29.
go back to reference Sacchi L, Dagliati A, Segagni D, Leporati P, Chiovato L, Bellazzi R. Improving risk-stratification of diabetes complications using temporal data mining. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2131–4. Sacchi L, Dagliati A, Segagni D, Leporati P, Chiovato L, Bellazzi R. Improving risk-stratification of diabetes complications using temporal data mining. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2131–4.
30.
go back to reference Sambo F, Di Camillo B, Franzin A, Facchinetti A, Hakaste L, Kravic J, Fico G, et al. A Bayesian Network analysis of the probabilistic relations between risk factors in the predisposition to type 2 diabetes. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2119–22. Sambo F, Di Camillo B, Franzin A, Facchinetti A, Hakaste L, Kravic J, Fico G, et al. A Bayesian Network analysis of the probabilistic relations between risk factors in the predisposition to type 2 diabetes. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2119–22.
31.
go back to reference Van Velsen L, van Gemert-Pijnen L, Nijland N, Beaujean D, Van Steenbergen J. Personas: the linking pin in holistic design for eHealth. proc. eTELEMED; 2012. Van Velsen L, van Gemert-Pijnen L, Nijland N, Beaujean D, Van Steenbergen J. Personas: the linking pin in holistic design for eHealth. proc. eTELEMED; 2012.
32.
go back to reference Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc. 2013;2(1):e21.CrossRef Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc. 2013;2(1):e21.CrossRef
33.
go back to reference Maurya A. Running lean: iterate from plan A to a plan that works. United States: O’Reilly Media, Inc.; 2012. Maurya A. Running lean: iterate from plan A to a plan that works. United States: O’Reilly Media, Inc.; 2012.
34.
go back to reference Wentzel J, Van Limburg M, Karreman J, Hendrix R, Van Gemert-Pijnen L. Co-creation with stakeholders: a Web 2.0 Antibiotic Stewardship Program. Proceedings of The Fourth International Conference on eHealth, Telemedicine, and Social Medicine: January 30, 2012 to February 4, 2012; Valencia. 2012:196–202. Wentzel J, Van Limburg M, Karreman J, Hendrix R, Van Gemert-Pijnen L. Co-creation with stakeholders: a Web 2.0 Antibiotic Stewardship Program. Proceedings of The Fourth International Conference on eHealth, Telemedicine, and Social Medicine: January 30, 2012 to February 4, 2012; Valencia. 2012:196–202.
35.
go back to reference Morgan DL. Focus groups as qualitative research. Thousand Oaks: Sage; 1997.CrossRef Morgan DL. Focus groups as qualitative research. Thousand Oaks: Sage; 1997.CrossRef
36.
go back to reference Saaty T. How to structure and make choices in complex problems. Hum Syst Manag. 1982;3:255–61. Saaty T. How to structure and make choices in complex problems. Hum Syst Manag. 1982;3:255–61.
37.
go back to reference Saaty TL. A scaling method for priorities in hierarchical structures. J Math Psychol. 1977;15:234–81.CrossRef Saaty TL. A scaling method for priorities in hierarchical structures. J Math Psychol. 1977;15:234–81.CrossRef
38.
go back to reference Pecchia L, Bath PA, Pendleton N, Bracale M: Web-based system for assessing risk factors for falls in community-dwelling elderly people using the analytic hierarchy process. International Journal of the Analytic Hierarchy Process. 2010;2(2):135–57. Pecchia L, Bath PA, Pendleton N, Bracale M: Web-based system for assessing risk factors for falls in community-dwelling elderly people using the analytic hierarchy process. International Journal of the Analytic Hierarchy Process. 2010;2(2):135–57.
39.
go back to reference Fico G, Gaeta E, Arredondo MT, Pecchia L. Analytic hierarchy process to define the most important factors and related technologies for empowering elderly people in taking an active role in their health. J Med Syst. 2015;39(9):1–7.CrossRef Fico G, Gaeta E, Arredondo MT, Pecchia L. Analytic hierarchy process to define the most important factors and related technologies for empowering elderly people in taking an active role in their health. J Med Syst. 2015;39(9):1–7.CrossRef
41.
go back to reference Nielsen J. Ten usability heuristics. United States: Nielsen Norman Group; 2005. Nielsen J. Ten usability heuristics. United States: Nielsen Norman Group; 2005.
42.
go back to reference Brooke J. SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189(194):4–7. Brooke J. SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189(194):4–7.
43.
go back to reference Hassenzahl M, Burmester M, Koller F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch & computer. Germany: Vieweg+ Teubner Verlag; 2003, 2003. p. 187–96. Hassenzahl M, Burmester M, Koller F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch & computer. Germany: Vieweg+ Teubner Verlag; 2003, 2003. p. 187–96.
44.
go back to reference International Organization for Standardization. Ergonomics of human-system interaction: part 210: human-centred design for interactive systems. United States: ISO; 2010. International Organization for Standardization. Ergonomics of human-system interaction: part 210: human-centred design for interactive systems. United States: ISO; 2010.
45.
go back to reference Sauro J, Lewis JR. Quantifying the user experience: practical statistics for user research. Burlington: Morgan Kaufmann; 2012. Sauro J, Lewis JR. Quantifying the user experience: practical statistics for user research. Burlington: Morgan Kaufmann; 2012.
46.
go back to reference Gülcü C. The complete log4j manual. QOS. ch; 2003. Gülcü C. The complete log4j manual. QOS. ch; 2003.
47.
go back to reference Nantz B. Open source. NET development: programming with NAnt, NUnit, NDoc, and More. United States: Addison-Wesley Professional; 2004. Nantz B. Open source. NET development: programming with NAnt, NUnit, NDoc, and More. United States: Addison-Wesley Professional; 2004.
48.
go back to reference Borsci S, Federici S, Lauriola M. On the dimensionality of the system usability scale: a test of alternative measurement models. Cogn Process. 2009;10(3):193–7.CrossRef Borsci S, Federici S, Lauriola M. On the dimensionality of the system usability scale: a test of alternative measurement models. Cogn Process. 2009;10(3):193–7.CrossRef
49.
go back to reference Borsci S, Federici S, Bacci S, Gnaldi M, Bartolucci F. Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int J Hum Comput Interact. 2015;31(8):484–95.CrossRef Borsci S, Federici S, Bacci S, Gnaldi M, Bartolucci F. Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int J Hum Comput Interact. 2015;31(8):484–95.CrossRef
50.
go back to reference Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. 1993. pp. 206–13. ACM. Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. 1993. pp. 206–13. ACM.
51.
go back to reference Dagliati A, Sacchi L, Tibollo V, Cogni G, Teliti M, Martinez-Millana A, et al. A dashboard-based system for supporting diabetes care. J Am Med Inform Assoc. 2018;25(5):538–47.CrossRef Dagliati A, Sacchi L, Tibollo V, Cogni G, Teliti M, Martinez-Millana A, et al. A dashboard-based system for supporting diabetes care. J Am Med Inform Assoc. 2018;25(5):538–47.CrossRef
52.
go back to reference Fico G, et al. User requirements for incorporating diabetes modeling techniques in disease management tools. In: 6th European conference of the international federation for medical and biological engineering. Switzerland: Springer International Publishing; 2015. Fico G, et al. User requirements for incorporating diabetes modeling techniques in disease management tools. In: 6th European conference of the international federation for medical and biological engineering. Switzerland: Springer International Publishing; 2015.
53.
go back to reference Cancela J, Hernandez L, Fico G, Waldmeyer MTA. Heuristic evaluation of a toolset for type 2 diabetes mellitus management. In: XIV Mediterranean conference on medical and biological engineering and computing. Switzerland: Springer International Publishing; 2016, 2016. p. 982–7.CrossRef Cancela J, Hernandez L, Fico G, Waldmeyer MTA. Heuristic evaluation of a toolset for type 2 diabetes mellitus management. In: XIV Mediterranean conference on medical and biological engineering and computing. Switzerland: Springer International Publishing; 2016, 2016. p. 982–7.CrossRef
54.
go back to reference Borsci S, Uchegbu I, Buckle P, Ni Z, Walne S, Hanna GB. Designing medical technology for resilience: integrating health economics and human factors approaches. Expert Rev Med Devices. 2018;15(1):15–26.CrossRef Borsci S, Uchegbu I, Buckle P, Ni Z, Walne S, Hanna GB. Designing medical technology for resilience: integrating health economics and human factors approaches. Expert Rev Med Devices. 2018;15(1):15–26.CrossRef
Metadata
Title
What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project
Authors
Giuseppe Fico
Liss Hernanzez
Jorge Cancela
Arianna Dagliati
Lucia Sacchi
Antonio Martinez-Millana
Jorge Posada
Lidia Manero
Jose Verdú
Andrea Facchinetti
Manuel Ottaviano
Konstantia Zarkogianni
Konstantina Nikita
Leif Groop
Rafael Gabriel-Sanchez
Luca Chiovato
Vicente Traver
Juan Francisco Merino-Torres
Claudio Cobelli
Riccardo Bellazzi
Maria Teresa Arredondo
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Type 2 Diabetes
Published in
BMC Medical Informatics and Decision Making / Issue 1/2019
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-019-0887-8

Other articles of this Issue 1/2019

BMC Medical Informatics and Decision Making 1/2019 Go to the issue