Skip to main content
Top
Published in: BMC Health Services Research 1/2020

01-12-2020 | Confusion | Research article

A drug identification model developed using deep learning technologies: experience of a medical center in Taiwan

Authors: Hsien-Wei Ting, Sheng-Luen Chung, Chih-Fang Chen, Hsin-Yi Chiu, Yow-Wen Hsieh

Published in: BMC Health Services Research | Issue 1/2020

Login to get access

Abstract

Background

Issuing of correct prescriptions is a foundation of patient safety. Medication errors represent one of the most important problems in health care, with ‘look-alike and sound-alike’ (LASA) being the lead error. Existing solutions to prevent LASA still have their limitations. Deep learning techniques have revolutionized identification classifiers in many fields. In search of better image-based solutions for blister package identification problem, this study using a baseline deep learning drug identification (DLDI) aims to understand how identification confusion of look-alike images by human occurs through the cognitive counterpart of deep learning solutions and thereof to suggest further solutions to approach them.

Methods

We collected images of 250 types of blister-packaged drug from the Out-Patient Department (OPD) of a medical center for identification. The deep learning framework of You Only Look Once (YOLO) was adopted for implementation of the proposed deep learning. The commonly-used F1 score, defined by precision and recall for large numbers of identification tests, was used as the performance criterion. This study trained and compared the proposed models based on images of either the front-side or back-side of blister-packaged drugs.

Results

Our results showed that the total training time for the front-side model and back-side model was 5 h 34 min and 7 h 42 min, respectively. The F1 score of the back-side model (95.99%) was better than that of the front-side model (93.72%).

Conclusions

In conclusion, this study constructed a deep learning-based model for blister-packaged drug identification, with an accuracy greater than 90%. This model outperformed identification using conventional computer vision solutions, and could assist pharmacists in identifying drugs while preventing medication errors caused by look-alike blister packages. By integration into existing prescription systems in hospitals, the results of this study indicated that using this model, drugs dispensed could be verified in order to achieve automated prescription and dispensing.
Literature
1.
go back to reference Gates PJ, Baysari MT, Mumford V, Raban MZ, Westbrook JI. Standardising the classification of harm associated with medication errors: the harm associated with medication error classification (HAMEC). Drug Saf. 2019;42(8):931–9.CrossRef Gates PJ, Baysari MT, Mumford V, Raban MZ, Westbrook JI. Standardising the classification of harm associated with medication errors: the harm associated with medication error classification (HAMEC). Drug Saf. 2019;42(8):931–9.CrossRef
2.
go back to reference Mekonnen AB, Alhawassi TM, McLachlan AJ, Brien JE. Adverse drug events and medication errors in African hospitals: a systematic review. Drugs Real World Outcomes. 2018;5(1):1–24.CrossRef Mekonnen AB, Alhawassi TM, McLachlan AJ, Brien JE. Adverse drug events and medication errors in African hospitals: a systematic review. Drugs Real World Outcomes. 2018;5(1):1–24.CrossRef
3.
go back to reference Zhong W, Feinstein JA, Patel NS, Dai D, Feudtner C. Tall man lettering and potential prescription errors: a time series analysis of 42 children's hospitals in the USA over 9 years. BMJ Qual Saf. 2016;25(4):233–40.CrossRef Zhong W, Feinstein JA, Patel NS, Dai D, Feudtner C. Tall man lettering and potential prescription errors: a time series analysis of 42 children's hospitals in the USA over 9 years. BMJ Qual Saf. 2016;25(4):233–40.CrossRef
4.
go back to reference Rash-Foanio C, Galanter W, Bryson M, Falck S, Liu KL, Schiff GD, Vaida A, Lambert BL. Automated detection of look-alike/sound-alike medication errors. Am J Health Syst Pharm. 2017;74(7):521–7.CrossRef Rash-Foanio C, Galanter W, Bryson M, Falck S, Liu KL, Schiff GD, Vaida A, Lambert BL. Automated detection of look-alike/sound-alike medication errors. Am J Health Syst Pharm. 2017;74(7):521–7.CrossRef
5.
go back to reference Tseng HY, Wen CF, Lee YL, Jeng KC, Chen PL. Dispensing errors from look-alike drug trade names. Eur J Hosp Pharm. 2018;25(2):96–9.CrossRef Tseng HY, Wen CF, Lee YL, Jeng KC, Chen PL. Dispensing errors from look-alike drug trade names. Eur J Hosp Pharm. 2018;25(2):96–9.CrossRef
6.
go back to reference Kang HJ, Park H, Oh JM, Lee EK. Perception of reporting medication errors including near-misses among Korean hospital pharmacists. Medicine. 2017;96(39):e7795.CrossRef Kang HJ, Park H, Oh JM, Lee EK. Perception of reporting medication errors including near-misses among Korean hospital pharmacists. Medicine. 2017;96(39):e7795.CrossRef
7.
go back to reference Aldhwaihi K, Schifano F, Pezzolesi C, Umaru N. A systematic review of the nature of dispensing errors in hospital pharmacies. Integr Pharm Res Pract. 2016;5:1–10.CrossRef Aldhwaihi K, Schifano F, Pezzolesi C, Umaru N. A systematic review of the nature of dispensing errors in hospital pharmacies. Integr Pharm Res Pract. 2016;5:1–10.CrossRef
8.
go back to reference Testa M, Pollard J. Safe pill-dispensing. Stud Health Technol Inform. 2007;127:139–46.PubMed Testa M, Pollard J. Safe pill-dispensing. Stud Health Technol Inform. 2007;127:139–46.PubMed
9.
go back to reference Harolds JA, Harolds LB. Quality and safety in health care, part X: other technology to reduce medication errors. Clin Nucl Med. 2016;41(5):376–8.CrossRef Harolds JA, Harolds LB. Quality and safety in health care, part X: other technology to reduce medication errors. Clin Nucl Med. 2016;41(5):376–8.CrossRef
10.
go back to reference Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: dispensing and administration--2014. Am J Health Syst Pharm. 2015;72(13):1119–37.CrossRef Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: dispensing and administration--2014. Am J Health Syst Pharm. 2015;72(13):1119–37.CrossRef
11.
go back to reference Johnson CL, Carlson RA, Tucker CL, Willette C. Using BCMA software to improve patient safety in veterans administration medical centers. J Healthc Inf Manag. 2002;16(1):46–51.PubMed Johnson CL, Carlson RA, Tucker CL, Willette C. Using BCMA software to improve patient safety in veterans administration medical centers. J Healthc Inf Manag. 2002;16(1):46–51.PubMed
12.
go back to reference Wang J-S, Ambikapathi A, Han Y, Chung S-L, Ting H-W, Chen C-F. Highlighted Deep Learning based Identification of Pharmaceutical Blister Package. In: IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA 2018); 2018. Wang J-S, Ambikapathi A, Han Y, Chung S-L, Ting H-W, Chen C-F. Highlighted Deep Learning based Identification of Pharmaceutical Blister Package. In: IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA 2018); 2018.
13.
go back to reference Suntronsuk S, Ratanotayanon S. Automatic text imprint analysis from pill images. In: 2017 9th International Conference on Knowledge and Smart Technology: Crunching Information of Everything, KST; 2017. p. 288–93.CrossRef Suntronsuk S, Ratanotayanon S. Automatic text imprint analysis from pill images. In: 2017 9th International Conference on Knowledge and Smart Technology: Crunching Information of Everything, KST; 2017. p. 288–93.CrossRef
14.
go back to reference Chen R-C, Chan Y-K, Chen Y-H, Bau C-T. An automatic drug image identification system based on multiple image features and dynamic weights. Int J Innovative Comput Inf Control. 2012;8(5):2995–3013. Chen R-C, Chan Y-K, Chen Y-H, Bau C-T. An automatic drug image identification system based on multiple image features and dynamic weights. Int J Innovative Comput Inf Control. 2012;8(5):2995–3013.
15.
go back to reference Lee YB, Park U, Jain AK, Lee SW. Pill-ID: matching and retrieval of drug pill images. Pattern Recogn Lett. 2012;33(7):904–10.CrossRef Lee YB, Park U, Jain AK, Lee SW. Pill-ID: matching and retrieval of drug pill images. Pattern Recogn Lett. 2012;33(7):904–10.CrossRef
16.
go back to reference Taran O, Rezaeifar S, Dabrowski O, Schlechten J, Holotyak T, Voloshynovskiy S. PharmaPack: mobile fine-grained recognition of pharma packages. Eur Signal Pr Conf. 2017;1:1917–21. Taran O, Rezaeifar S, Dabrowski O, Schlechten J, Holotyak T, Voloshynovskiy S. PharmaPack: mobile fine-grained recognition of pharma packages. Eur Signal Pr Conf. 2017;1:1917–21.
17.
go back to reference Saitoh K. Pharmaceutical Blister Pack Recognition using Local Features. In: MVA2013 IAPR International Conference on Machine Vision Applications. Kyoto; 2013. p. 355–8. Saitoh K. Pharmaceutical Blister Pack Recognition using Local Features. In: MVA2013 IAPR International Conference on Machine Vision Applications. Kyoto; 2013. p. 355–8.
18.
go back to reference Li Y, Huang C, Ding L, Li Z, Pan Y, Gao X. Deep learning in bioinformatics: introduction, application, and perspective in the big data era. Methods. 2019;166:4–21.CrossRef Li Y, Huang C, Ding L, Li Z, Pan Y, Gao X. Deep learning in bioinformatics: introduction, application, and perspective in the big data era. Methods. 2019;166:4–21.CrossRef
19.
go back to reference Ravi D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, Yang GZ. Deep learning for health informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21.CrossRef Ravi D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, Yang GZ. Deep learning for health informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21.CrossRef
20.
go back to reference Sainath TN, Kingsbury B, Saon G, Soltau H, Mohamed AR, Dahl G, Ramabhadran B. Deep convolutional neural networks for large-scale speech tasks. Neural Netw. 2015;64:39–48.CrossRef Sainath TN, Kingsbury B, Saon G, Soltau H, Mohamed AR, Dahl G, Ramabhadran B. Deep convolutional neural networks for large-scale speech tasks. Neural Netw. 2015;64:39–48.CrossRef
21.
go back to reference Lopez-de-Ipina K, Martinez-de-Lizarduy U, Calvo PM, Mekyska J, Beitia B, Barroso N, Estanga A, Tainta M, Ecay-Torres M. Advances on automatic speech analysis for early detection of Alzheimer disease: a non-linear multi-task approach. Curr Alzheimer Res. 2018;15(2):139–48.CrossRef Lopez-de-Ipina K, Martinez-de-Lizarduy U, Calvo PM, Mekyska J, Beitia B, Barroso N, Estanga A, Tainta M, Ecay-Torres M. Advances on automatic speech analysis for early detection of Alzheimer disease: a non-linear multi-task approach. Curr Alzheimer Res. 2018;15(2):139–48.CrossRef
22.
go back to reference Zou L, Yu S, Meng T, Zhang Z, Liang X, Xie Y. A technical review of convolutional neural network-based mammographic breast Cancer diagnosis. Comput Math Methods Med. 2019;2019:6509357.CrossRef Zou L, Yu S, Meng T, Zhang Z, Liang X, Xie Y. A technical review of convolutional neural network-based mammographic breast Cancer diagnosis. Comput Math Methods Med. 2019;2019:6509357.CrossRef
23.
go back to reference Deng W, Shi Q, Luo K, Yang Y, Ning N. Brain tumor segmentation based on improved convolutional neural network in combination with non-quantifiable local texture feature. J Med Syst. 2019;43(6):152.CrossRef Deng W, Shi Q, Luo K, Yang Y, Ning N. Brain tumor segmentation based on improved convolutional neural network in combination with non-quantifiable local texture feature. J Med Syst. 2019;43(6):152.CrossRef
24.
go back to reference Ekins S, Puhl AC, Zorn KM, Lane TR, Russo DP, Klein JJ, Hickey AJ, Clark AM. Exploiting machine learning for end-to-end drug discovery and development. Nat Mater. 2019;18(5):435–41.CrossRef Ekins S, Puhl AC, Zorn KM, Lane TR, Russo DP, Klein JJ, Hickey AJ, Clark AM. Exploiting machine learning for end-to-end drug discovery and development. Nat Mater. 2019;18(5):435–41.CrossRef
25.
go back to reference You J, McLeod RD, Hu P. Predicting drug-target interaction network using deep learning model. Comput Biol Chem. 2019;80:90–101.CrossRef You J, McLeod RD, Hu P. Predicting drug-target interaction network using deep learning model. Comput Biol Chem. 2019;80:90–101.CrossRef
26.
go back to reference Gao M, Igata H, Takeuchi A, Sato K, Ikegaya Y. Machine learning-based prediction of adverse drug effects: an example of seizure-inducing compounds. J Pharmacol Sci. 2017;133(2):70–8.CrossRef Gao M, Igata H, Takeuchi A, Sato K, Ikegaya Y. Machine learning-based prediction of adverse drug effects: an example of seizure-inducing compounds. J Pharmacol Sci. 2017;133(2):70–8.CrossRef
27.
go back to reference Wong YF, Ng HT, Leung KY, Chan KY, Chan SY, Loy CC. Development of fine-grained pill identification algorithm using deep convolutional network. J Biomed Inform. 2017;74:130–6.CrossRef Wong YF, Ng HT, Leung KY, Chan KY, Chan SY, Loy CC. Development of fine-grained pill identification algorithm using deep convolutional network. J Biomed Inform. 2017;74:130–6.CrossRef
28.
go back to reference Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. Proc Cvpr Ieee. 2015;1:1–9. Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. Proc Cvpr Ieee. 2015;1:1–9.
29.
go back to reference Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. P Ieee. 1998;86(11):2278–324.CrossRef Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. P Ieee. 1998;86(11):2278–324.CrossRef
30.
go back to reference Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.CrossRef Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.CrossRef
31.
go back to reference Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conf Comput Vision Pattern Recognit (Cvpr). 2014;2014:580–7. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conf Comput Vision Pattern Recognit (Cvpr). 2014;2014:580–7.
32.
go back to reference Ren SQ, He KM, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39:1137–49.CrossRef Ren SQ, He KM, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39:1137–49.CrossRef
33.
go back to reference Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. IEEE Conf Compute Vision Pattern Recognit (Cvpr). 2016;2016:779–88. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. IEEE Conf Compute Vision Pattern Recognit (Cvpr). 2016;2016:779–88.
34.
go back to reference Wang Y, Ribera J, Liu C, Yarlagadda S, Zhu F. Pill Recognition Using Minimal Labeled Data. In: 2017 IEEE Third International Conference on Multimedia Big Data (BigMM); 2017. p. 346–53.CrossRef Wang Y, Ribera J, Liu C, Yarlagadda S, Zhu F. Pill Recognition Using Minimal Labeled Data. In: 2017 IEEE Third International Conference on Multimedia Big Data (BigMM); 2017. p. 346–53.CrossRef
35.
go back to reference Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recognit. 2014;1:580. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recognit. 2014;1:580.
36.
go back to reference Pardo CE, Sosa LF, Gutierrez EA, Jiménez FR. Classification system for blister pack of pills. In: 2014 IEEE 5th Colombian Workshop on Circuits and Systems (CWCAS); 2014. p. 1–6. Pardo CE, Sosa LF, Gutierrez EA, Jiménez FR. Classification system for blister pack of pills. In: 2014 IEEE 5th Colombian Workshop on Circuits and Systems (CWCAS); 2014. p. 1–6.
37.
go back to reference Gilmartin JFM, Hussainy SY, Marriott JL. Medicines in Australian nursing homes: a cross-sectional observational study of the accuracy and suitability of re-packing medicines into pharmacy-supplied dose administration aids. Res Soc Adm Pharm. 2013;9(6):876–83.CrossRef Gilmartin JFM, Hussainy SY, Marriott JL. Medicines in Australian nursing homes: a cross-sectional observational study of the accuracy and suitability of re-packing medicines into pharmacy-supplied dose administration aids. Res Soc Adm Pharm. 2013;9(6):876–83.CrossRef
38.
go back to reference Custers D, Vandemoortele S, Bothy JL, De Beer JO, Courselle P, Apers S, Deconinck E. Physical profiling and IR spectroscopy: simple and effective methods to discriminate between genuine and counterfeit samples of Viagra (R) and Cialis (R). Drug Test Anal. 2016;8(3–4):378–87.CrossRef Custers D, Vandemoortele S, Bothy JL, De Beer JO, Courselle P, Apers S, Deconinck E. Physical profiling and IR spectroscopy: simple and effective methods to discriminate between genuine and counterfeit samples of Viagra (R) and Cialis (R). Drug Test Anal. 2016;8(3–4):378–87.CrossRef
Metadata
Title
A drug identification model developed using deep learning technologies: experience of a medical center in Taiwan
Authors
Hsien-Wei Ting
Sheng-Luen Chung
Chih-Fang Chen
Hsin-Yi Chiu
Yow-Wen Hsieh
Publication date
01-12-2020
Publisher
BioMed Central
Keyword
Confusion
Published in
BMC Health Services Research / Issue 1/2020
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-020-05166-w

Other articles of this Issue 1/2020

BMC Health Services Research 1/2020 Go to the issue