Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2023

Open Access 01-12-2023 | Telemedicine | Research

A Chinese telemedicine-dialogue dataset annotated for named entities

Authors: Shanshan Wang, Yajing Yan, Rong Yan, Ting Li, Kaijie Ma, Yani Yan

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Login to get access

Abstract

Background

A large collection of dialogues between patients and doctors must be annotated for medical named entities to build intelligence for telemedicine. However, since most patients involved in telemedicine deliver related named entities in informal and long multiword expressions, it is challenging to tag their telemedicine dialogue data. This study aims to address this issue.

Methods

With the telemedicine dialogue dataset for obstetrics and gynecology taken from haodf.com, we developed guidelines and followed a two-round procedure to tag six types of named entities, including disease, symptom, time, pharmaceutical, operation, and examination. Additionally, we developed four deep-learning models based on this dataset to establish a benchmark for named-entity recognition (NER).

Results

The distilled obstetrics and gynecology dataset contains 2,383 consultations between doctors and patients, of which 13,411 sentences were from doctors, and 17,929 were from patients. With 63,560 named entities in total, the average number of characters per named entity is 4.33. The experimental results suggest that LatticeLSTM performs best on our dataset in terms of accuracy, precision, recall, and F score.

Conclusion

Compared with other datasets, this dataset offers three novel facets. This study offers intricately tagged long multiword expressions for medical named entities. Second, this study is one of the first attempts to mark temporal entities in a medical dataset. Third, this annotated dataset is balanced across the six types of labels, which we believe will play a considerable role in expanding telemedicine artificial intelligence.
Literature
1.
go back to reference Ramshaw LA, Marcus MP. Text Chunking Using Transformation-Based Learning. In: Armstrong S, Church K, Isabelle P, Manzi S, Tzoukermann E, Yarowsky D, editors. Natural Language Processing Using Very Large Corpora. Text, Speech and Language Technology, vol. 11. Dordrecht: Springer; https://doi.org/10.1007/978-94-017-2390-9_10. Ramshaw LA, Marcus MP. Text Chunking Using Transformation-Based Learning. In: Armstrong S, Church K, Isabelle P, Manzi S, Tzoukermann E, Yarowsky D, editors. Natural Language Processing Using Very Large Corpora. Text, Speech and Language Technology, vol. 11. Dordrecht: Springer; https://​doi.​org/​10.​1007/​978-94-017-2390-9_​10.
3.
go back to reference Zhou Y, Yan Y, Han R, Caufield JH, Chang K-W, Sun Y, et al. Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, issue no. 16. 2021 p. 14647–14655. https://doi.org/10.1609/aaai.v35i16.17721. Zhou Y, Yan Y, Han R, Caufield JH, Chang K-W, Sun Y, et al. Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, issue no. 16. 2021 p. 14647–14655. https://​doi.​org/​10.​1609/​aaai.​v35i16.​17721.
4.
go back to reference Yang J, Zhang Y, Li L, Li X. YEDDA: A Lightweight Collaborative Text Span Annotation Tool. In Proceedings of ACL 2018, System Demonstrations. Melbourne: Association for Computational Linguistics; 2018. p. 31–36. Yang J, Zhang Y, Li L, Li X. YEDDA: A Lightweight Collaborative Text Span Annotation Tool. In Proceedings of ACL 2018, System Demonstrations. Melbourne: Association for Computational Linguistics; 2018. p. 31–36.
6.
go back to reference Hongying Z, Wenxin L, Kunli Z, Yajuan Y, Baobao C, Zhifang S. Building a Pediatric Medical Corpus: Word Segmentation and Named Entity Annotation. CLSW. 2020. p. 652–664. Hongying Z, Wenxin L, Kunli Z, Yajuan Y, Baobao C, Zhifang S. Building a Pediatric Medical Corpus: Word Segmentation and Named Entity Annotation. CLSW. 2020. p. 652–664.
7.
go back to reference Cai L, Wang ST, Liu J-H, Zhu Y-Y. Survey of data annotation. J Softw. 2020;31(2):302–20. Cai L, Wang ST, Liu J-H, Zhu Y-Y. Survey of data annotation. J Softw. 2020;31(2):302–20.
8.
go back to reference Carletta J. Assessing Agreement on Classification Tasks: The Kappa Statistic. Comput. Linguist. 1996;22(2):249–54. Carletta J. Assessing Agreement on Classification Tasks: The Kappa Statistic. Comput. Linguist. 1996;22(2):249–54.
9.
go back to reference Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inf Assoc. 2005;12(3):296–8.CrossRef Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inf Assoc. 2005;12(3):296–8.CrossRef
10.
go back to reference Yang J, Guan Y, He B, Qu C-Y, Yu Q-B, Liu Y-X, Zhao Y-J. Corpus construction for named entities and entity relations on chinese electronic medical records. J Softw. 2016;27(11):2725–46. Yang J, Guan Y, He B, Qu C-Y, Yu Q-B, Liu Y-X, Zhao Y-J. Corpus construction for named entities and entity relations on chinese electronic medical records. J Softw. 2016;27(11):2725–46.
11.
go back to reference Albright D, Lanfranchi A, Fredriksen A, Styler WF, Warner C, Hwang JD, Choi JD, Dligach D, Nielsen RD, Martin J. Towards comprehensive syntactic and semantic annotations of the clinical narrative. J Am Med Inform Assoc Jamia. 2013;20(5):922–30.CrossRefPubMed Albright D, Lanfranchi A, Fredriksen A, Styler WF, Warner C, Hwang JD, Choi JD, Dligach D, Nielsen RD, Martin J. Towards comprehensive syntactic and semantic annotations of the clinical narrative. J Am Med Inform Assoc Jamia. 2013;20(5):922–30.CrossRefPubMed
12.
go back to reference Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF Models for Sequence Tagging. CoRR abs/1508.01991. 2015. p. 1–9. Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF Models for Sequence Tagging. CoRR abs/1508.01991. 2015. p. 1–9.
13.
go back to reference Zhang Y, Yang J. Chinese NER Using Lattice LSTM. ACL. 2018;(1):1554–64. Zhang Y, Yang J. Chinese NER Using Lattice LSTM. ACL. 2018;(1):1554–64.
14.
go back to reference Huang K, Altosaar J, Ranganath R. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. CoRR abs/1904.05342. 2019. p. 1–6. Huang K, Altosaar J, Ranganath R. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. CoRR abs/1904.05342. 2019. p. 1–6.
15.
go back to reference Liu W, Fu X, Zhang Y, Xiao W. Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter, issue no. 1. ACL/IJCNLP. 2021. p. 5847–5858. Liu W, Fu X, Zhang Y, Xiao W. Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter, issue no. 1. ACL/IJCNLP. 2021. p. 5847–5858.
16.
go back to reference Xue N, Xia F, Chiou FD, Palmer M. The penn chinese treebank: Phrase structure annotation of a large corpus. Nat Lang Eng. 2005;11(2):207–38.CrossRef Xue N, Xia F, Chiou FD, Palmer M. The penn chinese treebank: Phrase structure annotation of a large corpus. Nat Lang Eng. 2005;11(2):207–38.CrossRef
17.
go back to reference Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed Representations of Words and Phrases and their Compositionality. NIPS. 2013. p. 3111–3119. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed Representations of Words and Phrases and their Compositionality. NIPS. 2013. p. 3111–3119.
18.
go back to reference Zeng G, Yang W, Ju Z, Yang Y, Wang S, Zhang R, et al. MedDialog: Large-scale Medical Dialogue Datasets. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. 2020. p. 9241–9250. Zeng G, Yang W, Ju Z, Yang Y, Wang S, Zhang R, et al. MedDialog: Large-scale Medical Dialogue Datasets. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. 2020. p. 9241–9250.
Metadata
Title
A Chinese telemedicine-dialogue dataset annotated for named entities
Authors
Shanshan Wang
Yajing Yan
Rong Yan
Ting Li
Kaijie Ma
Yani Yan
Publication date
01-12-2023
Publisher
BioMed Central
Keyword
Telemedicine
Published in
BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-023-02365-3

Other articles of this Issue 1/2023

BMC Medical Informatics and Decision Making 1/2023 Go to the issue