Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2023 | Research

RegEMR: a natural language processing system to automatically identify premature ovarian decline from Chinese electronic medical records

Authors: Jie Cai, Shenglin Chen, Siyun Guo, Suidong Wang, Lintong Li, Xiaotong Liu, Keming Zheng, Yudong Liu, Shiling Chen

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Abstract

Background

The ovarian reserve is a reservoir for reproductive potential. In clinical practice, early detection and treatment of premature ovarian decline characterized by abnormal ovarian reserve tests is regarded as a critical measure to prevent infertility. However, the relevant data are typically stored in an unstructured format in a hospital’s electronic medical record (EMR) system, and their retrieval requires tedious manual abstraction by domain experts. Computational tools are therefore needed to reduce the workload.

Methods

We presented RegEMR, an artificial intelligence tool composed of a rule-based natural language processing (NLP) extractor and a knowledge-based disease scoring model, to automatize the screening procedure of premature ovarian decline using Chinese reproductive EMRs. We used regular expressions (REs) as a text mining method and explored whether REs automatically synthesized by the genetic programming-based online platform RegexGenerator + + could be as effective as manually formulated REs. We also investigated how the representativeness of the learning corpus affected the performance of machine-generated REs. Additionally, we translated the clinical diagnostic criteria into a programmable disease diagnostic model for disease scoring and risk stratification. Four hundred outpatient medical records were collected from a Chinese fertility center. Manual review served as the gold standard, and fivefold cross-validation was used for evaluation.

Results

The overall F-score of manually built REs was 0.9444 (95% CI 0.9373 to 0.9515), with no significant difference (paired t test p > 0.05) compared with machine-generated REs that could be affected by training set sizes and annotation portions. The extractor performed effectively in automatically tracing the dynamic changes in hormone levels (F-score 0.9518–0.9884) and ultrasonographic measures (F-score 0.9472–0.9822). Applying the extracted information to the proposed diagnostic model, the program obtained an accuracy of 0.98 and a sensitivity of 0.93 in risk screening. For each specific disease, the automatic diagnosis in 76% of patients was consistent with that of the clinical diagnosis, and the kappa coefficient was 0.63.

Conclusion

A Chinese NLP system named RegEMR was developed to automatically identify high risk of early ovarian aging and diagnose related diseases from Chinese reproductive EMRs. We hope that this system can aid EMR-based data collection and clinical decision support in fertility centers.

Available only for authorised users

Sun H, Gong TT, Jiang YT, Zhang S, Zhao YH, Wu QJ. Global, regional, and national prevalence and disability-adjusted life-years for infertility in 195 countries and territories, 1990–2017: results from a global burden of disease study, 2017. Aging (Albany NY). 2019;11:10952–91.PubMedCrossRef

Gerrits T, Van Rooij F, Esho T, Ndegwa W, Goossens J, Bilajbegovic A, Jansen A, Kioko B, Koppen L, Kemunto Migiro S, et al. Infertility in the global south: raising awareness and generating insights for policy and practice. Facts Views Vis Obgyn. 2017;9:39–44.PubMedPubMedCentral

Barratt CLR, Björndahl L, De Jonge CJ, Lamb DJ, Osorio Martini F, McLachlan R, Oates RD, van der Poel S, St John B, Sigman M, et al. The diagnosis of male infertility: an analysis of the evidence to support the development of global WHO guidance-challenges and future research opportunities. Hum Reprod Update. 2017;23:660–80.PubMedPubMedCentralCrossRef

Grisendi V, Mastellari E, La Marca A. Ovarian reserve markers to identify poor responders in the context of Poseidon classification. Front Endocrinol (Lausanne). 2019;10:281.PubMedCrossRef

Nguyen HH, Milat F, Vincent A. Premature ovarian insufficiency in general practice: Meeting the needs of women. Aust Fam Physician. 2017;46:360–6.PubMed

Webber L, Davies M, Anderson R, Bartlett J, Braat D, Cartwright B, Cifkova R, de Muinck Keizer-Schrama S, European Society for Human R, Embryology Guideline Group on POI, et al. ESHRE Guideline: management of women with premature ovarian insufficiency. Hum Reprod. 2016;31:926–37.PubMedCrossRef

Practice Committee of the American Society for Reproductive Medicine. Electronic address aao, Practice Committee of the American Society for Reproductive M: Testing and interpreting measures of ovarian reserve: a committee opinion. Fertil Steril. 2020;114:1151–7.CrossRef

Pastore LM, Christianson MS, Stelling J, Kearns WG, Segars JH. Reproductive ovarian testing and the alphabet soup of diagnoses: DOR, POI, POF, POR, and FOR. J Assist Reprod Genet. 2018;35:17–23.PubMedCrossRef

Tal R, Seifer DB. Ovarian reserve testing: a user’s guide. Am J Obstet Gynecol. 2017;217:129–40.PubMedCrossRef

10.

Cohen J, Chabbert-Buffet N, Darai E. Diminished ovarian reserve, premature ovarian failure, poor ovarian responder–a plea for universal definitions. J Assist Reprod Genet. 2015;32:1709–12.PubMedPubMedCentralCrossRef

11.

Sills ES, Alper MM, Walsh AP. Ovarian reserve screening in infertility: practical applications and theoretical directions for research. Eur J Obstet Gynecol Reprod Biol. 2009;146:30–6.PubMedCrossRef

12.

Kreimeyer K, Foster M, Pandey A, Arya N, Halford G, Jones SF, Forshee R, Walderhaug M, Botsis T. Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J Biomed Inform. 2017;73:14–29.PubMedPubMedCentralCrossRef

13.

Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, Liu S, Zeng Y, Mehrabi S, Sohn S, Liu H. Clinical information extraction applications: a literature review. J Biomed Inform. 2018;77:34–49.PubMedCrossRef

14.

Jin M, Fan S, Zhang K, Bao X. A Scoping Review of Clinical Unstructured Text Information Extraction. In: In 2020 International Conference on Computational Science and Computational Intelligence (CSCI). 2020. p. 853–7.

15.

Zhang T, Huang Z, Wang Y, Wen C, Peng Y, Ye Y. Information extraction from the text data on traditional Chinese medicine: a review on tasks, challenges, and methods from 2010 to 2021. Evid Based Complement Alternat Med. 2022;2022:1679589.PubMedPubMedCentral

16.

Turchin A, Kolatkar NS, Grant RW, Makhni EC, Pendergrass ML, Einbinder JS. Using regular expressions to abstract blood pressure and treatment intensification information from the text of physician notes. J Am Med Inform Assoc. 2006;13:691–5.PubMedPubMedCentralCrossRef

17.

Voorham J, Denig P. Computerized extraction of information on the quality of diabetes care from free text in electronic patient records of general practitioners. J Am Med Inform Assoc. 2007;14:349–54.PubMedPubMedCentralCrossRef

18.

Frenz CM. Deafness mutation mining using regular expression based pattern matching. BMC Med Inform Decis Mak. 2007;7:1–6.CrossRef

19.

Denny JC, Miller RA, Waitman LR, Arrieta MA, Peterson JF. Identifying QT prolongation from ECG impressions using a general-purpose Natural Language Processor. Int J Med Informatics. 2009;78:S34–42.CrossRef

20.

Hinchcliff M, Just E, Podlusky S, Varga J, Chang RW, Kibbe WA. Text data extraction for a prospective, research-focused data mart: implementation and validation. BMC Med Inform Decis Mak. 2012;12:106.PubMedPubMedCentralCrossRef

21.

McCart JA, Finch DK, Jarman J, Hickling E, Lind JD, Richardson MR, Berndt DJ, Luther SL. Using ensemble models to classify the sentiment expressed in suicide notes. Biomed inform Insights. 2012;5:BII. S8931.CrossRef

22.

Stenner SP, Johnson KB, Denny JC. PASTE: patient-centered SMS text tagging in a medication management system. J Am Med Inform Assoc. 2012;19:368–74.PubMedCrossRef

23.

Bui DD, Zeng-Treitler Q. Learning regular expressions for clinical text classification. J Am Med Inform Assoc. 2014;21:850–7.PubMedPubMedCentralCrossRef

24.

Murtaugh MA, Gibson BS, Redd D, Zeng-Treitler Q. Regular expression-based learning to extract bodyweight values from clinical notes. J Biomed Inform. 2015;54:186–90.PubMedCrossRef

25.

Redd D, Kuang J, Mohanty A, Bray BE, Zeng-Treitler Q. Regular expression-based learning for METS value extraction. AMIA Summits Transl Sci Proceed. 2016;2016:213.

26.

Flores CA, Figueroa RL, Pezoa JE. Fregex: A feature extraction method for biomedical text classification using regular expressions. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. 2019. p. 6085–8.

27.

Bartoli A, Davanzo G, De Lorenzo A, Medvet E, Sorio E. Automatic synthesis of regular expressions from examples. Computer. 2014;47:72–80.CrossRef

28.

Bartoli A, De Lorenzo A, Medvet E, Tarlao F. Can a machine replace humans in building regular expressions? A case study. IEEE Intelligent Systems. 2016;31:15–21.CrossRef

29.

Bartoli A, De Lorenzo A, Medvet E, Tarlao F, Learning Text Patterns Using Separate-and-Conquer Genetic Programming. In: Machado, P., et al. Genetic Programming. EuroGP 2015. Lect Notes Comput Sci. 2015;9025. https://doi.org/10.1007/978-3-319-16501-1_2.

30.

Bartoli A, De Lorenzo A, Medvet E, Tarlao F. Inference of regular expressions for text extraction from examples. IEEE Trans Knowl Data Eng. 2016;28:1217–30.CrossRef

31.

Ferraretti AP, La Marca A, Fauser BC, Tarlatzis B, Nargund G, Gianaroli L. ESHRE consensus on the definition of “poor response” to ovarian stimulation for in vitro fertilization: the Bologna criteria. Hum Reprod. 2011;26:1616–24.PubMedCrossRef

32.

Chon SJ, Umair Z, Yoon MS. Premature ovarian insufficiency: past, present, and future. Front Cell Dev Biol. 2021;9:672890.PubMedPubMedCentralCrossRef

33.

Welt CK. Primary ovarian insufficiency: a more accurate term for premature ovarian failure. Clin Endocrinol (Oxf). 2008;68:499–509.PubMedCrossRef

34.

Ni Y, Bachtel A, Nause K, Beal S. Automated detection of substance use information from electronic health records for a pediatric population. J Am Med Inform Assoc. 2021;28:2116–27.PubMedPubMedCentralCrossRef

35.

Kormilitzin A, Vaci N, Liu Q, Nevado-Holgado A. Med7: a transferable clinical natural language processing model for electronic health records. Artif Intell Med. 2021;118:102086.PubMedCrossRef

36.

Qian Y, Santus E, Jin Z, Guo J, Barzilay R. GraphIE: A Graph-Based Framework for Information Extraction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: Association for Computational Linguistics. 2019;1(Long and Short Papers):751–761. https://doi.org/10.18653/v1/N19-1082.

37.

Leiter RE, Santus E, Jin Z, Lee KC, Yusufov M, Chien I, Ramaswamy A, Moseley ET, Qian Y, Schrag D, Lindvall C. Deep natural language processing to identify symptom documentation in clinical notes for patients with heart failure undergoing cardiac resynchronization therapy. J Pain Symptom Manage. 2020;60(948–958):e943.

38.

Liu H, Zhang Z, Xu Y, Wang N, Huang Y, Yang Z, Jiang R, Chen H. Use of BERT (bidirectional encoder representations from transformers)-based deep learning method for extracting evidences in chinese radiology reports: development of a computer-aided liver cancer diagnosis framework. J Med Internet Res. 2021;23:e19689.PubMedPubMedCentralCrossRef

39.

Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36:1234–40.PubMedCrossRef

40.

Huang K, Altosaar J, Ranganath R: Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:190405342 2019. https://doi.org/10.48550/arXiv.1904.05342.

41.

Friedman C, Alderson PO, Austin JH, Cimino JJ, Johnson SB. A general natural-language text processor for clinical radiology. J Am Med Inform Assoc. 1994;1:161–74.PubMedPubMedCentralCrossRef

42.

Savova GK, Masanz JJ, Ogren PV, Zheng J, Sohn S, Kipper-Schuler KC, Chute CG. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc. 2010;17:507–13.PubMedPubMedCentralCrossRef

43.

Aronson AR, Lang FM. An overview of MetaMap: historical perspective and recent advances. J Am Med Inform Assoc. 2010;17:229–36.PubMedPubMedCentralCrossRef

44.

Liu H, Bielinski SJ, Sohn S, Murphy S, Wagholikar KB, Jonnalagadda SR, Ravikumar KE, Wu ST, Kullo IJ, Chute CG. An information extraction framework for cohort identification using electronic health records. AMIA Jt Summits Transl Sci Proc. 2013;2013:149–53.PubMedPubMedCentral

45.

Wang H, Zhang W, Zeng Q, Li Z, Feng K, Liu L. Extracting important information from Chinese Operation Notes with natural language processing methods. J Biomed Inform. 2014;48:130–6.PubMedCrossRef

46.

Qiu X, Huang X. FudanNLP: A Toolkit for Chinese Natural Language Processing. 2013.

47.

Zaninovic N, Elemento O, Rosenwaks Z. Artificial intelligence: its applications in reproductive medicine and the assisted reproductive technologies. Fertil Steril. 2019;112:28–30.PubMedCrossRef

48.

Bartoli A, De Lorenzo A, Medvet E, Tarlao F. Active learning of regular expressions for entity extraction. IEEE Trans Cybern. 2018;48:1067–80.PubMedCrossRef

Title: RegEMR: a natural language processing system to automatically identify premature ovarian decline from Chinese electronic medical records
Authors: Jie Cai
Shenglin Chen
Siyun Guo
Suidong Wang
Lintong Li
Xiaotong Liu
Keming Zheng
Yudong Liu
Shiling Chen
Publication date: 01-12-2023
Publisher: BioMed Central
Published in: BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-023-02239-8

Keynote webinar | Spotlight on medication adherence

Springer Medicine

RegEMR: a natural language processing system to automatically identify premature ovarian decline from Chinese electronic medical records

Abstract

Background

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2023

Reducing prescribing of antibiotics for acute respiratory infections using a frontline nurse-led EHR-Integrated clinical decision support tool: protocol for a stepped wedge randomized control trial

Committee experiences of using formal consensus in healthcare guidelines: a longitudinal qualitative study

Construction of the XGBoost model for early lung cancer prediction based on metabolic indices

MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques

Using a decision tree algorithm to distinguish between repeated supra-therapeutic and acute acetaminophen exposures

Development and comparison of machine learning-based models for predicting heart failure after acute myocardial infarction