Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2024 | Epilepsy | Research

A comparative study of CNN-capsule-net, CNN-transformer encoder, and Traditional machine learning algorithms to classify epileptic seizure

Authors: Sergio Alejandro Holguin-Garcia, Ernesto Guevara-Navarro, Alvaro Eduardo Daza-Chica, Maria Alejandra Patiño-Claro, Harold Brayan Arteaga-Arteaga, Gonzalo A. Ruz, Reinel Tabares-Soto, Mario Alejandro Bravo-Ortiz

Published in: BMC Medical Informatics and Decision Making | Issue 1/2024

Abstract

Introduction

Epilepsy is a disease characterized by an excessive discharge in neurons generally provoked without any external stimulus, known as convulsions. About 2 million people are diagnosed each year in the world. This process is carried out by a neurological doctor using an electroencephalogram (EEG), which is lengthy.

Method

To optimize these processes and make them more efficient, we have resorted to innovative artificial intelligence methods essential in classifying EEG signals. For this, comparing traditional models, such as machine learning or deep learning, with cutting-edge models, in this case, using Capsule-Net architectures and Transformer Encoder, has a crucial role in finding the most accurate model and helping the doctor to have a faster diagnosis.

Result

In this paper, a comparison was made between different models for binary and multiclass classification of the epileptic seizure detection database, achieving a binary accuracy of 99.92% with the Capsule-Net model and a multiclass accuracy with the Transformer Encoder model of 87.30%.

Conclusion

Artificial intelligence is essential in diagnosing pathology. The comparison between models is helpful as it helps to discard those that are not efficient. State-of-the-art models overshadow conventional models, but data processing also plays an essential role in evaluating the higher accuracy of the models.

Thijs RD, Surges R, O’Brien TJ, Sander JW. Epilepsy in adults. Lancet. 2019;393(10172):689–701.CrossRefPubMed

Perucca P, Bahlo M, Berkovic SF. The genetics of epilepsy. Annu Rev Genomics Hum Genet. 2020;21:205–30.CrossRefPubMed

for Medical Education MF, Research. epilepsy symptoms-causes. Mayo Clin Proc. 2023. https://www.mayoclinic.org/es-es/diseases-conditions/epilepsy/symptoms-causes/syc-20350093. Accessed 14 Oct 2023.

Soufineyestani M, Dowling D, Khan A. Electroencephalography (EEG) technology applications and available devices. Appl Sci. 2020;10(21):7453.CrossRef

Shoeibi A, Khodatars M, Ghassemi N, Jafari M, Moridian P, Alizadehsani R, et al. Epileptic seizures detection using deep learning techniques: a review. Int J Environ Res Public Health. 2021;18(11):5780.CrossRefPubMedPubMedCentral

Zhou ZH. Machine learning. Springer Nature; 2021.

Lowery BR, Langou J. A Greedy Algorithm for Optimally Pipelining a Reduction. 2013. arXiv preprint arXiv:1310.4645.

Toraman S. Automatic recognition of preictal and interictal EEG signals using 1D-capsule networks. Comput Electr Eng. 2021;91:107033.CrossRef

Hu S, Liu J, Yang R, Wang Y, Wang A, Li K, et al. Exploring the Applicability of Transfer Learning and Feature Engineering in Epilepsy Prediction Using Hybrid Transformer Model. IEEE Trans Neural Syst Rehabil Eng. 2023;31:1321–32.CrossRefPubMed

10.

Akinyelu AA, Zaccagna F, Grist JT, Castelli M, Rundo L. Brain Tumor Diagnosis Using Machine Learning, Convolutional Neural Networks, Capsule Neural Networks and Vision Transformers, Applied to MRI: A Survey. J Imaging. 2022;8(8). https://doi.org/10.3390/jimaging8080205. https://www.mdpi.com/2313-433X/8/8/205.

11.

Wei Y, Liu Y, Li C, Cheng J, Song R, Chen X. TC-Net: A Transformer Capsule Network for EEG-based emotion recognition. Comput Biol Med. 2023;152:106463.CrossRefPubMed

12.

Xu G, Ren T, Chen Y, Che W. A one-dimensional cnn-lstm model for epileptic seizure recognition using eeg signal analysis. Front Neurosci. 2020;14:578126.CrossRefPubMedPubMedCentral

13.

Ma M, Cheng Y, Wei X, Chen Z, Zhou Y. Research on epileptic EEG recognition based on improved residual networks of 1-D CNN and indRNN. BMC Med Inf Decis Mak. 2021;21:1–13.

14.

San-Segundo R, Gil-Martín M, D’Haro-Enríquez LF, Pardo JM. Classification of epileptic EEG recordings using signal transforms and convolutional neural networks. Comput Biol Med. 2019;109:148–58.CrossRefPubMed

15.

Andrzejak RG, Schindler K, Rummel C. Nonrandomness, nonlinear dependence, and nonstationarity of electroencephalographic recordings from epilepsy patients. Phys Rev E. 2012;86(4):046206.ADSCrossRef

16.

Andrzejak RG, Lehnertz K, Mormann F, Rieke C, David P, Elger CE. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Phys Rev E. 2001;64(6):061907.ADSCrossRef

17.

Ahmadi A, Shalchyan V, Daliri MRA, new method for epileptic seizure classification in EEG using adapted wavelet packets. In: 2017 Electric Electronics, Computer Science, Biomedical Engineerings’ Meeting (EBBT). IEEE; 2017. p. 1–4.

18.

Wang L, Xue W, Li Y, Luo M, Huang J, Cui W, et al. Automatic epileptic seizure detection in EEG signals using multi-domain feature extraction and nonlinear analysis. Entropy. 2017;19(6):222.ADSCrossRef

19.

Shen M, Wen P, Song B, Li Y. Real-time epilepsy seizure detection based on EEG using tunable-Q wavelet transform and convolutional neural network. Biomed Sig Process Control. 2023;82:104566. https://doi.org/10.1016/j.bspc.2022.104566. https://www.sciencedirect.com/science/article/pii/S1746809422010205

20.

Chen W, Wang Y, Ren Y, Jiang H, Du G, Zhang J, et al. An automated detection of epileptic seizures EEG using CNN classifier based on feature fusion with high accuracy. BMC Med Inform Decis Mak. 2023;23(1). Cited by: 0; All Open Access, Gold Open Access. https://doi.org/10.1186/s12911-023-02180-w. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85159827390 &doi=10.1186%2fs12911-023-02180-w &partnerID=40 &md5=2b062961790c3fff122e44184c9b95d2.

21.

Liu J, Cai W, Shao X. Cancer classification based on microarray gene expression data using a principal component accumulation method. Sci China Chem. 2011;54:802–11.CrossRef

22.

Liashchynskyi P, Liashchynskyi P. Grid search, random search, genetic algorithm: a big comparison for NAS. 2019. arXiv preprint arXiv:1912.06059.

23.

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P,Weiss R, Dubourg V, et al. Scikit-learn: Machine learning in python. J Mach Learn Res 2011;12:2825–30.

24.

Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In: Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37. ICML’15. JMLR.org; 2015. p. 448–456. https://doi.org/10.1016/j.molstruc.2016.12.061. http://arxiv.org/abs/1502.03167, http://dl.acm.org/citation.cfm?id=3045118.3045167.

25.

Klambauer G, Unterthiner T, Mayr A, Hochreiter S. Self-Normalizing Neural Networks. 2017.

26.

Rasamoelina AD, Adjailia F, Sinčák PA, review of activation function for artificial neural network. In: 2020 IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE; 2020. p. 281–6.

27.

Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929–58.MathSciNet

28.

Camacho L, Douzas G, Bacao F. Geometric SMOTE for regression. Expert Syst Appl. 2022:116387.

29.

He H, Bai Y, Garcia EA, Li S, ADASYN: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE; 2008. p. 1322–8.

30.

Géron A. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. O’Reilly Media, Inc.; 2022. p. 153–203.

31.

Garavand A, Salehnasab C, Behmanesh A, Aslani N, Zadeh AH, Ghaderzadeh M, et al. Efficient model for coronary artery disease diagnosis: a comparative study of several machine learning algorithms. J Healthc Eng. 2022;2022.

32.

Sadoughi F, Ghaderzadeh M. A hybrid particle swarm and neural network approach for detection of prostate cancer from benign hyperplasia of prostate. In: e-Health–For Continuity of Care. IOS Press; 2014. p. 481–485.

33.

Breiman L. Random forests. Mach Learn. 2001;45:5–32.CrossRef

34.

Friedman JH. Stochastic gradient boosting. Comput Stat Data Anal. 2002;38(4):367–78.MathSciNetCrossRef

35.

Safavian SR, Landgrebe D. A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern. 1991;21(3):660–74.MathSciNetCrossRef

36.

Pandya VJ. Comparing handwritten character recognition by AdaBoostClassifier and KNeighborsClassifier. In: 2016 8th International Conference on Computational Intelligence and Communication Networks, (CICN). Tehri: IEEE; 2016. p. 271–4.

37.

Bottou L. Stochastic gradient descent tricks. Neural Networks: Tricks of the Trade. 2nd ed. 2012. p. 421–436.

38.

Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recognit. 2018;77:354–77.ADSCrossRef

39.

Sabour S, Frosst N, Hinton GE. Dynamic Routing Between Capsules. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc.; 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/2cad8fa47bbef282badbb8de5374b894-Paper.pdf.

40.

Dombetzki LA. An overview over capsule networks. Netw Archit Serv. 2018; 2–4.

41.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention Is All You Need. 2023.

42.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.

43.

Tabares-Soto R, Arteaga-Arteaga HB, Mora-Rubio A, Bravo-Ortíz MA, Arias-Garzón D, Alzate-Grisales JA, et al. Sensitivity of deep learning applied to spatial image steganalysis. PeerJ Comput Sci. 2021;7:616.CrossRef

44.

Hosseini A, Eshraghi MA, Taami T, Sadeghsalehi H, Hoseinzadeh Z, Ghaderzadeh M, et al. A mobile application based on efficient lightweight CNN model for classification of B-ALL cancer from non-cancerous cells: a design and implementation study. Inform Med Unlocked. 2023;39:101244.CrossRef

45.

Arteaga-Arteaga HB, Mora-Rubio A, Florez F, Murcia-Orjuela N, Diaz-Ortega CE, Orozco-Arias S, et al. Machine learning applications to predict two-phase flow patterns. PeerJ Comput Sci. 2021;7:798. https://doi.org/10.7717/peerj-cs.798.CrossRef

46.

Fernando GP, Brayan AAH, Florina AM, Liliana CB, Héctor-Gabriel AM, Reinel TS. Enhancing Intrusion Detection in IoT Communications Through ML Model Generalization With a New Dataset (IDSAI). IEEE Access. 2023;11:70542–59.

47.

Ghaderzadeh M, Aria M, Hosseini A, Asadi F, Bashash D, Abolghasemi H. A fast and efficient CNN model for B-ALL diagnosis and its subtypes classification using peripheral blood smear images. Int J Intell Syst. 2022;37(8):5113–33.CrossRef

48.

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.MathSciNet

49.

Hossin M, Sulaiman MN. A review on evaluation metrics for data classification evaluations. Int J Data Min Knowl Manag Process. 2015;5(2):1.CrossRef

50.

Powers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061. 2020.

51.

Selvaraju RR, Das A, Vedantam R, Cogswell M, Parikh D, Batra D. Grad-CAM: Why did you say that? arXiv preprint arXiv:1611.07450. 2016.

52.

Adler AI, Painsky A. Feature importance in gradient boosting trees with cross-validation feature selection. Entropy. 2022;24(5):687.ADSMathSciNetCrossRefPubMedPubMedCentral

Title: A comparative study of CNN-capsule-net, CNN-transformer encoder, and Traditional machine learning algorithms to classify epileptic seizure
Authors: Sergio Alejandro Holguin-Garcia
Ernesto Guevara-Navarro
Alvaro Eduardo Daza-Chica
Maria Alejandra Patiño-Claro
Harold Brayan Arteaga-Arteaga
Gonzalo A. Ruz
Reinel Tabares-Soto
Mario Alejandro Bravo-Ortiz
Publication date: 01-12-2024
Publisher: BioMed Central
Keywords: Epilepsy
Epileptic Seizure
Published in: BMC Medical Informatics and Decision Making / Issue 1/2024
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-024-02460-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

A comparative study of CNN-capsule-net, CNN-transformer encoder, and Traditional machine learning algorithms to classify epileptic seizure

Abstract

Introduction

Method

Result

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Introduction

Method

Result

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2024

MMIR: an open-source software for the registration of multimodal histological images

Which risk factor best predicts coronary artery disease using artificial neural network method?

Exploring machine learning strategies for predicting cardiovascular disease risk factors from multi-omic data

Feasibility of a wearable self-management application for patients with COPD at home: a pilot study

A hybrid framework for glaucoma detection through federated machine learning and deep learning models

Communicating the results of risk-based breast cancer screening through visualizations of risk: a participatory design approach