A Novel Progressive Attention-Based Bidirectional Encoder Transformer For Improved Cardiovascular Disease Detection

Dr. Matteo Bianchi; Prof. Liwen Zhao

Authors

Dr. Matteo Bianchi Department of Information Engineering, University of Pisa, Pisa, Italy
Prof. Liwen Zhao Institute of Artificial Intelligence, Tsinghua University, Beijing, China

Keywords:

Cardiovascular disease detection, progressive attention mechanism, bidirectional encoder transformer, deep learning

Abstract

Cardiovascular diseases (CVDs) remain the leading cause of mortality worldwide, underscoring the need for accurate and efficient diagnostic tools. This study proposes a novel Progressive Attention-Based Bidirectional Encoder Transformer (PABET) framework designed to enhance the detection of cardiovascular disease from clinical and physiological data. The model integrates progressive attention mechanisms that dynamically prioritize critical temporal and contextual features across multiple layers of the transformer architecture. The bidirectional encoder enables comprehensive representation learning by capturing both forward and backward dependencies inherent in sequential health records and electrocardiogram (ECG) signals. Experimental evaluations on benchmark cardiovascular datasets demonstrate that PABET outperforms conventional deep learning models, including recurrent neural networks and standard transformers, achieving superior accuracy, sensitivity, and specificity. The proposed approach offers a scalable and interpretable solution to improve early diagnosis and risk stratification of cardiovascular disease, supporting clinicians in making timely and informed decisions.

References

Jenifer A, Jeba G, Paulraj L, Kumar N, Yuvaraj T, Alen G, Edge-based heart disease prediction device using internet of things. In: 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC). Piscataway, NJ: IEEE; 2022. p. 1500–1504. doi:10.1109/ICAAIC53929.2022.9793104.

Ahmad GN, Ullah S, Algethami A, Fatima H, Akhter SM. Comparative study of optimum medical diagnosis of human heart disease using machine learning technique with and without sequential feature selection. IEEE Access. 2022;10: 23808–23828. doi:10.1109/ACCESS.2022.3153047.

Utsav A, Mohanty S, Bishwajeet K, Kumar T. IOT enabled heart disease detection system using KNN. In: 2022 13th International Conference on Computing Communication and Networking Technologies (ICCCNT). Piscataway, NJ: IEEE; 2022. p. 1–6. doi:10.1109/ICCCNT54827.2022.9984506.

Munagala NK, Langoju LR, Rani AD, Reddy DR. A smart IoT-enabled heart disease monitoring system using meta-heuristic-based Fuzzy-LSTM model. Biocybern Biomed Eng. 2022;42(4):1183–1204. doi:10.1016/j.bbe.2022.10.001.

Wang G, Lauri F, El Hassani AH. Feature selection by mRMR method for heart disease diagnosis. IEEE Access. 2022;10: 100786–100796. doi:10.1109/ACCESS.2022.3207492.

Safa M, Pandian A, Gururaj HL, Ravi V, Krichen M. Real time health care big data analytics model for improved QoS in cardiac disease prediction with IoT devices. Health Technol. 2023;13(3):473–483. doi:10.1007/s12553-023-00747-1.

Raheja N, Manocha AK. An IoT enabled secured clinical health care framework for diagnosis of heart diseases. Biomed Signal Process Control. 2023;80: 104368. doi:10.1016/j.bspc.2022.104368.

Sianga BE, Mbago MC, Msengwa AS. Predicting the prevalence of cardiovascular diseases using machine learning algorithms. Intell Based Med. 2025;11: 100199. doi:10.1016/j.ibmed.2025.100199.

Islam MN, Raiyan KR, Mitra S, Mannan MR, Tasnim T, Putul AO, Predictis: an IoT and machine learning-based system to predict risk level of cardio-vascular diseases. BMC Health Serv Res. 2023;23(1):171. doi:10.1186/s12913-023-09104-4.

Ramkumar G, Seetha J, Priyadarshini R, Gopila M, Saranya G. IoT-based patient monitoring system for predicting heart disease using deep learning. Measurement. 2023;218: 113235. doi:10.1016/j.measurement.2023.113235.

Rahman MZ, Akbar MA, Leiva V, Tahir A, Riaz MT, Martin-Barreiro C. An intelligent health monitoring and diagnosis system based on the internet of things and fuzzy logic for cardiac arrhythmia COVID-19 patients. Comput Biol Med. 2023;154: 106583. doi:10.1016/j.compbiomed.2023.106583.

Utsha UT, Hua Tsai I, Morshed BI. A smart health application for real-time cardiac disease detection and diagnosis using machine learning on ECG data. In: IFIP International Internet of Things Conference. Cham: Springer; 2023. p. 135–150. doi:10.1007/978-3-031-45878-1_10.

Akila D, Thyagaraj M, Senthil D, Adhikari S, Kavitha K. Internet of things heart disease detection with machine learning and EfficientNet-B0. In: International Conference on Micro-Electronics and Telecommunication Engineering. Cham: Springer; 2023. p. 457–468. doi:10.1007/978-981-99-9562-2_38.

Ezilarasan MR, Sathyasri B, MuthuKumaran D. IoT based detection and monitoring for coronary artery disease. In: 2023 9th International Conference on Smart Structures and Systems (ICSSS). Piscataway, NJ: IEEE; 2023. p. 1–5. doi:10.1109/ICSSS58085.2023.10407838.

Roy TS, Roy JK, Mandal N. Conv-random forest based IoT: a deep learning model based on CNN and random forest for classification and analysis of valvular heart diseases. IEEE Open J Instrum Meas. 2023;2: 2500717. doi:10.1109/OJIM.2023.3320765.

Yenurkar GK, Mal S, Wakulkar A, Umbarkar K, Bhat A, Bhasharkar A, Future prediction for precautionary measures associated with heart-related issues based on IoT prototype. Multimed Tools Appl. 2024;1–31. doi:10.1007/s11042-024-19135-6.

Mishra J, Tiwari M. IoT-enabled ECG-based heart disease prediction using three-layer deep learning and meta-heuristic approach. Signal Image Video Process. 2024;18(1):361–367. doi:10.1007/s11760-023-02743-4.

Hannan A, Cheema SM, Pires IM. Machine learning-based smart wearable system for cardiac arrest monitoring using hybrid computing. Biomed Signal Process Control. 2024;87: 105519. doi:10.1016/j.bspc.2023.105519.

Ullah T, Ullah SI, Ullah K, Ishaq M, Khan A, Ghadi YY, Machine learning-based cardiovascular disease detection using optimal feature selection. IEEE Access. 2024;12: 16431–16446. doi:10.1109/ACCESS.2024.3359910.

Naeem AB, Senapati B, Bhuva D, Zaidi A, Bhuva A, Sudman MS, Heart disease detection using feature extraction and artificial neural networks: a sensor-based approach. IEEE Access. 2024;12: 37349–37362. doi:10.1109/ACCESS.2024.3373646.

Razin V, Krasnov A, Karchkov D, Moskalenko V, Rodionov D, Zolotykh N, Solving the problem of diagnosing a disease by ECG on the PTB-XL dataset using deep learning. In: International Conference on Neuroinformatics. Cham: Springer; 2023. p. 13–21. doi:10.1007/978-3-031-44865-2_2.

Kumar NK, Sindhu GS, Prashanthi DK, Sulthana AS. Analysis and prediction of cardio vascular disease using machine learning classifiers. In: 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS). Coimbatore, India: IEEE; 2020. p. 15–21. doi:10.1109/ICACCS48705.2020.9074183.

Rahim A, Rasheed Y, Azam F, Anwar MW, Rahim MA, Muzaffar AW. An integrated machine learning framework for effective prediction of cardiovascular diseases. IEEE Access. 2021;9: 106575–106588. doi:10.1109/ACCESS.2021.3098688.

Abubaker MB, Babayiğit B. Detection of cardiovascular diseases in ECG images using machine learning and deep learning methods. IEEE Trans Artif Intell. 2022;4(2):373–382. doi:10.1109/TAI.2022.3159505.

Chakraborty S, Ide K, Balachandran B. Missing values imputation in ocean buoy time series data. Ocean Eng. 2025;318: 120145. doi:10.1016/j.oceaneng.2024.120145.

Benjamin M. Hybrid Feature Encoding Techniques for Ensemble Learning [Internet], ResearchGate, 2025.

Kappal S. Data normalization using median median absolute deviation MMAD based z-score for robust predictions vs. min–max normalization. Lond J Res Sci: Nat Form. 2019;19(4):39–44.

Carlini EM, Del Pizzo F, Giannuzzi GM, Lauria D, Mottola F, Pisani C. Online analysis and prediction of the inertia in power systems with renewable power generation based on a minimum variance harmonic finite impulse response filter. Int J Electr Power Energy Syst. 2021;131: 107042.

Zhou T, Du Y, Mao J, Peng C, Wang H, Zhou Z. Parallel attention multi-scale mandibular fracture detection network based on CenterNet. Biomed Signal Process Control. 2024;95: 106338. doi:10.1016/j.bspc.2024.106338.

Zheng X, Lei Z, Feng Z, Chen L. Legendre multi-wavelet transform and its application in bearing fault detection. Appl Sci. 2023;14(1):219. doi:10.3390/app14010219.

Fang N, Cao Q. Leaf in wind optimization: a new metaheuristic algorithm for solving optimization problems. IEEE Access. 2024;12: 56291–56308. doi:10.1109/ACCESS.2024.3390670.

Frontiers in Emerging Artificial Intelligence and Machine Learning

Article Details Page