Improvised Failure Detection for Centrifugal Pumps Using Delta and Python: How Effectively Iot Sensors Data Can Be Processed and Stored for Monitoring to Avoid Latency in Reporting

Sai Nikhil Donthi

doi:10.64917/fecsit/Volume02Issue10-03

Open Access

Improvised Failure Detection for Centrifugal Pumps Using Delta and Python: How Effectively Iot Sensors Data Can Be Processed and Stored for Monitoring to Avoid Latency in Reporting

https://doi.org/10.64917/fecsit/Volume02Issue10-03

PDF

Sai Nikhil Donthi ¹

⁴ Department of Software Engineering, University of Houston Clear Lake, Client – Oil and Gas Industry Houston, Texas , USA

Abstract

Centrifugal pumps are critical in industrial operations where unplanned failures can cause costly downtime and safety risks. Traditional maintenance methods often fail to detect early anomalies in vibration, motor current, and seal integrity. This study addresses that gap by developing an IoT-enabled failure detection framework that leverages Delta Lake and Python to enable scalable, real-time monitoring and predictive maintenance. The proposed system integrates MEMS accelerometers, SCT 013 current sensors, and leak detection modules through an ESP32 microcontroller, ensuring reliable data acquisition and ACID-compliant storage. Delta Lake facilitates seamless handling of both streaming and batch data while maintaining version control and schema integrity. Experimental validation using real and simulated datasets demonstrates reliable anomaly detection, efficient schema evolution, and resilience under high-frequency sensor loads. The research provides strong quantitative validation that Delta Lake with Python enables 25% less downtime, 30% higher maintenance efficiency and with improved real-time responsiveness (<500 ms), proving Delta Lake as a more robust, scalable, ACID- compliant, and high-performance data framework for IoT- based predictive maintenance in centrifugal pumps compared to traditional formats like Parquet. Overall, this research contributes to enhance Industry 4.0-driven predictive maintenance by integrating dependable IoT sensing with resilient data management and analytics for centrifugal pump monitoring.

How to Cite

Sai Nikhil Donthi. (2025). Improvised Failure Detection for Centrifugal Pumps Using Delta and Python: How Effectively Iot Sensors Data Can Be Processed and Stored for Monitoring to Avoid Latency in Reporting. Frontiers in Emerging Computer Science and Information Technology, 2(10), 24–37. https://doi.org/10.64917/fecsit/Volume02Issue10-03

⬇ Endnote/Zotero/Mendeley (RIS) ⬇ BibTeX

References

📄 Donthi, Sai Nikhil (2025). Evaluating effectiveness of Delta Lake over Parquet in Python pipeline. International Journal of Data Science and Machine Learning, 5(2), 126–144. https://doi.org/10.55640/ijdsml-05-02-12

📄 Luo, Y., Zhang, W., Fan, Y., Han, Y., Li, W., & Acheaw, E. (2021). Analysis of vibration characteristics of centrifugal pump mechanical seal under wear and damage degree. Shock and Vibration, 2021, Article 6670741. https://doi.org/10.1155/2021/6670741

📄 Kumar, V. N., V. A., & Kumar, M. L. H. (2024).IoT-based predictive maintenance for electrical machines and industrial automation. International Research Journal of Modern Engineering and Technology Science, 6(12), 4013–4027. https://doi.org/10.56726/IRJMETS65758

📄 Chevtchenko, S. F., et al. (2023, October). Predictive maintenance model based on anomaly detection in induction motors: A machine learning approach using real-time IoT data. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, 3173–3180. https://doi.org/10.3850/978-981-18-8071-1_P578-cd

📄 Varanis, M., Silva, A., Mereles, A., & Pederiva, R. (2018, November). MEMS accelerometers for mechanical vibrations analysis: A comprehensive review with applications. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 40(11). https://doi.org/10.1007/S40430-018-1445-5

📄 Y. Ziegler, S., Woodward, R. C., Iu, H. H. C., & Borle, L. J. (2009, April). Current sensing techniques: A review. IEEE Sensors Journal, 9(4), 354–376. https://doi.org/10.1109/JSEN.2009.2013914

📄 Bruinsma, S., Geertsma, R. D., Loendersloot, R., & Tinga, T. (2024, February). Motor current and vibration monitoring dataset for various faults in an E-motor-driven centrifugal pump. Data in Brief, 52. https://doi.org/10.1016/J.DIB.2023.109987

📄 Zöller, M. A., Mauthe, F., Zeiler, P., Lindauer, M., & Huber, M. F. (2023). Automated machine learning for remaining useful life predictions. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, 2907–2912. https://doi.org/10.1109/SMC53992.2023.10394031

📄 Balali, F., Nouri, J., Nasiri, A., & Zhao, T. (2020, January). Data intensive industrial asset management: IoT-based algorithms and implementation (pp. 1– 236). https://doi.org/10.1007/978-3-030-35930-0 Wu, F., Wu, Q., Tan, Y., & Xu, X. (2024, May 27).

📄 Remaining useful life prediction based on deep learning: A survey. Sensors (Basel), 24(11), 3454. https://doi.org/10.3390/s24113454

📄 Garcia, J., Rios-Colque, L., Peña, A., & Rojas, L. (2025). Condition monitoring and predictive maintenance in industrial equipment: An NLP- assisted review of signal processing, hybrid models, and implementation challenges. Applied Sciences, 15(10), 5465. https://doi.org/10.3390/app15105465

📄 Jang, H.-C., & Chang, H.-P. (2024). AL-powered EdgeFL: Achieving low latency and high accuracy in federated learning. In Proceedings of the 2024 IEEE 99th Vehicular Technology Conference (VTC2024- Spring) (pp. 1–5). IEEE. https://doi.org/10.1109/VTC2024-Spring62846.2024.10683166

📄 Albshaier, L., Almarri, S., & Albuali, A. (2025). Federated learning for cloud and edge security: A systematic review of challenges and AI opportunities. Electronics, 14(5), 1019. https://doi.org/10.3390/electronics14051019

📄 Zhan, S., Huang, L., Luo, G., Zheng, S., Gao, Z., & Chao, H. C. (2025). A review on federated learning architectures for privacy-preserving AI: Lightweight and secure cloud–edge–end collaboration. Electronics, 14(13), 2512. https://doi.org/10.3390/electronics14132512

Most read articles by the same author(s)

Sai Nikhil Donthi, Ansible-Based End-To-End Dynamic Scaling on Azure Paas for Refinery Turnarounds: Cold-Start Latency and Cost–Performance Trade-Offs , Frontiers in Emerging Computer Science and Information Technology: Vol. 2 No. 11 (2025): Volume 02 Issue 11 November

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors retain the copyright of their articles published in this journal. All articles are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited.