AI-Powered EHR Engineering for Precision Healthcare
Keywords:
EHR Analytics, Precision Care, Health Data, Data Ingestion, Data Pipelines, Data Preprocessing, Data Lakes, Cloud Systems, On Premise, Clinical Analytics, Patient Stratification, Treatment Optimization, Machine Learning, Model Retraining, Data Sharing, Reproducibility, Population Health, Atrial Fibrillation, Emergency Care, Clinical Decision.Abstract
Research aims, methods, key results, and implications for scalable EHR analytics and precision care. Electronic Health Records (EHRs) are a historical, comprehensive, and continually updated source of health data for patients. Although EHRs have been widely used for retrospective population studies, research prototypes, and clinical decision support systems, the actual uptake of AI-driven technologies in clinical practice and patient care remains limited. Automatic ingestion of clinical data, standardized data lakes, and periodic model retraining are essential for seamless operation and scalable usage. The objectives of this research are to design data ingestion and preprocessing pipelines that run automatically, to support cloud- and on-premises-based clinical analytics, and to apply patient stratification and personalized treatment optimization methods.
The proposed technologies are scalable, reproducible, and support precision healthcare delivery, with a global public health perspective. The results of the clinical data engineering effort are publicly available. Reproducibility was achieved by adhering to open-source principles, data-sharing best practices, and the use of common data-collection and storage formats. Population analyses demonstrated predefined hypotheses in the domain of obesity and revealed new insights into atrial fibrillation. In acute care and emergency medicine, AI-supported rapid data ingestion and the clinical application of complex models during patient triage were successfully demonstrated. Although deployment-specific bias was identified in these specific domains, adequate technical control of both cloud-based and on-premises configurations was supported.
References
1. Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., Zhang, Y., Magoc, T., Harle, C. A., Lipori, G., Mitchell, D. A., Hogan, W. R., Shenkman, E. A., Bian, J., & Wu, Y. (2022). A large language model for electronic health records. npj Digital Medicine, 5(1), 194.
2. Li, J., Cairns, B. J., Li, J., & Zhu, T. (2023). Generating synthetic mixed-type longitudinal electronic health records for artificial intelligent applications. npj Digital Medicine, 6(1), 98.
3. Cheng, A. C., Banasiewicz, M. K., Johnson, J. D., Sulieman, L., Kennedy, N., Delacqua, F., Lewis, A. A., Joly, M. M., Bistran-Hall, A., Collins, S., Self, W. H., Shotwell, M. S., Lindsell, C. J., & Harris, P. A. (2023). Evaluating automated electronic case report form data entry from electronic health records. Journal of Clinical and Translational Science, 7(1), e29.
4. Suryanarayanan, P., Epstein, E. A., Malvankar, A., Lewis, B. L., DeGenaro, L., Liang, J. J., Tsou, C. H., & Pathak, D. (2021). Timely and efficient AI insights on EHR: System design. AMIA Annual Symposium Proceedings, 2020, 1180–1189.
5. Senathirajah, Y., Cho, H., Fawcett, J., Mondejar, K. M., Cato, K., Broadwell, P., & Yoon, S. (2022). Application of natural language processing to learn insights on the clinician’s lived experience of electronic health records. Studies in Health Technology and Informatics, 289, 81–84.
6. Johnson, K. B., Stead, W. W., & Neta, G. (2022). Making electronic health records both SAFER and SMARTER. JAMA, 328(6), 523–524.
7. Johnson, K. B., Neuss, M. J., & Detmer, D. E. (2021). Electronic health records and clinician burnout: A story of three eras. Journal of the American Medical Informatics Association, 28(5), 967–973.
8. Bompelli, A., Wang, Y., Wan, R., Singh, E., Zhou, Y., Xu, L., Oniani, D., Kshatriya, B. S. A., Balls-Berry, J. E., & Zhang, R. (2021). Social and behavioral determinants of health in the era of artificial intelligence with electronic health records: A scoping review. Health Data Science, 1(1), 1–15.
9. Li, I., Pan, J., Goldwasser, J., Verma, N., Wong, W. P., Nuzumlalı, M. Y., Rosand, B., Li, Y., Zhang, M., Chang, D., Taylor, R. A., Krumholz, H. M., & Radev, D. (2021). Neural natural language processing for unstructured data in electronic health records: A review. Yearbook of Medical Informatics, 30(1), 188–198.
10. Mohsen, F., Ali, H., El Hajj, N., & Shah, Z. (2022). Artificial intelligence-based methods for fusion of electronic health records and imaging data. Information Fusion, 88, 102–118.
11. Caruana, A., Bandara, M., Musial, K., Catchpoole, D., & Kennedy, P. J. (2023). Machine learning for administrative health records: A systematic review of techniques and applications. Artificial Intelligence in Medicine, 143, 102609.
12. Wang, L., Porter, B., Maynard, C., Evans, G., Bryson, C., Sun, H., Gupta, I., & Gupta, S. (2021). Predictive models for chronic disease management using electronic health records. BMC Medical Informatics and Decision Making, 21(1), 112.
13. Topol, E. J. (2021). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 27(1), 44–56.
14. Rajkomar, A., Dean, J., & Kohane, I. (2021). Machine learning in medicine. New England Journal of Medicine, 385(14), 1347–1358.
15. Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2021). Deep learning for healthcare: Review, opportunities and challenges. Briefings in Bioinformatics, 22(2), 1236–1246.
16. Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2021). A guide to deep learning in healthcare. Nature Medicine, 27(1), 24–29.
17. Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2021). Deep EHR: A survey of recent advances in deep learning techniques for electronic health record analysis. IEEE Journal of Biomedical and Health Informatics, 25(8), 2989–3003.
18. Luo, Y., Thompson, W. K., Herr, T. M., Zeng, Z., Berendsen, M. A., Jonnalagadda, S. R., Jafari, N., Carson, M. B., Starren, J. B., & Horvath, M. M. (2021). Natural language processing for EHR-based computational phenotyping. Journal of Biomedical Informatics, 118, 103785.
19. Zhang, Y., Chen, Q., Yang, Z., Lin, H., & Lu, Z. (2021). BioWordVec, improving biomedical word embeddings with subword information and MeSH. Scientific Data, 8(1), 52.
20. Xiao, C., Choi, E., & Sun, J. (2021). Opportunities and challenges in developing deep learning models using electronic health records data. Journal of the American Medical Informatics Association, 28(1), 95–104.
21. Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021). Med-BERT: Pretrained contextualized embeddings on large-scale structured electronic health records. npj Digital Medicine, 4(1), 86.
22. Li, R. C., Asch, S. M., & Shah, N. H. (2021). Developing a delivery science for artificial intelligence in healthcare. NPJ Digital Medicine, 4(1), 107.
23. Beam, A. L., & Kohane, I. S. (2021). Big data and machine learning in health care. JAMA, 325(13), 1317–1318.
24. Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., Ossorio, P. N., Thadaney-Israni, S., & Goldenberg, A. (2021). Do no harm: A roadmap for responsible machine learning for health care. Nature Medicine, 27(8), 1337–1340.
25. Matheny, M. E., Israni, S. T., Ahmed, M., & Whicher, D. (2021). Artificial intelligence in health care: The hope, the hype, the promise, the peril. National Academy of Medicine Perspectives, 1, 1–15.
26. Sendak, M. P., D’Arcy, J., Kashyap, S., Gao, M., Nichols, M., Corey, K., Ratliff, W., & Balu, S. (2021). A path for translation of machine learning products into healthcare delivery. EMJ Innovations, 5(1), 55–62.
27. Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2022). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 4(11), e745–e750.
28. Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022). Large language models are few-shot clinical information extractors. EMNLP Proceedings, 1998–2005.
29. Lehman, E., Jain, S., Pichotta, K., Goldberg, Y., & Wallace, B. C. (2023). Hype or reality? Large language models for clinical NLP. Nature Medicine, 29(1), 30–35.
30. Singhal, K., Azizi, S., Tu, T., Mahdavi, S., Wei, J., Chung, H., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Scharli, N., Chowdhery, A., Mansfield, P., & Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180.
31. Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616(7956), 259–265.
32. Wornow, M., Xu, Y., Thapa, R., Patel, B., Steinberg, E., Fleming, S., Pfeffer, M., Jones, S., & Shah, N. H. (2023). The shaky foundations of large language models and foundation models for electronic health records. NPJ Digital Medicine, 6(1), 135.
33. Chen, I. Y., Joshi, S., Ghassemi, M., & Bärnighausen, T. (2022). Treating health disparities with artificial intelligence. Nature Medicine, 28(4), 666–668.
34. Zhang, Z., Ho, K. M., & Hong, Y. (2022). Machine learning for the prediction of volume responsiveness in critical care. Journal of Intensive Care, 10(1), 9.
35. Subbaswamy, A., & Saria, S. (2022). From development to deployment: Dataset shift, causality, and shift-stable models in healthcare AI. Biostatistics, 23(2), 289–303.
36. McCradden, M. D., Stephenson, E. A., & Anderson, J. A. (2022). Clinical research underlies ethical integration of healthcare artificial intelligence. Nature, 607(7919), 39–41.
37. He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2022). The practical implementation of artificial intelligence technologies in medicine. Nature Medicine, 28(1), 30–36.
38. Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2022). Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 31(2), 148–155.
39. Chen, M., Decary, M., & Gagnon, M. P. (2022). Integrating AI into healthcare workflows: A systematic review. Journal of Medical Systems, 46(6), 43.
40. Kalliamvakou, E., Bird, C., Zimmermann, T., & DeLine, R. (2022). AI-assisted software engineering for health informatics systems. IEEE Software, 39(5), 64–71.
41. Wang, F., Kaushal, R., & Khullar, D. (2022). Should health care demand interpretable artificial intelligence or accept black box medicine? Annals of Internal Medicine, 175(8), 1209–1210.
42. Sendak, M. P., Gao, M., Nichols, M., Lin, A., & Balu, S. (2022). Machine learning in healthcare operations: Opportunities and challenges. Healthcare, 10(1), 100607.
43. Adibuzzaman, M., DeLaurentis, P., Hill, J., & Benneyworth, B. (2021). Big data in healthcare – The promises, challenges and opportunities from a research perspective. Health Information Science and Systems, 9(1), 19.
44. Verma, A. A., Murray, J., Greiner, R., Cohen, J. P., Shojania, K. G., Straus, S. E., & Bell, C. M. (2021). Implementing machine learning in medicine. CMAJ, 193(35), E1351–E1357.
45. Goldstein, B. A., Navar, A. M., Carter, R. E., & Moving Beyond Regression Techniques. (2021). Opportunities and challenges in developing risk prediction models with EHR data. Journal of the American Heart Association, 10(7), e019634.
46. Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2021). Artificial intelligence in healthcare: Past, present and future. Stroke and Vascular Neurology, 6(2), 230–243.
47. Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2021). A governance model for the application of AI in health care. Journal of the American Medical Informatics Association, 28(3), 491–497.
48. Kaissis, G., Makowski, M., Rückert, D., & Braren, R. (2021). Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 3(6), 473–484.
49. Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, Q., Costa, A. B., Wood, B. J., Tsai, C. S., & others. (2021). Federated learning for predicting clinical outcomes in patients with COVID-19. Nature Medicine, 27(10), 1735–1743.
50. Jordon, J., Yoon, J., & van der Schaar, M. (2021). PATE-GAN: Generating synthetic data with differential privacy guarantees. International Conference on Learning Representations, 1–14.
51. Xu, J., Wang, F., Perez, M., Wang, Y., & Jiang, X. (2022). Privacy-preserving machine learning for healthcare applications. IEEE Reviews in Biomedical Engineering, 15, 312–329.
52. Luo, J., Wu, M., Gopukumar, D., & Zhao, Y. (2022). Big data application in biomedical research and health care: A literature review. Biomedical Informatics Insights, 14, 1–15.
53. Wu, Y., Jiang, X., Kim, J., & Ohno-Machado, L. (2022). Federated learning for healthcare informatics. Journal of Healthcare Informatics Research, 6(2), 123–145.
54. Bohr, A., & Memarzadeh, K. (2022). The rise of artificial intelligence in healthcare applications. Artificial Intelligence in Healthcare, 25–60.
55. Choi, E., Xu, Z., Li, Y., Dusenberry, M. W., Flores, G., Xue, Y., & Dai, A. M. (2022). Learning the graphical structure of electronic health records with graph neural networks. Machine Learning for Healthcare Conference Proceedings, 89–103.
56. Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., & Liu, H. (2021). Clinical information extraction applications: A literature review. Journal of Biomedical Informatics, 120, 103846.
57. Seneviratne, M. G., Shah, N. H., & Chu, L. (2022). Bridging the implementation gap of machine learning in healthcare. BMJ Innovations, 8(1), 45–52.
58. Chen, J. H., & Asch, S. M. (2022). Machine learning and prediction in medicine—Beyond the peak of inflated expectations. New England Journal of Medicine, 386(26), 2507–2509.
59. Haug, C. J., Drazen, J. M., & Kohane, I. S. (2023). Generative artificial intelligence in medicine and the future of healthcare. New England Journal of Medicine, 389(17), 1569–1571.
60. Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023). Capabilities of GPT-4 on medical challenge problems. arXiv Preprint, arXiv:2303.13375.
61. Agrawal, A., Choudhary, A., & Narayanan, V. (2023). Explainable artificial intelligence for healthcare: A review of challenges and opportunities. Artificial Intelligence Review, 56(8), 7565–7598.
62. Topol, E. J. (2023). ChatGPT and generative artificial intelligence in healthcare. Nature Medicine, 29(6), 1310–1312.
Additional Files
Published
Data Availability Statement
The results of the clinical data engineering effort are publicly available.
