Time-Aware and Multi-Source Feature Fusion for Transformer-Based Medical Text Analysis

Abstract

This study proposes an improved Transformer-based model for automatic classification and risk prediction using electronic medical records (EMRs). The goal is to address the limitations of traditional methods in semantic understanding when processing unstructured medical texts. The model introduces a hierarchical attention mechanism, a multi-source feature fusion structure, and time-aware embeddings. These enhancements improve the model's ability to capture semantic relationships and track patient condition progression. In the experimental section, the study uses the MIMIC-III dataset to design comparative experiments across three dimensions: embedding strategies, proportion of unstructured text, and prediction window length. These experiments validate the model's advantages in classification accuracy and risk prediction performance. In addition, a joint loss function is constructed to enable multi-task optimization. This further improves the model's adaptability to multi-objective prediction tasks. Experimental results show that the proposed model outperforms mainstream pre-trained language models across all evaluation metrics. It demonstrates strong performance stability and effective text understanding capabilities.