ADVANCED MULTIMODAL AI FRAMEWORK FOR ENHANCED DIABETIC RETINOPATHY DIAGNOSIS AND SEVERITY CLASSIFICATION

Abstract

Diabetic Retinopathy (DR) continues to be a primary cause of visual impairment and blindness among individuals with diabetes globally, affecting millions and underscoring the urgent need for robust, scalable screening mechanisms. This extended study presents an advanced multimodal deep learning framework that synergistically combines fundus photography and Optical Coherence Tomography (OCT) imaging modalities to achieve superior detection and severity grading of DR. By employing a hybrid architecture that integrates Convolutional Neural Networks (CNNs) for spatial feature extraction and Transformers for capturing long-range dependencies in sequential data, the proposed model leverages attention mechanisms to fuse multimodal inputs effectively. Evaluated on benchmark datasets such as EyePACS, IDRiD, and Duke OCT, the framework demonstrates exceptional performance metrics: 98% accuracy, 96% sensitivity, 97% specificity, and an Area Under the Curve (AUC) of 0.99 for binary classification tasks, surpassing existing state-of-the-art approaches. To enhance interpretability, the model incorporates Explainable AI (XAI) techniques, including Gradient-weighted Class Activation Mapping (Grad-CAM), for precise lesion localization. Class imbalance issues are mitigated through sophisticated data augmentation strategies, including Synthetic Minority Over-sampling Technique (SMOTE) and generative adversarial networks (GANs)-based synthesis. This work provides detailed mathematical derivations for custom loss functions, evaluation metrics, and optimization algorithms, accompanied by comprehensive visualizations such as confusion matrices, Receiver Operating Characteristic (ROC) curves, precision-recall curves, and training convergence plots. From a Computer Science Engineering (CSE) perspective, the framework emphasizes computational efficiency, enabling real-time inference on edge devices and potential deployment in resource-constrained environments, thereby reducing healthcare costs and improving accessibility in underserved regions. This extension expands on the original contributions by including in-depth ablation studies, comparative analyses with recent 2025 models, ethical considerations, and deployment strategies.