Beyond Doctors: Future Health Prediction from Multimedia and Multimodal Observations

Although chronic diseases cannot be cured, they can be effectively controlled as long as we understand their progressions based on the current observational health records, which is often in the form of multimedia data. A large and growing body of literature has investigated the disease progression problem. However, far too little attention to date has been paid to jointly consider the following three observations of the chronic disease progression: 1) the health statuses at different time points are chronologically similar; 2) the future health statuses of each patient can be comprehensively revealed from the current multimedia and multimodal observations, such as visual scans, digital measurements and textual medical histories; and 3) the discriminative capabilities of different modalities vary significantly in accordance to specific diseases. In the light of these, we propose an adaptive multimodal multi-task learning model to co-regularize the modality agreement, temporal pro...

• Type: Journal
• Year: 2015
• Where: MM
• Authors: Liqiang Nie, Luming Zhang, Yi Yang, Meng Wang, Richang Hong, Tat-Seng Chua
• Keywords: MM 2015, Multimedia