MEDIMP: 3D Medical Images with clinical Prompts from limited tabular data for renal transplantation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

MEDIMP: 3D Medical Images with clinical Prompts from limited tabular data for renal transplantation

Abstract

Renal transplantation emerges as the most effective solution for end-stage renal disease. Occurring from complex causes, a substantial risk of transplant chronic dysfunction persists and may lead to graft loss. Medical imaging plays a substantial role in renal transplant monitoring in clinical practice. However, graft supervision is multi-disciplinary, notably joining nephrology, urology, and radiology, while identifying robust biomarkers from such high-dimensional and complex data for prognosis is challenging. In this work, taking inspiration from the recent success of Large Language Models (LLMs), we propose MEDIMP -- Medical Images with clinical Prompts -- a model to learn meaningful multi-modal representations of renal transplant Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE MRI) by incorporating structural clinicobiological data after translating them into text prompts. MEDIMP is based on contrastive learning from joint text-image paired embeddings to perform this challenging task. Moreover, we propose a framework that generates medical prompts using automatic textual data augmentations from LLMs. Our goal is to learn meaningful manifolds of renal transplant DCE MRI, interesting for the prognosis of the transplant or patient status (2, 3, and 4 years after the transplant), fully exploiting the limited available multi-modal data most efficiently. Extensive experiments and comparisons with other renal transplant representation learning methods with limited data prove the effectiveness of MEDIMP in a relevant clinical setting, giving new directions toward medical prompts. Our code is available at https://github.com/leomlck/MEDIMP.
Fichier principal
Vignette du fichier
midl23_85_camera_ready.pdf (718.58 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04040697 , version 1 (22-03-2023)
hal-04040697 , version 2 (29-04-2023)

Licence

Attribution

Identifiers

  • HAL Id : hal-04040697 , version 2

Cite

Leo Milecki, Vicky Kalogeiton, Sylvain Bodard, Dany Anglicheau, Jean-Michel Correas, et al.. MEDIMP: 3D Medical Images with clinical Prompts from limited tabular data for renal transplantation. MIDL 2023 - Medical Imaging with Deep Learning, Jul 2023, Nashville (Tennessee), United States. ⟨hal-04040697v2⟩
102 View
219 Download

Share

Gmail Facebook X LinkedIn More