See the PhD Opportunities section to see if this project is currently open for applications via MIBTP.
Please Note: The main page lists projects via BBSRC Research Theme(s) quoted and then relevant Topic(s).
Multimodal Learning and Vision-Language Models for Understanding Human Organ Proportions
Secondary Supervisor(s): Dr Mark Thomas
University of Registration: University of Birmingham
BBSRC Research Themes:
Project Outline
The human body works as a system, and how organ size relates to height, weight and other organs determines the way they are assessed, matched and monitored in clinical practice. Healthy organs provide the reference, showing what these relationships look like when nothing is wrong and giving a basis for identifying pathological change. However, establishing organ relationships is challenging, as it requires many healthy organs to be measured across large numbers of people. Recent studies focus on each organ independently, without capturing the links between them. They are also usually limited to a single imaging modality. Vision-language models can now be prompted to segment a named organ in any modality, and multimodal learning can combine imaging with radiology reports and health records, enabling organ relationships to be measured at scale.
Aims: This project will use AI models to measure organ size and discover its relationship to height, weight and other organs. Objectives include:
- Develop vision-language methods that segment and measure named organs across multiple imaging modalities.
- Develop multimodal models that identify healthy organs using imaging, radiology reports and health records.
- Characterise how organ size scales with height, weight and other organs, and test whether this improves on age and sex alone.
Methods: Models will be trained on large public CT, MRI and PET datasets, using self-supervised pretraining, vision-language grounding and cross-modality learning. Training will run on national supercomputers and over 200 NVIDIA A100 and H100 GPUs. Measurements will be validated against released labels and assessed with clinical experts.
Outcomes and Impact: The project will deliver open models, benchmarks and a description of how human organs scale together. The student will gain training in multimodal learning, vision-language models, medical imaging and quantitative biology, plus generic skills through the Postgraduate School.
