Title: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities

Abstract: Biomedical image analysis is fundamental for biomedical discovery. Holistic image analysis comprises interdependent subtasks such as segmentation, detection and recognition, which are tackled separately by traditional approaches. Here, we propose BiomedParse, a biomedical foundation model that can jointly conduct segmentation, detection and recognition across nine imaging modalities. This joint learning improves the accuracy for individual tasks and enables new applications such as segmenting all relevant objects in an image through a textual description. To train BiomedParse, we created a large dataset comprising over 6 million triples of image, segmentation mask and textual description by leveraging natural language labels or descriptions accompanying existing datasets. We showed that BiomedParse outperformed existing methods on image segmentation across nine imaging modalities, with larger improvement on objects with irregular shapes. We further showed that BiomedParse can simultaneously segment and label all objects in an image. In summary, BiomedParse is an all-in-one tool for biomedical image analysis on all major image modalities, paving the path for efficient and accurate image-based biomedical discovery.

Bio: Theodore Zhao is a Senior Applied Scientist at Microsoft Health and Life Sciences Research, working on multimodal biomedical imaging models as well as biomedical natural language processing. Theodore earned his PhD in Applied Mathematics degree from University of Washington, where his research applied machine learning, stochastic modeling and optimization to applications in finance and healthcare. His research interests focus on machine learning, self-supervised learning, multimodal models, and mathematical modeling.

Paper Link: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities | Nature MethodsLink opens in a new windowLink opens in a new window

How to attend: Either turn up to the event on the day, or if you want to attend online then please contact Adam Shephard (adam.shephard@warwick.ac.uk) for more details.

Title: AI Agents in Oncology

Abstract: AI agents expand upon traditional AI systems by autonomously executing multi-step tasks in cancer research and oncology. These systems can interact with software, plan iteratively, and suggest therapeutic strategies or optimize drug design with minimal human input. AI agents show promise in automating intellectual tasks in cancer research and supporting clinical decision-making in oncology. Significant challenges remain regarding regulatory frameworks, interpretability, and ethical considerations for healthcare implementation. This talk provides a primer on AI agents in cancer research and oncology, examining their capabilities, applications, and constraints.

Bio: Professor Jakob Kather holds dual appointments in medicine and computer science at the Technical University (TU) Dresden, Germany, serves as a senior physician in medical oncology at the University Hospital Dresden and holds an additional affiliation with the National Center for Tumor Diseases (NCT) in Heidelberg. His research is focused on applying artificial intelligence in precision oncology. Prof. Kather’s research team at TU Dresden is using deep learning techniques to analyze a spectrum of clinical data, including histopathology, radiology images, textual records, and multimodal datasets. Guided by the belief that medical and tech expertise needs to be combined, medical researchers in his team learn computer programming and data analysis, while computer scientists are immersed in cancer biology and oncology. Prof. Kather chairs the “Working group on Artificial Intelligence” at the German Society of Hematology and Oncology (DGHO) and is a member of the pathology task force of the American Association for Cancer Research (AACR). His work is supported by numerous European and national grants, which enable the team to develop new deep learning methods for medical data analysis techniques and to apply them in precision oncology.nk opens in a new windowLink opens in a new windowLink opens in a new window

How to attend: Either turn up to the event on the day, or if you want to attend online then please contact Adam Shephard (adam.shephard@warwick.ac.uk) for more details.