Department Events

Title: Quantifying Pathology Foundation Model Robustness against Medical Center Variation: the Robustness Index

Abstract: Pathology Foundation Models (FMs) hold great promise for healthcare. Their clinical application is challenged by well-documented variations between medical centers, which can potentially lead to biased downstream models. To enable the use of pathology FMs in clinical practice, it is essential to ensure they are robust to such variation. This requires the ability to measure and quantify robustness. We measure how strongly current pathology FMs encode biological features like tissue and cancer type, and how strongly they encode confounding medical center signatures introduced by staining procedure and other differences. The relation between these two factors defines the Robustness Index. This novel robustness metric quantifies to what degree biological features dominate confounding features. We evaluate the robustness of current pathology FMs. We find that all FMs encode medical centers to some degree, and that medical center variation can dominate the organization of the embedding space, depending on the application domain. The influence of this confounding information on the embedding space varies, and significant differences in the robustness index are observed. The robustness index provides a critical new benchmark for evaluating and improving pathology FMs, accelerating progress towards their safe and reliable clinical adoption.

Bio: Edwin de Jong has been fascinated by how humans and machines can produce intelligent behavior for over three decades. He studied at Delft University of Technology and received his PhD from the VUB AI Lab in 2000. As a postdoctoral researcher at Prof. Jordan Pollack’s DEMO Lab at Brandeis University, he started a research line in representation learning and open-ended self-improvement which he continued in The Netherlands at Utrecht University. In 2004/2005, he co-founded Adapticon, among the first companies worldwide to apply LSTM in industry. After other roles in the tech industry, as of 2018 Edwin's focus is on helping AI transform healthcare. At ScreenPoint Medical, he contributed to the Transpara algorithm which was found to substantially reduce radiologist workload and identify 29% more cancer cases in the MASAI randomized controlled trial; a milestone recognized as a Notable Advance by Nature Medicine. His current focus is on developing robust pathology and multimodal foundation models as Principal Machine Learning Scientist at Aignostics, a leading European scaleup in AI-powered pathology, where he works to advance computational diagnostics for drug discovery and precision medicine.

Paper Link: [2501.18055] Current Pathology Foundation Models are unrobust to Medical Center Differences

How to attend: Either turn up to the event on the day, or if you want to attend online then please contact Adam Shephard (adam.shephard@warwick.ac.uk) for more details.