Our work in multimodal learning includes stepwise story illustration using images, news image caption generation, multimodal fake news detection, and multimodal event representation learning.

Participants

Vishwash Batra, Lin Gui, Wenjia Zhang

Publications

• W. Zhang, L. Gui and Y. He. Supervised Contrastive Learning for Multi-modal Unreliable News Detection in COVID-19 Pandemic, The 30th ACM International Conference on Information and Knowledge Management (CIKM), Nov. 2021.
• D. Zhou, K. Sun, M. Hu and Y. He. Image Generation from Text with Entity Information Fusion, Knowledge-Based Systems, Vol. 227, 107200, 2021.
• L. Zhang, D. Zhou, Y. He and Z. Yang. MERL: Multimodal Event Representation Learning in Heterogeneous Embedding Spaces, The 35th AAAI Conference on Artificial Intelligence (AAAI), Feb. 2021.
• V. Batra, A. Haldar, Y. He, H. Ferhatosmanoglu, G. Vogiatzis and T. Guha. Variational Recurrent Sequence-to-Sequence Retrieval for Stepwise Illustration. The 42nd European Conference on Information Retrieval (ECIR), Lisbon, Portugal, Apr. 2020.
• M. Hu, D. Zhou and Y. He. Variational Conditional GAN for Fine-grained Controllable Image Generation. The 11th Asian Conference on Machine Learning (ACML), Nagoya, Japan, Nov. 2019.
• V. Batra, Y. He and G. Vogiatzis. A Deep Learning Approach to Automatic Caption Generation for News Images, The 11th International Conference on Language Resources and Evaluation (LREC), Miyazaki, Japan, May 2018.