Abstract

3D reconstruction of the human body shape is a fundamental problem in computer vision, which is valuable for various human-centric applications such as computer animation, virtual reality, and clothing design, to name a few. 3D scanning is a popular technology for acquiring the geometry of a subject based on which a 3D body reconstruction can be produced. Although countless body scanners were developed to meet different industrial requirements and a lot of advanced algorithms were proposed for optimizing the reconstructed body models, many problems are still not properly solved. These problems, however, are difficult to address using conventional methods.

Recent years have witnessed the rapid development of artificial intelligence, especially deep learning. Encouraged by the significant success of deep learning in image processing, an increasing number of researchers attempted to extend deep learning to deal with 3D data. Following this trend, we proposed deep learning-based solutions to several challenges existing in modern 3D body scanning and reconstruction.

In this presentation, we focus on three challenges of 3D body scanning, namely, (i) estimation of body shape under clothing, (ii) registration of non-overlapping point clouds, and (iv) animatable body reconstruction using a single depth camera. The first challenge arises from the fact that existing 3D scanning solutions require the subjects to get scanned with minimal clothing as the scanning device can only record the outmost surface of objects. This scanning procedure is inconvenient to most people and is also an infringement of the right to privacy. The second challenge is a classical problem: partial point cloud registration. We found that existing methods mainly rely on the assumption that the source and the target point clouds have sufficient overlap and none of them could handle non-overlapping registration. The last challenge is addressed as many applications demand dynamic human body models. Traditional methods require expensive professional devices to produce such models.

We have addressed these three challenges by leveraging the deep learning paradigm. Our first contribution is to propose the first deep learning-based method in the literature for estimating the body shape under clothing from a single 3D dressed body scan. To facilitate the proposed model, a novel dataset consisting of large-scale dressed body scans and corresponding ground-truth body shapes is proposed. Our second contribution is the first deep learning-based method in the literature to align non-overlapping partial point clouds. Using this method, an omnidirectional body can be obtained from only two non-overlapping body scans. The last contribution is to propose a novel deep learning-based method to reconstruct an animatable body shape from only two depth images and at the same time allow for large pose variations between the camera shots.

Bio: Pengpeng Hu is currently an Assistant Professor with the Centre for Computational Science and Mathematical Modelling, Coventry University, Coventry, U.K., and was a Senior Researcher with the Electronics and Informatics Department, Vrije Universiteit Brussel (VUB), Brussels, Belgium. In 2016, he was a Visiting Scholar with the School of Informatics, Edinburgh University, Edinburgh, U.K. In 2017, he was a Post-Doctoral Fellow with the Computer and Information Sciences Department, Northumbria University, Newcastle upon Tyne, U.K. Since 2018, he has been with VUB. His research interests include biometrics, geometric deep learning, 3-D human body reconstruction, point cloud processing, and measurement. Pengpeng is an Early Career Advisory Board Member of Measurement, Measurement: Sensors, the Journal of Textile Research, and Journal of Silk. He is also an Editorial Member of the Journal of Modern Industry and Manufacturing and a Topical Advisory Panel Member of Sensors (MDPI) and Designs (MDPI). He was the Guest Editor of the Sensors (MDPI), the Technical Support Chair of BMVC 2018, and a member of the Program Committee in SKIMA 2017, SKIMA 2018, and SKIMA 2019. He was the Outstanding Paper Winner of the Emerald Literati Award in 2019.