by Dr. Fanghui Liu, University of Warwick)

Abstract: In this talk, I will discuss the relationship between kernel methods and (two-layer) neural networks for generalization, which aims to theoretically understand the separation from the perspective of function spaces. First, I will start with random features models (a typical two-layer neural network, also a kernel method) from under-parameterized regime to over-parameterized regime, which recovers the double descent and demonstrates the benefits behind over-parameterization. Second, I will compare kernel methods and neural networks via random features from reproducing kernel Hilbert space (RKHS) to Barron space, which leaves an open question: what is the suitable function space of neural networks?