“Towards Massive Scale Deep Learning”

Speaker: Sanjiv Kumar
Distinguished Scientist at Google Research, NY

Abstract: Recent successes of deep neural networks in a large number of domains have spurred a renewed interest in both theory and applications of these models. However training and inference in such models at massive scale still remains extremely challenging. In this talk, I will highlight a number of challenges related to both speed and quality for the problems with large output spaces containing billions of outputs that drive real-world relevance search and recommendation systems. I will describe advancements in low-level fast matrix-vector products via structured matrices, provably convergent adaptive optimization, and design of appropriate loss functions, making robust massive scale learning feasible.  

Speaker bio: Sanjiv Kumar is a Distinguished Scientist at Google Research, NY where he is currently leading research in theory and applications of large scale machine learning.  His research interests include massive scale deep learning, fast training and inference with large output spaces, distributed and privacy preserving learning, and data-dependent hashing.  His work on convergence properties of Adam received best paper award in ICLR 2018. He had been an adjunct faculty at Columbia University where he taught a new course on large-scale machine learning. He is currently serving as an Action Editor of JMLR. Sanjiv holds a PhD from the School of Computer Science, Carnegie Mellon University.

.