Understanding Infinite-Width Limit of Neural Networks

페이지 정보

작성자 hjm0203 댓글 0건 조회 136회 작성일 20-12-29 17:45


2021년 1월 14일(목) 10:00부터 4단계 BK21사업을 수행하고 있는 서울대학교 인공지능 혁신인재양성 연구단에서 초청 세미나가 진행되오니, 
많은 관심과 참여 부탁드립니다.

*세미나 종류 : 일반 세미나 

                                               <서울대 인공지능 혁신인재양성 연구단(BK4) 세미나 개최> 

■ 주제 : Understanding Infinite-Width Limit of Neural Networks

■ 연사 : 이재훈 박사 (Google Brain, Research Scientist)

■ 일시 : 2021  1  14 () 10:00~

■ 장소 : 코로나-19감염대비를 위해 화상으로 진행(https://snu-ac-kr.zoom.us/j/2262026611)


■ 초록  

Infinite-width limit of neural networks has been key to recent breakthroughs in our understanding of deep learning. This limit is unique in giving an exact theoretical description of large scale neural networks.
Because of this, we believe it will continue to play a transformative role in deep learning theory. In this talk, we will review recent progress in the study of the infinite-width limit of neural networks focused around Neural Network Gaussian Process (NNGP) and Neural Tangent Kernel (NTK). This correspondence allows us to understand wide neural networks as kernel methods and provides 1) exact Bayesian inference without ever initializing or training a network and 2) closed form solution of network function under gradient descent training. We will discuss recent advances, applications and remaining challenges of the infinite-width limit of neural networks. Lastly, we will have a brief overview of python open-source software library, Neural Tangents, for those who want to get their hands dirty exploring this research space. 

■ 약력

Jaehoon Lee is a Research Scientist at Google Brain team. His main research interests is fundamental understanding of deep neural networks; actively working on the infinite-width limit of neural networks and their correspondence to the kernel methods. In 2017, he joined Google and started a research career in machine learning as part of the Google Brain Residency program. Before that he was a postdoctoral fellow at University of British Columbia from 2015-2017 working on theoretical high energy physics. Jaehoon obtained his PhD in physics at the Center for Theoretical Physics, Massachusetts Institute of Technology (MIT) in 2015. He served as co-organizer of ICML 2019 workshop on Theoretical Physics for Deep Learning and Aspen Center for Physics 2019 winter conference on Theoretical Physics for Machine Learning.

초청자 : 지능정보융합학과 곽노준 교수 (연락처 : 031-888-9166)