February 26, 2018 –
Title: Visual Recognition and Prediction of Human Actions in Videos
Speaker: Dr. Yu Kong
Abstract: Most of us are curious about what is happening now and what will happen next. As humans, we actually have the capability of describing, recognizing and predicting human actions because we have already visually observed certain correlations between visual elements in videos and we have such knowledge. It would be interesting to know, “can we program a machine with such capability, and build an intelligent framework that can accurately make decisions based upon video data, and even to an extreme extent, based upon temporally incomplete data?” Such a framework would be of great importance because not only would it demonstrate full comprehension on temporal rhythms of human actions, it would also serve an efficient and proactive alerting of forthcoming events before they are fully executed. However, it is challenging mainly due to the lack of rich knowledge about spatial and temporal correlations between visual elements in action videos, which are crucial to understanding complex actions. In this talk, I will describe how to learn such spatiotemporal correlations from action videos and use them for robust and prompt action understanding tasks, including human interaction recognition and action prediction.
Bio: Dr. Yu Kong is now a postdoctoral research associate in the Electrical and Computer Engineering, Northeastern University, Boston, MA. He received B.Eng. degree in automation from Anhui University in 2006, and PhD degree in computer science from Beijing Institute of Technology, China, in 2012. He was a visiting student at the National Laboratory of Pattern Recognition (NLPR), Chinese Academy of Science from 2007 to 2009, and a visiting research scholar at the Department of Computer Science and Engineering, State University of New York, Buffalo in 2012. He won the First Place in MSR Image Recognition Challenge, 2016. He also serves as reviewers and PC members for prestige journals and conferences, including T-PAMI, T-IP, T-NNLS, T-CSVT, AAAI, and IJCAI, etc. Dr. Kong's research interests are computer vision, social media analytics, and machine learning.
 Yu Kong, Yunde Jia, and Yun Fu. Learning Human Interaction by Interactive Phrases. ECCV 2012
 Yu Kong, Yunde Jia, and Yun Fu. Interactive Phrases: Semantic Descriptions for Human Interaction Recognition. TPAMI 2014
 Yu Kong, Dmitry Kit, and Yun Fu. A Discriminative Model with Multiple Temporal Scales for Action Prediction. ECCV 2014
 Yu Kong and Yun Fu. Max-Margin Action Prediction Machine. TPAMI 2016
 Yu Kong, Zhiqiang Tao, and Yun Fu. Deep Sequential Context Networks for Action Prediction. CVPR 2017