Kfir Aberman (Google) - Learning the Structure of Motion

Learning the Structure of Motion

Human motion is a fundamental attribute, underlying human actions, gestures, and behavior. It is inherently a 4D entity, commonly represented as a low-level encoding: a temporal sequence of poses, specified as a set of joint positions and/or angles. However, motion is a high-level, abstract, attribute that is concealed under this representation. For example, the same motion performed by two individuals with different skeletons might have significantly different low-level encoding. 

In this talk, we will present two learning frameworks that aim to learn the essence of motion and enables retargeting the motion of one performer to another. The first deals with the retargeting of video-captured motion between different human performers, without the need to explicitly reconstruct 3D poses and/or camera parameters, and the second is aiming at 3D motion retargeting between skeletons, which may have different structures and number of joints, yet derived from the same topology. 

The shared key idea in both frameworks is the disentanglement of a motion signal into fundamental, abstract, components that enables separating the dynamic aspects of motion from the static ones, using unique deep neural network structures. Our framework demonstrates state-of-the-art performances in both of the retargeting tasks, and can be further exploited for other motion analysis and synthesis tasks such as motion retrieval and motion style transfer.

Host: Rana Hanocka