Synthesizing Realistic Image-based Avatars by Body Sway Analysis

We analyze body sway in videos of people standing and join points where motion is close to zero. Synthesizing sway videos of arbitrary duration gives an avatar human-like movement during its waiting periods.

People sway slightly even when they are standing and waiting. For an on-screen guide, natural movement matters not only during speech or actions but also while it waits. We proposed a way to add these small, human-like movements to an image-based avatar made from video of a real person.

We record a standing person with one camera and measure how different body parts move over time. We then find points in the video where movement is close to zero and randomly connect such points. This allows us to create a new video in which small body movements continue for as long as needed.

Participants assessed the resulting avatars and compared them with avatars produced using baseline methods. Avatars with body sway generated by our method appeared more alive. The research addresses how to portray waiting naturally during the periods before and after interaction, and between an avatar's actions.

• Masashi Nishiyama, Tsubasa Miyauchi, Hiroki Yoshimura, and Yoshio Iwai
  Synthesizing Realistic Image-based Avatars by Body Sway Analysis,
  Proceedings of 4th International Conference on Human-Agent Interaction (HAI), pp. 155 - 162, October 2016.

• 宮内 翼, 吉村 宏紀, 西山 正志, 岩井 儀雄,
  身体動揺の計測による待ち状態の実写アバタ生成,
  電子情報通信学会論文誌 D, Vol.J100-D, No.3, pp.365-375, March 2017.

• Synthesized videos