Conversation activity recognition using interaction video sequences in pedestrian groups

We generate video sequences in a virtual space that represent pedestrians' physical interactions using time-series data on their postures and walking positions. These interaction videos serve as features for recognizing conversation activity within a group.

When friends talk while walking, their body movements and positions relative to one another can change with the conversation. We study how to recognize how active a conversation is from color video of a group walking outdoors. The goal goes beyond deciding whether people are simply walking or talking together.

First, we extract how each person's posture and walking position change over time. We use this information to create a video in a virtual space that represents the group's physical interactions. This interaction video provides clues for a computer to classify conversation activity. It also lets a person view the movements used as those clues.

In experiments, we compared the method with approaches that use the appearance of people's bodies or directly use the sequences of changes in their postures and positions. The interaction videos enabled more accurate recognition of conversation activity than these comparison methods.

• Tsubasa Kondo, Michiko Inoue, Shunsuke Yoneda, and Masashi Nishiyama
  Reducing Computational Cost in Pedestrian Conversation Activity Recognition through Skeleton Spatiotemporal Graphs, Proceedings of IEEE 14th Global Conference on Consumer Electronics (GCCE), OS-AIP, pp. 815 - 818, September 2025. [Outstanding Prize Excellent Student Paper Awards] [Slides]
• Wataru Ganaha, Takumi Ozaki, Michiko Inoue, and Masashi Nishiyama,
  Conversation activity recognition using interaction video sequences in pedestrian groups, Proceedings of 27th International Conference on Pattern Recognition (ICPR), Part XIV, pp. 359 - 374, December 2024. [Poster]
• 近藤 翼, 井上 路子, 米田 駿介, 西山 正志,
  歩行者の会話の活発さ認識の軽量化における骨格時空間グラフの有効性検証,
  画像センシングシンポジウム (SSII), IS2-36, pp. 1 - 8, May 2025. [Poster] [Short movie]
• 尾崎 匠，井上 路子，西山 正志,
  会話の活発さ認識における歩行者間のインタラクションの重要度の可視化,
  ビジョン技術の実利用ワークショップ (ViEW), IS4-12, pp. 525 - 531, December 2023. [Poster]
• 尾崎 匠，我那覇 航，井上 路子，西山 正志,
  歩行者間の会話の盛衰認識における複数グループ対応の検討,
  画像の認識・理解シンポジウム(MIRU), IS1-71, July 2023. [Poster]