RESEARCH

3D heatmaps of where people look in sports videos

Key Points

When a climber reaches for the next hold, do people follow the hand or look at the face? We map viewers’ gaze to parts of the athlete’s body instead of points on the screen. The colors show where people looked at a particular moment and which body parts drew their eyes over the whole clip.

The top row shows climbing and the bottom row tennis. The columns show a video frame, gaze on the athlete at that instant, and gaze over the entire clip on a model in a fixed pose. Redder areas indicate more concentrated gaze.

Abstract

If you follow a tennis player’s right hand, your eyes move across the screen even though you are looking at the same hand. We estimate the athlete’s pose in each video frame, then map viewers’ gaze to a 3D body model in that pose. If someone looks just beyond the model’s surface, for example at the edge of the athlete’s hair or clothing, we also count that gaze toward nearby body parts.

In our experiment, participants watched clips of climbing, tennis and a martial arts kick and judged whether the athletes’ movements felt dynamic. Combining the gaze measurements on a model in a fixed pose let us compare the three clips. In the climbing clip, viewers tended to look at the head, arms and right side of the body. In the tennis and kicking clips, they looked mainly at the head.

Publications

otama1.png