Key Points
We collect observers' gaze while they judge personal attributes and create a task-oriented gaze map for each attribute. Regions that attract more attention receive greater weights when extracting image features for personal attribute classification.
Abstract
Where do people look when judging someone's characteristics from a photograph? We investigated the idea that regions attracting human attention may also contain useful information for a computer. The aim is to use observers' gaze to improve the classification of personal attributes, or characteristics of the person shown.
Several observers judged attributes in pedestrian images while we recorded their gaze positions. For each attribute, we combined these positions into a task-oriented gaze map: a map showing where people look when making that particular judgment. When extracting information from an image, our method gives greater weight to regions that attract more attention.
The experiments showed that observers mainly looked at different body parts depending on the attribute they were judging. Combining the gaze-based method with a deep learning network that learns image features, or with a method that learns image similarity, improved attribute classification. Statistical analysis supported these improvements, showing how human viewing behavior can help computer vision.
Publications
- Michiko Inoue, Masashi Nishiyama, and Yoshio Iwai,
Age Group Identification Using Gaze-Guided Feature Extraction, Proceedings of IEEE 12th Global Conference on Consumer Electronics (GCCE), POS, pp. 711 - 714, October 2023. [Poster] - Masashi Nishiyama, Riku Matsumoto, Hiroki Yoshimura, Yoshio Iwai,
Extracting Discriminative Features using Task-oriented Gaze Maps Measured from Observers for Personal Attribute Classification,
Pattern Recognition Letters (PRL), vol. 112, pp.241 - 248, September 2018. - Riku Matsumoto, Hiroki Yoshimura, Masashi Nishiyama, Yoshio Iwai,
Feature Extraction using Gaze of Participants for Classifying Gender of Pedestrians in Images, Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 3545 - 3549, September 2017.
