Key Points
We incorporate the gaze distributions of observers judging gender into a deep learning self-attention mechanism. This guides attention toward regions less affected by the background and improves recognition when training and test images have different backgrounds.
Abstract
A computer trained to classify gender from pedestrian images may rely on the background rather than the person. Its accuracy can then fall when the backgrounds in new images differ from those seen during training. The problem is that it has learned to depend too much on information that is not directly related to the person.
We measure where people look when judging gender in pedestrian images. We use the resulting gaze patterns to guide the part of a deep learning model that decides which image regions deserve more attention. This helps the model use information from regions that people look at, instead of being overly influenced by the training backgrounds.
We tested the method on publicly available datasets under conditions where training and test images had different backgrounds. The gaze-guided method achieved higher gender-classification accuracy than the existing attention-based methods used for comparison.
Publications
- Masashi Nishiyama, Michiko Inoue, Yoshio Iwai,
Gender recognition using a gaze-guided self-attention mechanism robust against background bias in training samples,
IEICE Transactions on Information and Systems, Vol. E105-D, No. 2, pp. 415 - 426, February 2022.
