Comparing the Recognition Accuracy of Humans and Deep Learning on a Simple Visual Inspection Task

We design a visual inspection task in which participants distinguish normal from anomalous symbols without specialist knowledge. We compare humans and deep learning in terms of accuracy and how they detect anomalies, and investigate the number of training samples needed.

How many practice examples does a computer need to find abnormalities in images more accurately than a person? We compared people and deep learning on a simple inspection task that required no specialist knowledge: distinguishing normal symbols from unusual ones.

Deep learning learns image patterns from many examples. We automatically generated large numbers of images containing normal and anomalous symbols, then varied the number used for training. We examined two tasks: finding where an unusual symbol was located, and separating out the precise image region occupied by that symbol. We also compared how people and the models found abnormalities.

The experiments required several thousand training examples for detecting locations and tens of thousands for separating symbol regions. Deep learning also had both strengths and weaknesses compared with people. These findings apply to the simple symbol-inspection tasks tested here. They illustrate the value of checking data requirements and differences in performance for the particular task being studied.

• Naoto Kato, Michiko Inoue, Masashi Nishiyama, Yoshio Iwai,
  Comparing the Recognition Accuracy of Humans and Deep Learning on a Simple Visual Inspection Task, Proceedings of 5th Asian Conference on Pattern Recognition (ACPR), 142, pp. 1 - 14, November 2019.

• 加藤 直人, 井上 路子, 西山 正志, 岩井 儀雄,
  目視検査を模擬した単純タスクにおけるラベル欠損の検出精度の比較 ～深層学習が人間の目視と同等となるための訓練サンプル数の検証～,
  精密工学会誌, vol. 86, no. 12, pp. 997 - 1005, December 2020.