about
Does Object Recognition Work for Everyone? (arxiv.org)
4 points by sohkamyung on Jun 7, 2019 | hide | past | pdf | 1 comment on HN

In plain words: Tested public object-recognition systems on photos of household items gathered from many countries. They were less accurate on items common in low-income countries, mainly because those items look different or show up in unusual places.

Abstract

The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset. This dataset contains household items and was designed to have a more representative geographical coverage than commonly used image datasets in object recognition. We find that the systems perform relatively poorly on household items that commonly occur in countries with a low household income. Qualitative analyses suggest the drop in performance is primarily due to appearance differences within an object class (e.g., dish soap) and due to items appearing in a different context (e.g., toothbrushes appearing outside of bathrooms). The results of our study suggest that further work is needed to make object-recognition systems work equally well for people across different countries and income levels.

Terrance DeVries, Ishan Misra, Changhan Wang, Laurens van der Maaten
arXiv:1906.02659 · cs.CV, cs.LG · submitted Jun 6, 2019 · updated Jun 18, 2019
abstract · pdf · html

add comment on HN

I believe this kind of works is very important. Ore ML systems are biased in so many way that discovering them can provide many insights about how they work and how improve them.