Abstract by Ben Clough
Eye Spy: Leveraging Human Eye Tracking in Computer Image Identification
Computers are becoming increasingly good at identifying objects in images. New methods in deep learning have allowed for greater accuracy in recognizing objects across varying contexts and use cases. Fine-grained visual categorization (FGVC) takes these techniques a step further. Rather than identifying a bird or a car, FGVC works to identify sub categories, such as the genus and species or the make and model. Humans experts are very good at fine-grained identification within a particular domain. The focus of our research is to learn what human experts are doing differently in the process of analyzing an image that allows them to recognize objects in their domain of expertise quickly and accurately. Using eye tracking built into a VR (virtual reality) headset, data is collected on where people look as they seek to recognize the genus and species of birds. Leveraging this data, we are working to endow neural networks with the potential to recognize these fine-grained categories with human-like expertise.