BYU

Abstract by Ben Clough

Personal Infomation


Presenter's Name

Ben Clough

Co-Presenters

Matthew Gwilliam

Degree Level

Undergraduate

Co-Authors

Matthew Gwilliam

Abstract Infomation


Department

Computer Science

Faculty Advisor

Ryan Farrell

Title

Eye Spy: Leveraging Human Eye Tracking in Computer Image Identification

Abstract

Computers are becoming increasingly good at identifying objects in images. New methods in deep learning have allowed for greater accuracy in recognizing objects across varying contexts and use cases. Fine-grained visual categorization (FGVC) takes these techniques a step further. Rather than identifying a bird or a car, FGVC works to identify sub categories, such as the genus and species or the make and model. Humans experts are very good at fine-grained identification within a particular domain. The focus of our research is to learn what human experts are doing differently in the process of analyzing an image that allows them to recognize objects in their domain of expertise quickly and accurately. Using eye tracking built into a VR (virtual reality) headset, data is collected on where people look as they seek to recognize the genus and species of birds. Leveraging this data, we are working to endow neural networks with the potential to recognize these fine-grained categories with human-like expertise.