Eye-tracking based relevance feedback for iterative face image retrieval

Mengli Sun; Jiajun Wang; Zheru Chi

doi:10.1117/12.2524340

6 May 2019 Eye-tracking based relevance feedback for iterative face image retrieval

Mengli Sun, Jiajun Wang, Zheru Chi

Proceedings Volume 11069, Tenth International Conference on Graphics and Image Processing (ICGIP 2018); 110691Z (2019) https://doi.org/10.1117/12.2524340
Event: Tenth International Conference on Graphic and Image Processing (ICGIP 2018), 2018, Chengdu, China

Abstract

The performance of computer-vision based face image retrieval system declines significantly when large illumination, pose, and facial expression variations are presented. To tackle such a problem, we propose a closed loop face image retrieval system with implicit eye-tracking based feedback. It combines the state-of-the-art computer vision method Face++ with the powerful cognitive ability of human. In this system, the Face++ provides initial retrieving results corresponding to a target sample face image whose top ranked 36 images are then displayed on the screen for collecting eye-tracking data of the users. Upon mining the user’s cognition results from the eye-tracking data with a deep neural network and feeding them back to the system, the system begins its new round retrieving process. Experimental results from 10 volunteers in a face database containing 1,500 images of 50 celebrities show that the performance of our system becomes better and better over iterations and finally our system achieve an average precision of higher than 0.918 and an average recall rate of higher than 0.897 upon convergence.

Citation Download Citation

Mengli Sun, Jiajun Wang, and Zheru Chi "Eye-tracking based relevance feedback for iterative face image retrieval", Proc. SPIE 11069, Tenth International Conference on Graphics and Image Processing (ICGIP 2018), 110691Z (6 May 2019); https://doi.org/10.1117/12.2524340

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available