Assisting the visually impaired: multitarget warning through millimeter wave radar and RGB-depth sensors

Ningbo Long; Kaiwei Wang; Ruiqi Cheng; Kailun Yang; Weijian Hu; Jian Bai

doi:10.1117/1.JEI.28.1.013028

7 February 2019 Assisting the visually impaired: multitarget warning through millimeter wave radar and RGB-depth sensors

Ningbo Long, Kaiwei Wang, Ruiqi Cheng, Kailun Yang, Weijian Hu, Jian Bai

Author Affiliations +

Journal of Electronic Imaging, Vol. 28, Issue 1, 013028 (February 2019). https://doi.org/10.1117/1.JEI.28.1.013028

Abstract

It is very difficult for visually impaired people (VIP) to perceive and avoid obstacles at a distance. To address this problem, we propose a sensor fusion system, which combines the RGB-depth (RGB-D) sensor and millimeter wave (MMW) radar sensor, to perceive the surrounding obstacles. The position and velocity information of the multiple targets are detected by the MMW radar based on the principle of frequency modulated continuous wave. The depth and position information of the obstacles are verified by the RGB-D sensor based on the MeanShift algorithm. The data fusion based on the joint probabilistic data association algorithm and Kalman filter enable the navigation assistance system to obtain more accurate state estimates compared with using only one sensor. The nonsemantic stereophonic interface is utilized to transfer the obstacle detection results to the VIP. The experiment results show that multiple objects with different ranges and angles are detected by the radar and the RGB-D sensor. The effective detection range is expanded up to 80 m compared to using only the RGB-D sensor. Moreover, the measurement results are stable under diverse illumination conditions. As a wearable system, the sensor fusion system has the characteristics of versatility, portability, and cost-effectiveness.

Citation Download Citation

Ningbo Long, Kaiwei Wang, Ruiqi Cheng, Kailun Yang, Weijian Hu, and Jian Bai "Assisting the visually impaired: multitarget warning through millimeter wave radar and RGB-depth sensors," Journal of Electronic Imaging 28(1), 013028 (7 February 2019). https://doi.org/10.1117/1.JEI.28.1.013028

Received: 8 October 2018; Accepted: 17 January 2019; Published: 7 February 2019

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available