Unmanned ground vehicle with virtual reality vision

الملخص EN

This paper aims to describe the design and implementation of an Unmanned Ground Vehicle (UGV) and a smart phone virtual reality (VR) head mounted display (HMD) which enables visual situation awareness by giving the operator the feel of "head on rover" while sending the video feeds to separate operator computer for object detection and 3-D model creation of the UGV surrounding objects.

The main contribution of this paper is of three folds: (i) the novel design of the HMD; the paper proposes an alternative design to the 3-D interface designs recently used in tele-operated search and rescue (SAR) UGVs.

Unlike other designs that suggest to automatically move the whole UGV about two axes (pitch and yaw) with the movement of the head, this design suggests to let a separate unit of the UGV automatically move with the movement of the head and provide the user with VR.

(ii) the distributed feature; the design allows multiple users to connect to the UGV using a wireless link in a secure way to receive video feeds from three on-board cameras.

This feature facilitates cooperative team work in urban search and rescue (USAR) applications (a contemporary research issue in SAR UGV).

(iii) a novel feature of the design is the simultaneous video feeds which are sent to the operator station computer for object detection using the scale-invariant feature transform (SIFT) algorithm and 3-D model construction of the UGV’s surrounding objects from 2-D images of these objects.

The design was realized using a smart phone-based HMD, which captures head movements in real time using its inertial measurement unit (IMU) and transmits it to three motors mounted on a rover to provide the movement about three axes (pitch, yaw and roll).

The operator controls the motors via the HMD or a gamepad.

Three on-board cameras provide video feeds which are transmitted to the HMD and operator computer.

A software performs object detection and builds a 3-D model from the captured 2-D images.

The realistic design constraints were identified, then the hardware/software functions that meet the constraints were listed.

The UGV was implemented in a laboratory environment.

It was tested over soft and rough terrain.

Results showed that the UGV has higher visual- inspection capabilities compared to other existing SAR UGVs.

Furthermore, it was found that the maximum speed of 3.3 m/s, six-wheel differential-drive chassis and spiked air-filled rubber tires of the rover gave it high manoeuvrability in open rough terrain compared to other SAR UGVs found in literature.

The high visual inspection capabilities and relatively high speed of the UGV make it a good choice for planetary exploration and military reconnaissance.

The three-motors and stereoscopic camera can be easily mounted as a separate unit on a chassis that uses different locomotion mechanism (e.g.

leg type or tracked type) to extend the functionality of a SAR UGV.

The design can be used in building disparity maps and in constructing 3-D models, or in real time face recognition, real time object detection and autonomous driving based on disparity maps.