Upbeat: Augmented Reality-Guided Dancing for Prosthetic Rehabilitation of Upper Limb Amputees

Abstract EN

Unsuccessful rehabilitation therapy is a widespread issue amongst modern day amputees.

Of the estimated 10 million amputees worldwide, 3 million of whom are upper limb amputees, a large majority are discontent and experience rejection with their current prosthesis during activities of daily living (ADL).

Here we introduce Upbeat, an augmented reality (AR) dance game designed to improve rehabilitation therapies in upper limb amputees.

In Upbeat, the patient is instructed to follow a virtual dance instructor, performing choreographed dance movements containing hand gestures involved in upper limb rehabilitation therapy.

The patient’s position is then tracked using a Microsoft Kinect sensor while the hand gestures are analyzed using EMG data collected from a Myo Armband.

Additionally, a gamified score is calculated based on how many gestures and movements were correctly performed.

Upon completion of the game, a diagnostic summary of the results is shown in the form of a graph summarizing the collected EMG data, as well as with a video displaying an augmented visualization of the patient’s upper arm muscle activity during gameplay.

By gamifying the rehabilitation process, Upbeat has the potential to improve therapy on upper limb amputees by enabling the start of rehabilitation immediately after trauma, providing personalized feedback which professionals can utilize to accurately assess patient’s progress, and increasing patient excitement, therefore increasing patient willingness to complete rehabilitation.

This paper is concerned with the description and evaluation of our prototypic implementation of Upbeat that will serve as the basis for conducting clinical studies to evaluate its impact on rehabilitation.