Guitarist Fingertip Tracking by Integrating a Bayesian Classifier into Particle Filters

Abstract EN

We propose a vision-based method for tracking guitar fingerings made by guitar players.

We present it as a new framework for tracking colored finger markers by integrating a Bayesian classifier into particle filters.

This adds the useful abilities of automatic track initialization and recovery from tracking failures in a dynamic background.

Furthermore, by using the online adaptation of color probabilities, this method is able to cope with illumination changes.

Augmented Reality Tag (ARTag) is then utilized to calculate the projection matrix as an online process which allows the guitar to be moved while being played.

Representative experimental results are also included.

The method presented can be used to develop the application of human-computer interaction (HCI) to guitar playing by recognizing the chord being played by a guitarist in virtual spaces.

The aforementioned application would assist guitar learners by allowing them to automatically identify if they are using the correct chords required by the musical piece.