Using Ultrasonic Hand Tracking to Augment Motion Analysis Based Recognition of Manipulative Gestures

Summary:

Ogris et al. use ultrasonics to track hand motion in a 3D environment. The ultrasonics, when combined with other data from motion sensors, can greatly improve recognition rates.

Ultrasonics emit a sound beacon, which is then reflected back to sensors. Because ultrasonics use sound waves, the beacon is susceptible to reflection, occlusion, and temporal issues. Reflection is where the wave reflects off a surface at an odd angle, occlusions are blocked signals, and the temporal issues involve the time it takes for the sound to bounce back and forth. These issues limit ultrasonics to controlled, indoor scenarios. Placing the sensors on hands or other moving appendages is also a problem with ultrasonics, since all of the above problems can occur with fast moving parts.

To test the ultrasonics, the authors used a bicycle repair setup where the performer had 3 ultrasonic sensors and 9 gyroscopes on their arms, legs, and body. The performer then made various bicycle repair gestures, such as screwing/unscrewing, pumping, and wheel spinning.

Using a k-nearest-neighbor (kNN) approach to classification, the accuracy of the system jumps when using ultrasonics as opposed to just using motion sensors.


Discussion:

The use of ultrasonics probably does help the system. I am still not convinced that the ultrasonics themselves are useful, though. More sensors can almost always improve accuracy of a system, but since they "overlapped" the gyroscopes with ultrasonics at points the accuracy jump must be from the sensor type and not quantity.

My main issue is that ultrasonics seem to have an incredibly low Hz rate, or at least the sensors the authors were using were quite poor. Furthermore, noise problems (via bouncing signals, background sonics, or fast-moving sensors) seem to heavily detract from the ultrasonic's usage.