3D Object Modeling Using Spatial and Pictographic Gestures

Summary:

Nishino et al. designed a 3D object modeling system that uses stereoscopic glasses, CyberGloves, and polhemus trackers.

The system allows the creation of superellipsoids that can have smooth or squarish parameters. These primitive shapes can be bent, stretched, twisted, and merged with other shapes. Hand postures control these actions, such as grasping and pointing. Virtual hands are displayed on a 200-inch arched screen, along with the object, in stereoscopic mode. The virtual hands allow the user to easily see where they can touch and modify the 3D model.

The authors tested the system by having users attempt two types of objects: symmetric and asymmetric. The symmetric object was a bottle, and the asymmetric object was a teapot.
Creation of the objects took up to 120 minutes. The size of the stored objects was much less than a competing program, Open Inventor.


Discussion:

For a paper in 1998, this was a pretty advanced system and seemed to offer some benefits over other systems. I would have liked to have seen feedback from the users, though, since I'm not sure how hard the system is to use.

Glove-TalkII--A Neural-Network Interface which Maps Gestures to Parallel Formant Speech Synthesizer Controls

Summary:

Fels and Hinton created Glove-TalkII, a system designed to synthesize voice using complicated glove and feet controls.

The artificial vocal track (AVT) is controlled using a CyberGlove, ContactGlove, polhemus sensor, and foot pedal. The ContactGlove controls 9 stop consonants, such as CH, T, and NG. The foot pedal controls the volume of the speech. Hand position corresponds to a vowel sound. Hand postures map to nonstop consonant phonemes.

The neural networks used include a vowel/consonant network to determine if the sensors are reading a vowel or consonant, and then separate vowel and consonant networks to distinguish between the phonemes.

A single user had to undergo 100 hours of training to be able to use the system.


Discussion:

Impractical. I'm shocked that they had someone train the system for 100 hours, and the fact that it takes a person that long to train the system should indicate that this is a poor way to synthesize voice. The person's final voice is even described as "intelligible and somewhat natural-sounding", which is not a good complement.

Requiring a person to walk around with a one-handed keyboard and type their words is a better solution. The keyboard wouldn't even have a foot pedal.

Articulated Hand Tracking by PCA-ICA Approach

Summary:

Kato et al. used Independent Component Analysis (ICA) to find basis vectors for hand motion features. The authors first use Principle Components Analysis to reduce the dimensionality of their system, and then they use ICA to find a set of vectors that are statistically independent from each other (i.e., basis vectors).

Data on 20 angles was collected with a glove. The authors then sandwiched all of the data for the 20 sensors together into one large vector; each sensor was sampled across 100 time points, and the data from all 20 sensors was merged into a 2000-dimension vector.

ICA is used to find the basis vectors for a hand such that a linear combination of these vectors will produce a desired hand movement. The basis vectors U are found through a weight matrix W and a sample of motion data X (where X is a matrix of hand motions). A neural learning algorithm (in this case, gradient descent) is used to calculate the weights. The resulting 5 basis vectors are the movement of each finger individually.

The authors then deviated from their abstract and discussed actually tracking a hand using particle filtering. A hand's current position can be estimated from its prior positions, so each basis vector can estimate where it believes the finger will be given its prior positions. The authors also segment a hand out of an image by doing some thresholding on an image and overlaying a hand model in the image to find the hand location.

There are no results.

Discussion:

There are no results.
The basis vectors seem obvious, but I'm glad that ICA found them.
There are no results.

Using Ultrasonic Hand Tracking to Augment Motion Analysis Based Recognition of Manipulative Gestures

Summary:

Ogris et al. use ultrasonics to track hand motion in a 3D environment. The ultrasonics, when combined with other data from motion sensors, can greatly improve recognition rates.

Ultrasonics emit a sound beacon, which is then reflected back to sensors. Because ultrasonics use sound waves, the beacon is susceptible to reflection, occlusion, and temporal issues. Reflection is where the wave reflects off a surface at an odd angle, occlusions are blocked signals, and the temporal issues involve the time it takes for the sound to bounce back and forth. These issues limit ultrasonics to controlled, indoor scenarios. Placing the sensors on hands or other moving appendages is also a problem with ultrasonics, since all of the above problems can occur with fast moving parts.

To test the ultrasonics, the authors used a bicycle repair setup where the performer had 3 ultrasonic sensors and 9 gyroscopes on their arms, legs, and body. The performer then made various bicycle repair gestures, such as screwing/unscrewing, pumping, and wheel spinning.

Using a k-nearest-neighbor (kNN) approach to classification, the accuracy of the system jumps when using ultrasonics as opposed to just using motion sensors.


Discussion:

The use of ultrasonics probably does help the system. I am still not convinced that the ultrasonics themselves are useful, though. More sensors can almost always improve accuracy of a system, but since they "overlapped" the gyroscopes with ultrasonics at points the accuracy jump must be from the sensor type and not quantity.

My main issue is that ultrasonics seem to have an incredibly low Hz rate, or at least the sensors the authors were using were quite poor. Furthermore, noise problems (via bouncing signals, background sonics, or fast-moving sensors) seem to heavily detract from the ultrasonic's usage.