A Method for Recognizing a Sequence of Sign Language Words Represented in a Japanese Sign Language Sentence

Summary:

Sagawa and Takeuchi created a Japanese Sign Language recognition system that uses "rule-based matching" and segments gestures based on hand velocity and direction changes.

There are thresholds of direction vector changes that account for the segmentation. There are also issues to determine which hand (or both) are being used for gestures, and these are determined by the direction and velocity change thresholds.

The system achieved 86.6% accuracy for signed words, and 58% accuracy for signed sentences.


Discussion:

There's not much to discuss with this paper. The "nugget" of research is with the use of direction and velocity changes to segment the gestures. I became more interested in this paper since I learned it was published a year before Sezgin's, but not by much.

A similarity measure for motion stream segmentation and recognition

Chuanjun, L. and B. Prabhakaran (2005). A similarity measure for motion stream segmentation and recognition. Proceedings of the 6th international workshop on Multimedia data mining: mining integrated media and complex data. Chicago, Illinois, ACM.

Summary:

Li and Prabhakaran propose a way to "segment" streams of motion data by using singular value decomposition (SVD). SVD is similar to principal component analysis (PCA), and the technique finds the underlying geometric structure of a matrix (i.e., its eigenvectors and values). By using the singular values of matrices storing motion data, the matrices can be compared in similarity by measuring the angular differences (dot products) of these vectors.

The authors store motion data in a matrix consisting of columns of features and rows of timesteps. The first 6 eigenvectors are used when comparing matrix similarity; this value was empirically determined. The segmentation part of the paper involves separating this stream of data after every l timesteps, and then comparing the similarity of the segmented matrix to stored eigenvectors and values for a known motion.

To test their system, the authors merged individual motions together into a "stream" of data and inserted noise inbetween motions. The authors noted that the number of eigenvectors needed to distinguish between matrices (originally, k = 6) varied depending on the data collection method. Their paper reported recognition rates in the mid 90s, but these results depend on how similar motions are to one another.


Discussion:

Although the paper has little to do with segmentation, the actual algorithm for comparing motion data seems interesting and appears to achieve relatively accurate results. I would like to know the actual motions that users performed, since I have no idea what motions are required in Taiqi and Indian dances. They also did not mention the number of people involved in the data capturing, and I assume this number to be close to 1 since they needed a user to wear a motion suit.

Hand Tension as a Gesture Segmentation Cue

Philip A. Harling and Alistair D. N. Edwards. Hand tension as a gesture segmentation cue. Progress in Gestural Interaction: Proceedings of Gesture Workshop '96, pages 75--87, Springer, Berlin et al., 1997


Summary:

Harling and Edwards describe a way to segment hand gestures based on hand tension. The basic idea is that as a user dynamically moves between static postures a and b, their hand will reach a "relaxed", low-tension minimum position c that is less tense than either a or b.

Smaller details:

1. To find the tension for each finger, the authors use Hooke's Law and treat a finger as if it were a spring
2. The total hand tension is the sum of the finger tensions
3. They used a Mattel PowerGlove
   

Discussion:

The idea behind the paper was actually quite good for recognizing between static postures. I have a feeling that the hand tension will not work well for moving gestures since there would be small segmentations within the gesture.

I'm disappointed at their lack of results. I can forgive other papers that were user studies, but I cannot forgive a paper that does not report easily obtainable results when they spent 8 pages discussing a topic that I summarized in one sentence. Segmentation is rather simple to gather data for, and a published paper should at least attempt to find an accuracy number.

On a technical note, I'm curious as to how hand tension is affected by the type of glove worn. I have a feeling that my "hand relaxed" position is going to be different for a P5 glove than it will be for a CyberGlove or even a CyberGlove with a Flock of Birds attached. All the extra weight will most likely force my hand into resting upon the equipment for support.

A Dynamic Gesture Interface for Virtual Environments Based on Hidden Markov Models

Qing, C., A. El-Sawah, et al. (2005). A dynamic gesture interface for virtual environments based on hidden Markov models. Haptic Audio Visual Environments and their Applications, 2005. IEEE International Workshop on.


Summary:

The authors of this paper used the HMM & CyberGlove dynamic duo in conjunction with standard deviations.

Qing et al. claim that using the standard deviation of finger positions allows them to fix the "gesture spotting" (segmentation/fragmentation) issue with a continuous data stream. The glove data is sampled at 10Hz, and then the standard deviations of each sensor are calculated. The standard deviations also help transform a series of vectors (observations) into a single vector. They then take this vector and perform VQ on it to get a discrete value.

The three gestures they used to test their system controlled the rotation of a cube. The gestures included 1 finger bending, 2 fingers bending, and a twisting motion with your thumb.


Discussion:

Sigh, no results. I have no idea how the system actually solves the gesture spotting problem because they are just trading the "is this observation the start of a gesture?" problem into a "does this standard deviation vector look like it might be the start of a gesture?" problem.

Also, with only three gestures standard deviations might work for distinguishing between gestures. But continually moving one's hand indicates that the standard deviation for every finger will be fluctuating wildly.

I now know more about the bone structure of a hand.