Drawing and the Non-verbal Mind

Lange-Kuttner, C. and Vitner, A. "Drawing and the Non-Verbal Mind: A Life-Span Perspective." Cambridge University Press, September 15, 2008.

Summary:

The editors discussed hundreds of experiments dealing with drawings, most focused on children.

Some interesting points of note are:

1. Young children (3-4 yrs) often cannot recognize their own drawings after some time has passed since the original. (p. 55)
   
2. Children often have a "constant depiction strategy", such as drawing everything as a sunburst or as a scribbled dot. The depiction looks closer to the actual object with age. (p. 64)
   
3. A drawing can be affected by the question and how the child interprets the objects, such as individual objects or in a group. Grouping of objects can happen more often if the objects are similar: "two circles" vs. "circle and triangle" (pp. 165-173)
   
4. People suffering from diseases such as semantic dementia often forget the distinguishing characteristic of an object they should be drawing after a short period of time. (Rhino -> generic animal, p. 286)

Discussion:

The findings presented are too numerous to list, so I simply mentioned the ones I found most interesting. Actual child or mental development would be difficult to measure using sketch recognition techniques (the drawings are simply too abstract). If I ever work with children, items 2 and 3 will probably be helpful with either distinguishing between children or simply with the phrasing of the questions to the children.

SKIT: A Computer-Assisted Sketch Instruction Tool

Greg Coombe and Brian Salomon
Department of Computer Science, University of North Carolina

This paper discusses a system, SKIT, that assists users in sketching line drawings by using some artist techniques. The system breaks a full outline sketch into subdrawings that the user can draw at various sizes. This allows the user to see the model they are drawing as a set of smaller, more geometric objects. The subdrawings are then merged back together in the end.

The small user study was more qualitative and showed users improved when using SKIT.


Discussion:

The paper presents a technique or two that might be helpful in starting a user-training program for sketching.

The 3D Tractus: A Three-Dimensional Drawing Board

Summary:

Lapides et al. designed and built a Tablet PC stand that can move vertically, allowing for a 3D drawing platform that switches the screen's view as the table is moved. The authors state that using the 3D Tractus will allow for a "direct mapping between physical and virtual spaces."

The frame of the 3D Tractus consists of aluminum bars and a table top, along with a counterweight that will balance the weight of the tablet and allow for the table top to slide up and down easier. The counterweight has to be tuned for each tablet's weight. A height sensor is built into the frame.

The drawing software for the system takes into account the height of the table when displaying a viewing angle to the user. The system uses line width as a depth cue, with farther lines thin and closer lines thick. An orthographic (cube) projection is used to demonstrate 3D depth, as well. Also, nothing of the sketch is displayed above the current tablet surface.


Discussion:

Although the idea of having a tactile way to sketch in 3D sounds appealing, the system could be implemented much better without a tactile, movable desk. Instead, having a z-axis button/wheel/control in the software will alleviate the issues with custom counterweights, a height constraint, awkward hand/arm positioning, and lack of mobility.

Also, the system is rather constrained with any large sketches since the user can move in the tablet's plane in infinite direction, but the vertical plane is limited to something like 40 centimeters.

HoloSketch: A Virtual Reality Sketching / Animation Tool

Deering, Michael F. HoloSketch: A Virtual Reality Sketching/Animation Tool. (1995) ACM Transactions on Computer-Human Interaction.

Summary:

Deering's 3D VR system, HoloSketch, aimed to allow the creation of three-dimensional objects in a virtual reality environment. Users donned VR goggles with a supercool 960x680 20'' CRT monitor and interacted with the virtual world via a six-axis wand. The head-tracking goggles allow the user to look around images hovering in front of them.

HoloSketch prides itself in displaying stable images that do not "float" or "swim" as the user moves their head. They accomplish this by having a highly accurate absolute orientation tracker in the goggles. The use of a flat-screen CRT also helps, as well as program corrections for interocular distances.

A good chunk of the paper focused on user interactions, such as menu navigation. Deering's system uses a 3D pie (radial) menu that can be activated by holding down the right click on the wand. The user can then navigate the menu while holding the button and "poke" the menu to activate submenus and items. To create and draw objects, the user first selects a primitive from the menu and then places the primitive by hitting a button on the wand. The user can then rotate, size, and position the object using a combination of wand-waving and keyboard buttons.

Users can also create animations with the system. Some animations require still shots of slightly altered objects that can be grouped temporally (like a VR flipbook). Other animations can be added to objects or groups, such as a rotor property or blinking colors.

An artist tested the system for a month and provided feedback. Overall the artist found the tool easy to work with after a few days, although some of the features available in other applications were missing from HoloSketch. One issue that Deering noticed was the lack of a user's head movement when trying to view the object; users are too used to stable heads that examining an object from different angles was not intuitive.


Discussion:

HoloSketch seems like an interesting application and provides a variety of ideas, some which I believe are beneficial, while others are not. The "poking" of menus seems intuitive, and if the system has a high absolute accuracy this should work well. Yet, Deering mentioned how user's arms can get tired and are unstable, and supporting an arm and wrist is out of the question when you try to make an environment natural. Instead HoloSketch had some button that reduced the jitter somehow when activated, which seems like a hack that allows for a quick fix of a potentially serious issue with using the system.

I also understand why people would not want to constantly move their head around the display. If the display was on a round table this would be a non-issue, but constantly moving around in a chair and leaning different directions is a strain to a user. Furthermore, the 20" CRT is not that large of a screen the the user would be able to "see" all around the object; I would have liked to know the actual viewing angle.

Overall, though, I liked the system and the paper itself was well-written and gave a good overview of the features.

Speech and Sketching: An Empirical Study of Multimodal Interaction

Summary:

In this paper, Adler and Davis explore multimodal speech and sketch interfaces through a user study. Their goal is to allow the computer to provide feedback to the user as the user talks and draws, and the computer will influence the design during this process by asking questions and clarifying information. Having the computer understand everything about the design is not the goal; instead, the computer should know enough to ask motivating questions when necessary in order to engage the user. The system also does not want to constrain the user's drawing or speech style.

The user study conducted involved 18 users in a Wizard-of-Oz study. The users were asked to design a floor plan, full adder, AC/DC transformer, and a digital circuit. Sketches was done on Tablet PCs in software that allowed for drawing and highlighting in 5 different colors. During the study, the experimenter sat at a table across from the user. The study was filmed and the audio, visual, and sketching components of the study were synchronized.

The study showed some interesting results concerning color, questions, and speech timing. Users tended to rely on multiple colors to indicate portions of the sketch. The color linked parts of the sketch together, referred back to previous parts, and reflected the real-world colors of objects. When speaking, users typically had phrase and word repetition when they were thinking aloud. This could allow the computer to discern key words from the user-computer dialogue. Responses from computer questions also caused the user to repeat the questions, and simple questions could prompt more information than what was asked. Some users even redesigned their drawings after simple questions were asked, such as inquiring if two objects were similar. Speech and sketching started simultaneously in the study. Yet, certain parts of the speech, such as an entire phrase, tended to start before the sketch, and certain key words said alone tended to be heard after a sketch was started.

Discussion:

The two best components of Adler's study show how computers can assist humans during design steps by relying on the human design and thought process, instead of having an actual understanding. In lieu of training the computer to understand all of the components of a design, basic understanding of object similarity and grouping should be enough to produce a motivating dialogue. Also, the fact that the user constantly repeats words provides the computer with an indication of important information without the need of a large vocabulary.

I wish the study also went into more interface issues, such as when the computer should ask a question (e.g. during sketching, during a pause, etc.). Also, it would have been beneficial to see the average pause time of a user and if the user was speaking or mumbling during the pause by going "hmm" or something similar. Do the pauses for sketch indicate that the user is speaking, and do pauses for speaking indicate the user is sketching? Do the pauses for both modes line up?