|
|
|
Design of Gaze–Based Interaction as Part of Multimodal Human–Computer Interaction
Katharina Seifert, Dr.-Ing. Dipl.-Psych.
Email: seifert@zmms.tu-berlin.de
WWW: www.mms.tu-berlin.de/seiten/PERSONEN/seifert/kseifert.html
|
| |
|
Overview
|
|
| |
|
What does multimodal human–computer interaction mean?
|
|
mUltimo 3D: an experimental multimodal system developed at the Heinrich–Hertz–Institute Berlin
|
|
Goals of the design of multimodal systems
|
|
Design of multimodal human-computer interaction
|
|
Requirements for gaze-based interaction
- Ease of use should be reflected in:
- high acceptance
- low levels of strain
- high performance level
- Consideration of the context/situation of use in the interpretation of user’s gazes:
- as the area of visual attention
- as the location of currently executed actions
- Physiological parameters of the modality:
- Gaze is controlled bottom up or top down
- Automatic programming of saccades
- Saccades start several times a second
|
|
A short view on the Baltic Sea...
Do you remember how you explored the picture? In this way?
Or in that way?
Or absolutely different?
|
|
Different concepts for the design of gaze-based interaction
- Transparency of the system status
- Feedback is an important feature to keep users in the loop
(Wandmacher 93, Shneiderman 92, Norman 88, Nielsen 93, Mayhew 92, Oppermann et al. 92, Preece et al. 94, Smith & Mosier 86)
- Natural Interaction
- Consider the specific features of the modality as an information channel
(Oviatt 99)
|
|
Transparency of the system status
- Well-known style guide hints:
- Feedback of continuous input of changing positions by a movable cursor
- Feedback of the discrete selection of an interface object by highlighting the selected object
|
|
Natural interaction
- Natural design of gaze-based interaction:
- No specific graphical or acoustic feedback of the recognized gaze position (like our experience in the natural environment)
|
|
What is the right way to design gaze-based interaction?
- The interpreted gaze position is continuously feed back by a “gaze cursor” on the display
- A gaze sensitive interface object is highlighted when the recognized gaze position matches the object
- The interpreted gaze position is not feed back
|
|
Empirical investigation of the design alternatives
Description of the experimental task
- Dual task paradigm
- Main task:
- Changing letters
[T,A,C,U,F,E,H,P,S,L]
every 1.7 seconds
- Acoustic presentation of a target letter
“A like Apple”
- Users were asked to stop the target letter on all three positions
- Letters were stopped by looking at them and concurrently pressing a button
- Secondary task:
- A yellow dot was displayed irregularly
- The dot appeared in the peripheral visual field
- Stayed on the display for 2 seconds
- Users were asked to acknowledge it by pressing a button
|
|
Experimental setting
- Users were seated approx. 80 cm in front of an 18” display
- Display resolution 1024 x 1280 pixel
- Head tracker and gaze tracker were calibrated and worked contact free
- Sequence of the experimental conditions was systematically varied
- 18 paid users participated
|
|
Dependent variables
- Performance in the main task
- Reaction times
- False alarms
- Misses
- Performance in the secondary task
- Reaction times
- False alarms
- Misses
- Duration of each trial
- Perceived mental workload
- Subjective preference for one of the feedback alternatives
|
|
Hypotheses part 1
If the transparency of the system status is most important for the users:
- Highlight condition
- highest performance
- lowest workload
- highest ranked preference
- Cursor condition
- average performance
- moderate workload
- average ranked preference
- Condition without feedback
- lowest performance
- highest workload
- lowest ranked preference
|
|
Hypotheses part 2
If the non-feedback experience of gaze in the physical world is most important:
- Condition without feedback
- highest performance
- lowest workload
- highest ranked preference
- Highlight condition
- average performance
- moderate workload
- average ranked preference
- Cursor condition
- lowest performance
- highest workload
- lowest ranked preference
|
|
Results
- Performance in the main task
- Reaction times in the gaze-based interaction
Highlight = Cursor > Without feedback
- Misses in the gaze-based interaction
Highlight = Cursor > Without feedback
- False alarms in the gaze-based interaction
Highlight = Cursor = Without feedback
- Performance in the secondary task
- Reaction times in the gaze-based interaction
Highlight = Cursor > Without feedback
- Misses in the gaze-based interaction
Highlight = Cursor > Without feedback
- False alarms in the gaze-based interaction
Highlight = Cursor > Without feedback
- Duration of each trial
Highlight = Cursor > Without feedback
- Perceived workload
- NASA-TLX
Highlight = Cursor > Without feedback
- SEA
Highlight = Cursor > Without feedback
- Acceptance measured as preference
Highlight < Cursor < Without feedback
|
|
Discussion
- Gaze-based interaction should be designed like the conditions experienced in the physical world
- Integration of the gaze-based interaction into the mUltimo3D system without added visual or acoustic feedback
- Natural interaction style is different for every modality
- Approved style guides need to be verified for the new interaction modalities and the combination of different modalities in advanced computer interfaces
|
|
|
