From tiny fitness trackers to huge industrial robots, we are interacting today with devices with shapes, sizes, and capabilities that would have been hard to imagine when the traditional graphical user interface (GUI) first became popular in the 1980’s. It is becoming increasingly apparent that the decades-old GUI interface is a poor fit for today’s computer-human interactions, as we move farther and farther away from the classic desktop paradigm, with input limited to mouse and keyboard, and a large screen as the only output modality. While the growth of touch interfaces has been especially dramatic, we are now also starting to see applications that make use of many other forms of interaction, including voice, handwriting, emotion recognition, natural language understanding, and object recognition.
As these forms of interaction (modalities) are combined into systems, the importance of having standard ways for them to communicate with each other and with application logic is apparent. The sheer variety and complexity of multimodal technologies makes it impractical for most implementers to handle the full range of possible modalities (current and future) with proprietary API's.
To address this need, the World Wide Web Consortium (W3C) has developed a comprehensive set of standards for multimodal interaction which are well-suited as the basis of interoperable multimodal applications. However, most of the information about these standards is currently available only in the formal standards documents, conference presentations, and a few academic journal papers. All of these can be hard to find, and are not very accessible to most technologists. In addition, papers on applications that use the standards are similarly scattered among many different resources.
This book addresses this gap with clearly-presented overviews of the full suite of W3C multimodal standards. In addition, to illustrate the standards in use, it includes case studies of a number of applications that use the standards. Finally, a future directions section discusses new ideas for other standards as well as new applications.
Overviews of the following standards:
Applications using the W3C standards for multimodal interaction:
Publication Date: November, 2016