Conversational Technologies

Speech Technology Consulting Services

Home

News

Projects

About Us

Online NL Demos

Multimodal Interaction

EMMA

InkML

Deborah Dahl is the Chair of the World Wide Web Consortium's Multimodal Interaction Working Group, which develops standards that support interaction on the web using capabilities such as speech and handwriting. W3C Multimodal Interaction standards include:
 
1.
Extensible Multimodal Interaction (EMMA), a modality-independent format for representing user input
2.
Ink Markup Language (InkML) for representing digital ink
3.
Emotion Markup Language for representing emotion
4.
The Multimodal Architecture, which provides a framework for integrating multiple modalities into a single application

MMI Javascript Libraries

Some Javascript libraries for using the MMI Architecture from a browser-based Interaction Manager with a cloud-based client for emotion recognition from natural language. There is an MMI Architecture library for building and parsing MMI Life Cycle Events, an EMMA library for building and parsing EMMA documents, a simple function for extracting EmotionML from EMMA, and some communications functions that can be used with HTTP and Ajax for transport. The full demo of emotion recognition can be seen here.
 
EMMA
A demo of EMMA.
 
Speech Application Framework
A simple open source framework for Android speech applications, using the native Android speech recognizer with HTML5, Javascript, CSS and Apache Cordova (PhoneGap). Github source.