Fulltext

Description languages for multimodal interaction: a set ofguidelines and its illustration with SMUIML

Dumas, Bruno ; Lalanne, Denis ; Ingold, Rolf

In: Journal on Multimodal User Interfaces, 2010, vol. 3, no. 3, p. 237-247

Add to personal list
    Title
    • Description languages for multimodal interaction: a set ofguidelines and its illustration with SMUIML
    Author
    • Dumas, Bruno. Computer Science Department, University of Fribourg, Boulevard de Pérolles 90, 1700, Fribourg, Switzerland
    • Lalanne, Denis. Computer Science Department, University of Fribourg, Boulevard de Pérolles 90, 1700, Fribourg, Switzerland
    • Ingold, Rolf. Computer Science Department, University of Fribourg, Boulevard de Pérolles 90, 1700, Fribourg, Switzerland
    Document Type
    • Postprint
    OAI-PMH Identifier
    • oai:doc.rero.ch:309696
    Summary
    This article introduces the problem of modeling multimodal interaction, in the form of markup languages. After an analysis of the current state of the art in multimodal interaction description languages, nine guidelines for languages dedicated at multimodal interaction description are introduced, as well as four different roles that such language should target: communication, configuration, teaching and modeling. The article further presents the SMUIML language, our proposed solution to improve the time synchronicity aspect while still fulfilling other guidelines. SMUIML is finally mapped to these guidelines as a way to evaluate their spectrum and to sketch future works