1	Standards-Based Programming Model Rafah Hosn, Team Lead Interaction Middleware and Standards for Portal Server IBM T.J. Watson Research Center Yorktown Heights, NY
2	OUTLINE Programming Model What is a programming model? Why do we need to standardize it? Next Generation Multimodal Application Features & Requirements Standard Based Multimodal Programming Model Remaining Challenges Conclusion
3	What Is A Programming Model?
4	Programming Model A programming model defines: How applications are authored. Processing model of components on which applications are deployed. Examples: Web programming model. Java programming model.
5	Why Do We Need A Standard?
6	Open Standards Transparency Open architecture. Well defined control flow. Governance Evolution of technology governed by group of vendors. Eliminates dominance of any one vendor. Consistency Components can be assembled from multiple providers. Important for rapid development.
7	Features Of Next Generation Multimodal Applications
8	Features Of Next Generation Multimodal Applications Support diversity of clients and platforms Interoperability. Distributed processing. Support rich conversational user interfaces Pluggable dialog management strategies. Appropriate interfaces for natural language processing, statistical language models, etc.
9	Features Of Next Generation Multimodal Applications (cont’d) Reduce latency time Partial user interface updates. Efficient distribution. Rapid Experimentation Interoperability. Higher level abstractions in authoring.
10	Requirements For The Multimodal Programming Model Extensible. Interoperable. Distributable. Easy to use.
11	Standards-Based Multimodal Programming Model
12	Elements Of A Standards-Based Multimodal Programming Model Foundation Blocks Event driven processing MVC Architecture Declarative Authoring
13	Elements Of A Standards-Based Multimodal Programming Model Event driven processing Glues components of a multimodal framework together. Behaviour of components can be affected by events raised externally, this implies: Same eventing interfaces. Same eventing model. No dependency on underlying implementation. Enables interoperability Enables efficient distribution
14	Elements Of A Standards-Based Multimodal Programming Model Foundation Blocks Event driven processing MVC Architecture Declarative Authoring
15	Elements Of A Standards-Based Multimodal Programming Model (cont’d) Model-View-Controller (MVC) architecture Centralized application state held in model: Efficient for passing between components. Used for partial update of client view. Separate controller Pluggable customized dialog strategies. Decreases latency time Increases extensibility
16	Elements Of A Standards-Based Multimodal Programming Model Foundation Blocks Event driven processing MVC Architecture Declarative Authoring
17	Elements Of A Standards-Based Multimodal Programming Model (cont’d) Declarative authoring Provides a higher level of abstraction Easier to learn. Easier to maintain. Reduces development time. Enables rapid experimentation
18	Web Programming Model Why? Proliferation of Web access. Maturity of Web standards. Satisfy multimodal requirements. Standards For Web programming model W3C XML data model (XForms), XML Schema. W3C DOM 2 Event propagation algorithm, XML Events. W3C standards for multiple modalities Keyboard, mouse access (XHTML), Speech (VoiceXML) W3C Multimodal Interaction (MMI) working group.
19	Standards-Based Multimodal Web Programming Model Multimodal Mobile Real Estate Agent Configuration HP iPaq WiFi/2.5G XHTML+Voice application XHTML browser on device. VoiceXML platform on server. Includes Natural Language (NL) processor Interaction Manager on server. Sample Interaction “Show me one or two bedroom condos in Manhattan, upper west side“ XHTML list of matching results returned to client Top 3 matches are played to user
20	Standards-Based Multimodal Web Programming Model
21	Distributed Standards-Based Multimodal Web Programming Model Distributed Programming Model Device has GPRS, can automatically fill Zip Code Handler to fill Zip Code runs on server User clicks on Zip Code field “Click” event is raised on XHTML browser XML Event is dispatched on server GPRS location & XML Event sent to server Script to resolve GPRS invoked server side Results returned as XML instance Partial update to client view using XML instance User sees Zip Code result
22	Distributed Standards-Based Multimodal Web Programming Model
23	Standard-Based Multimodal Web Programming Model Author XML Declarative authoring (XHTML, VoiceXML) XML data model, XML Schema for application state DOM events to wire interaction Consume XML Web based clients that consume and render XML Support DOM 2 event propagation Distribute Using XML XML Events XML data model
24	Remaining Challenges
25	Remaining Challenges Interaction management Introduce declarative hooks for dialog management. Adapt synchronization in a distributed environment. Context management Dynamic context in mobile space W3C MMI Systems & Environment group Incorporate context Application data and history for dialog management User preferences & static device profile for output generation
26	Remaining Challenges (cont’d) Distributed programming model Hard to automate distribution. Affects usability. Additional XML standards for Gaze Gesture Pen input Touch point 3D Gestures
27	Conclusion
28	Conclusion Multimodal interaction is a necessity for next generation applications. Multimodal is an emerging field with remaining technology challenges. Standards-based multimodal programming model provides an open, extensible platform for testing and deploying innovating ideas in this area.
29	END !!