http://www.idiap.ch/uist2006/ Translating Keyword Commands into Executable Code en Tue, 17 Oct 2006 00:00:00 +0200 no Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL Translating Keyword Commands into Executable Code Greg Little and Robert C. Miller MIT CSAIL This work was supported in part by the National Science Foundation under award number IIS-0447800. UkST3006 CSAIL Translating Keyword Commands into Executable Code Greg Little and Robert C. Miller MIT CSAIL This work was supported in part by the National Science Foundation under award number IIS-0447800. UkST3006 00:00:33 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Chickenfoot allows users to script the web without dealing with HTML CXX.CX3W atized Ham IS News Goo e Search r m F l.uc UIST Introduction CSAIL Chickenfoot allows users to script the web without dealing with HTML CXX.CX3W atized Ham IS News Goo e Search r m F l.uc UIST 00:00:13 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Chickenfoot allows users to script the web using Javascript CKX.CCXl News Goo e Search r m F UIST 2006 Introduction CSAIL Chickenfoot allows users to script the web using Javascript CKX.CCXl News Goo e Search r m F UIST 2006 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL All Scriptable Fimfox UlST 21106 Introduction CSAIL All Scriptable Fimfox UlST 21106 00:00:05 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Translating Keyword Commands into Executable Code Translating Keyword Commands into Executable Code 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Must leam new syntax Firefox UIST 2006 Introduction CSAIL Must leam new syntax Firefox UIST 2006 00:00:13 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Google uses keywords to search the space of web pages UIST 2006 Introduction CSAIL Google uses keywords to search the space of web pages UIST 2006 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le Ima e Search Purple Cow Image Search UIST 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le Ima e Search Purple Cow Image Search UIST 2006 00:00:07 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le e Search Purple Cow Image Search Why not use keywords to search the space of script commands onnected 00 P ddres 8 UIST 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le e Search Purple Cow Image Search Why not use keywords to search the space of script commands onnected 00 P ddres 8 UIST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le e Search Purple Cow Image Search Why not use keywords to search the space of script commands left margin 2 inches ActiveDocument.PageSetup.LeftMargin Inches ToPoi nts2 UIST 2006 Introduction CSAIL Google uses keywords to search the space of web pages Goo le e Search Purple Cow Image Search Why not use keywords to search the space of script commands left margin 2 inches ActiveDocument.PageSetup.LeftMargin Inches ToPoi nts2 UIST 2006 00:00:18 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL Language User study UIST 2006 CSAIL Language User study UIST 2006 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL Original Javascript T- findTextboxstreet address enterPO Box 777 UIST 2006 Language CSAIL Original Javascript T- findTextboxstreet address enterPO Box 777 UIST 2006 00:00:13 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL 0 Original Javascript Remove punctuation enter PO Box 777 findTextbox street address UIST 2006 Language CSAIL 0 Original Javascript Remove punctuation enter PO Box 777 findTextbox street address UIST 2006 00:00:07 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL Original Javascript Remove punctuation Split compound tokens enter PO Box 777 find Textbox street address UIST 2006 Language CSAIL Original Javascript Remove punctuation Split compound tokens enter PO Box 777 find Textbox street address UIST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL 0 Original Javascript Remove punctuation Split compound tokens TT find Textbox stre Rearrange phrases PO Box 777 enter find T Textbox street address UIST 2006 Language CSAIL 0 Original Javascript Remove punctuation Split compound tokens TT find Textbox stre Rearrange phrases PO Box 777 enter find T Textbox street address UIST 2006 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL Original Javascript Remove punctuation Split compound tokens find Textbox stree Rearrange phrases PO Box 777 enter find Remove words PO Box 777 enter street address UIST 2006 Language CSAIL Original Javascript Remove punctuation Split compound tokens find Textbox stree Rearrange phrases PO Box 777 enter find Remove words PO Box 777 enter street address UIST 2006 00:00:09 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL tl Original Javascript Remove punctuation Split compound tokens 7 find Textbox stree Rearrange phrases PO Box 777 enter find Remove words Add extraneous words PO Box 777 enter into street address UIST 2016 Language CSAIL tl Original Javascript Remove punctuation Split compound tokens 7 find Textbox stree Rearrange phrases PO Box 777 enter find Remove words Add extraneous words PO Box 777 enter into street address UIST 2016 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Language CSAIL 0 Original Javascript Remove punctuation Split compound tokens 777 find Textbox stree Rearrange phrases PO Box 777 enter find Remove words Add extraneous words 0 All are valid Keyword Commands, including ongmal Javascnpt UIST 2006 Language CSAIL 0 Original Javascript Remove punctuation Split compound tokens 777 find Textbox stree Rearrange phrases PO Box 777 enter find Remove words Add extraneous words 0 All are valid Keyword Commands, including ongmal Javascnpt UIST 2006 00:00:05 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Interface CSAIL UIST 2006 User Interface CSAIL UIST 2006 00:00:05 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 mac Nemls inmtD Images Show Smati images V tt-td click the images link press advanced all words pencil search 32x 32 pocels lk-g mac Nemls inmtD Images Show Smati images V tt-td click the images link press advanced all words pencil search 32x 32 pocels lk-g 00:00:08 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Maps News more m IODD Small mages 1-111 afalmw Pimm eanj Maps News more m IODD Small mages 1-111 afalmw Pimm eanj 00:00:37 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Maps News moce m Moderate IS ID ID ID ID Smatl imacjes 11-td click the images link ovw on small i show ad im sizes press advancedj all words pencil search 32x32 lk gd NW JIStOn net 34J Maps News moce m Moderate IS ID ID ID ID Smatl imacjes 11-td click the images link ovw on small i show ad im sizes press advancedj all words pencil search 32x32 lk gd NW JIStOn net 34J 00:00:20 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 kozilla Firefox GO to google, co click the images link press advanced all words pencil Googe Search r m FeE search qrns Business Solulb kozilla Firefox GO to google, co click the images link press advanced all words pencil Googe Search r m FeE search qrns Business Solulb 00:00:07 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 GooSe Image Search Goo le Ima e Search Go to google, corn Image Search click the press advanced Web Vi News mmm D Maps all words pencil Search Image search Bulless SOILItD GooSe Image Search Goo le Ima e Search Go to google, corn Image Search click the press advanced Web Vi News mmm D Maps all words pencil Search Image search Bulless SOILItD 00:00:09 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Goe Advancect image Search relat to the exEt pilmt press advancec all words pencil Rmmm mlqm lhat ae search Goe Advancect image Search relat to the exEt pilmt press advancec all words pencil Rmmm mlqm lhat ae search 00:00:13 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Google Advanced Image Search OTJ I TOP google, cure click the images link press advanced reialmd to my afthe imt all words penci ISD wab search Rmm inmxjes the sle or clm Google Advanced Image Search OTJ I TOP google, cure click the images link press advanced reialmd to my afthe imt all words penci ISD wab search Rmm inmxjes the sle or clm 00:00:09 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Interface CSAIL UIST 2006 User Interface CSAIL UIST 2006 00:00:06 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL Your Personal Irrfon na lekts ale teqU UIST 2006 CSAIL Your Personal Irrfon na lekts ale teqU UIST 2006 00:00:25 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL enterPO Box 777, findTextboxstreet address UlST 2006 CSAIL enterPO Box 777, findTextboxstreet address UlST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL enterPO Box 777n, findTextboxstreet addressn, 2 UIST 2006 CSAIL enterPO Box 777n, findTextboxstreet addressn, 2 UIST 2006 00:00:06 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL Chickenfoot enterPO Box 777, findTextboxstreet addressn, 2 UIST 2006 CSAIL Chickenfoot enterPO Box 777, findTextboxstreet addressn, 2 UIST 2006 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 CSAIL Chickenfoot enterPO Box 777n, findTextboxstreet addressn, 2 UIST 2006 CSAIL Chickenfoot enterPO Box 777n, findTextboxstreet addressn, 2 UIST 2006 00:00:16 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 UIST 2006 UIST 2006 00:00:06 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address Tokenize UIST 2006 address Tokenize UIST 2006 00:00:05 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address Split token UIST 2006 address Split token UIST 2006 00:00:14 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address Correct spelling UIST 2906 address Correct spelling UIST 2906 00:00:09 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address Correct spelling Chickenfoot UtST 2006 address Correct spelling Chickenfoot UtST 2006 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address Correct spelling APII Chickenfoot UIST 2006 address Correct spelling APII Chickenfoot UIST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 address LIL G UlST 2006 address LIL G UlST 2006 00:00:04 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 enter po box 777 UIST 2006 street address 2 enter po box 777 UIST 2006 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 enter po box 777 1. Try every root function in the API UlST 2006 street address 2 enter po box 777 1. Try every root function in the API UlST 2006 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 enter po box 777 1. Try every root function in the API 2. Try all ways to divide tokens between function and arguments UIST 2006 street address 2 enter po box 777 1. Try every root function in the API 2. Try all ways to divide tokens between function and arguments UIST 2006 00:00:07 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 enter po box 777 1. Try every root function in the API 2. Try all ways to divide tokens between function and arguments 3. Recursively find arguments UlST 2006 street address 2 enter po box 777 1. Try every root function in the API 2. Try all ways to divide tokens between function and arguments 3. Recursively find arguments UlST 2006 00:00:04 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 enter s po box 777 Textbox UIST 2006 street address 2 enter s po box 777 Textbox UIST 2006 00:00:19 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address 2 Textbox UIST 2006 street address 2 Textbox UIST 2006 00:00:14 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 street address findTextbox WebpageWords UIST 2006 street address findTextbox WebpageWords UIST 2006 00:00:27 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox s Weet address UIST 2006 findTextbox s Weet address UIST 2006 00:00:07 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox street address UIST 2006 findTextbox street address UIST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox s Weet address UIST 2066 findTextbox s Weet address UIST 2066 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox street address UIST 2006 findTextbox street address UIST 2006 00:00:04 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox street address UIST 2006 findTextbox street address UIST 2006 00:00:12 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox s Weet address UIST 2906 findTextbox s Weet address UIST 2906 00:00:10 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox street address UIST 2006 findTextbox street address UIST 2006 00:00:21 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox PO Box 777w street address UIST 2006 findTextbox PO Box 777w street address UIST 2006 00:00:04 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox PO Box 777w street address NOTE works without quotes but gets lower score UlST 2006 findTextbox PO Box 777w street address NOTE works without quotes but gets lower score UlST 2006 00:00:16 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox PO Box 777 street address Octal 2006 UIST 2006 findTextbox PO Box 777 street address Octal 2006 UIST 2006 00:00:08 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 findTextbox PO Box 777 sleet address UIST 2016 findTextbox PO Box 777 sleet address UIST 2016 00:00:14 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Study CSAIL Given a familiar domain, can users form valid Keyword Commands, without instructions UIST 2006 User Study CSAIL Given a familiar domain, can users form valid Keyword Commands, without instructions UIST 2006 00:00:28 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Study CSAIL Given a familiar domain, can users form valid Keyword Commands, without instructions Web domain chosen Familiar to many non-programmers UlST 2006 User Study CSAIL Given a familiar domain, can users form valid Keyword Commands, without instructions Web domain chosen Familiar to many non-programmers UlST 2006 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Study CSAIL Participants Solicited from public college mailing list 4 non-programmers, 5 programmers UTZIXI6 User Study CSAIL Participants Solicited from public college mailing list 4 non-programmers, 5 programmers UTZIXI6 00:00:27 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Study CSAIL Instructions lozitla Firefox UIST 2006 User Study CSAIL Instructions lozitla Firefox UIST 2006 00:00:19 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 User Study CSAIL o Tasks News Local S chotar I111ofe UUDI Business Solull ms- AIDU GOU UIST 21106 User Study CSAIL o Tasks News Local S chotar I111ofe UUDI Business Solull ms- AIDU GOU UIST 21106 00:00:18 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subieefs Results News rams- Business Solu ns- About Goo mum Qugh UlST 2006 subieefs Results News rams- Business Solu ns- About Goo mum Qugh UlST 2006 00:00:08 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 tasks subjects Results Rows are tasks Columns are users 111314261241 4 UIST 2006 tasks subjects Results Rows are tasks Columns are users 111314261241 4 UIST 2006 00:00:04 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subieets Results Rows are tasks Columns are users Cells are attempts This user tried twice, and succeeded on the second attempt 441 13 4 2 6124t11 4 45121211 5611451 571 2 g Z UIST 2006 subieets Results Rows are tasks Columns are users Cells are attempts This user tried twice, and succeeded on the second attempt 441 13 4 2 6124t11 4 45121211 5611451 571 2 g Z UIST 2006 00:00:26 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subjects Results 84o o non-programmers 95o o programmers 51121111 52111111 UIST subjects Results 84o o non-programmers 95o o programmers 51121111 52111111 UIST 00:00:14 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 tasks subjects Results 840 o non-programmers 95o o programmers 1.7 attempts per task Both groups 4411 4 2 12 45121211 511211111 Octal 2OO6 UIST 2006 tasks subjects Results 840 o non-programmers 95o o programmers 1.7 attempts per task Both groups 4411 4 2 12 45121211 511211111 Octal 2OO6 UIST 2006 00:00:13 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 tasks subjects Results 84o o non-programmers 95o o programmers 1.7 attempts per task 32 1 1 1 33111211111 3417394441,10 Both groups 72o o non-programmers 77o o programmers UIST 2D6 tasks subjects Results 84o o non-programmers 95o o programmers 1.7 attempts per task 32 1 1 1 33111211111 3417394441,10 Both groups 72o o non-programmers 77o o programmers UIST 2D6 00:00:08 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subjects Results Few verbs button I m Feeling Lucky GIF filetypes 4411 4 2 12 451212111 51121111 UlST 2006 subjects Results Few verbs button I m Feeling Lucky GIF filetypes 4411 4 2 12 451212111 51121111 UlST 2006 00:00:25 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subjects Results Wide variety of syntax field 3 777 Home Drive 777 Home Drive Street Address Street777 Home Drive er as a busmes 777 Home On UIST subjects Results Wide variety of syntax field 3 777 Home Drive 777 Home Drive Street Address Street777 Home Drive er as a busmes 777 Home On UIST 00:00:32 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 tasks subjects tasks 11 13. 13 14 Where it went wrong Unclear task Lack of synonyms search o the low, ng m. pages to s4decz Imns bald dmm me rlu. ke lc uses helcl dram me key cou Home olmges UIST 2006 tasks subjects tasks 11 13. 13 14 Where it went wrong Unclear task Lack of synonyms search o the low, ng m. pages to s4decz Imns bald dmm me rlu. ke lc uses helcl dram me key cou Home olmges UIST 2006 00:01:01 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 tasks subjects Where it went wrong Sltuff that mnm 33 3 35 X 73944410 Moppy UIST 2006 tasks subjects Where it went wrong Sltuff that mnm 33 3 35 X 73944410 Moppy UIST 2006 00:00:22 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 subieets Where it went wrong Paradigm errors UIST 2006 subieets Where it went wrong Paradigm errors UIST 2006 00:00:52 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application UIST 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application UIST 2006 00:00:41 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application Commands may not work in the future UIST 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application Commands may not work in the future UIST 2006 00:00:14 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application Commands may not work in the future This problem exists for formal languages Javascript can break as browsers change UIST 2006 Discussion CSAIL Interpretation depends on Weights and heuristics API Context state of application Commands may not work in the future This problem exists for formal languages Javascript can break as browsers change UIST 2006 00:00:45 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Related Work CSAIL Mandelin et al PLDI 05 Natural language programming Natural Java Pro rammin in Natural Lan U UIST 2006 Related Work CSAIL Mandelin et al PLDI 05 Natural language programming Natural Java Pro rammin in Natural Lan U UIST 2006 00:00:41 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Future Work CSAIL Apply Keyword Commands to more applications e.g. MS Word Apply to larger domains e.g. Java Use with speech recognition UIST 2006 Future Work CSAIL Apply Keyword Commands to more applications e.g. MS Word Apply to larger domains e.g. Java Use with speech recognition UIST 2006 00:00:24 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2006 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2006 00:04:20 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2906 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2906 00:00:11 no Greg Little, Robert Miller, Massachusetts Institute of Technology Wed, 18 Oct 2006 00:00:00 +0200 Translating Keyword Commands into Executable Code SCIENCE > 2006 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2006 Conclusion CSAIL Keyword Commands are programs with minimal and flexible syntax Brute force interpretation works in web scripting domain Users can generate keyword commands without instructions UIST 2006 00:04:49 no Greg Little, Robert Miller, Massachusetts Institute of Technology