http://www.idiap.ch/uist2006/ Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition en Tue, 17 Oct 2006 00:00:00 +0200 no Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Sensing from the Basement A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition James Fogarty Carolyn Au Scott E. Hudson dub use build university of weshington Sensing from the Basement A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition James Fogarty Carolyn Au Scott E. Hudson dub use build university of weshington 00:00:18 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Trend in Percent of 65 Individuals in United States from 1900 to 2030 213.1313 12.50 12.79 5.00 407 431 4.35 45 191M 1920 1940 1950 1960 1970 1980 1990 2OOO 2010 2120 2O3O Trend in Percent of 65 Individuals in United States from 1900 to 2030 213.1313 12.50 12.79 5.00 407 431 4.35 45 191M 1920 1940 1950 1960 1970 1980 1990 2OOO 2010 2120 2O3O 00:00:57 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Technology for Aging Place Many interventions have been proposed Providing awareness and peace of mind to distant adult children Collecting detailed activity logs for examination by a medical professional Detecting anomalous behavior patterns that may suggest a fall or urgent situation These applications generally include both sensing and interactive components Technology for Aging Place Many interventions have been proposed Providing awareness and peace of mind to distant adult children Collecting detailed activity logs for examination by a medical professional Detecting anomalous behavior patterns that may suggest a fall or urgent situation These applications generally include both sensing and interactive components 00:00:50 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Sensing from the Basement This work is focused on a new approach to sensing for home activity recognition Our approach is to reach into the home via the existing infrastructure, using just a few inexpensive sensors at key locations Sensing from the Basement This work is focused on a new approach to sensing for home activity recognition Our approach is to reach into the home via the existing infrastructure, using just a few inexpensive sensors at key locations 00:00:33 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Sensing from the Basement This work is focused on a new approach to sensing for home activity recognition Our approach is to reach into the home via the existing infrastructure, using just a few inexpensive sensors at key locations Wated Sensing from the Basement This work is focused on a new approach to sensing for home activity recognition Our approach is to reach into the home via the existing infrastructure, using just a few inexpensive sensors at key locations Wated 00:00:22 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Sensing from the Basement Sensing from the Basement 00:00:29 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Overview Introduction and Motivation Related Work Field Study Environment Prototype Sensor Development Data Collection and Overview Recognition Algorithm and Results Discussion and Conclusion Overview Introduction and Motivation Related Work Field Study Environment Prototype Sensor Development Data Collection and Overview Recognition Algorithm and Results Discussion and Conclusion 00:00:30 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Motivating Applications Monitoring an elder s activities of daily living in support of a local or remote caregiver Digital Family Portrait natt et al, CHI CHI Rowan and Mynatt. CareNet Display Motivating Applications Monitoring an elder s activities of daily living in support of a local or remote caregiver Digital Family Portrait natt et al, CHI CHI Rowan and Mynatt. CareNet Display 00:00:57 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Existing Approach Extensive Sensor Installation Rowan and Mynatt s field study used strata sensors installed under a home s floorboards 19NN Several groups have proposed systems based on cameras and mtcrophones tn a home s living space Existing Approach Extensive Sensor Installation Rowan and Mynatt s field study used strata sensors installed under a home s floorboards 19NN Several groups have proposed systems based on cameras and mtcrophones tn a home s living space 00:01:00 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Existing Approach Many Simple Sensors Munguia Tapia et al examine home activity recognition using many simple contact switches Wilson and Atkeson use home security hardware to track the locations of people in a home Penrasive 2005 Existing Approach Many Simple Sensors Munguia Tapia et al examine home activity recognition using many simple contact switches Wilson and Atkeson use home security hardware to track the locations of people in a home Penrasive 2005 00:01:01 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Existing Approach Wearable Sensing Phdipose et ai present a RFID wearable RFID reader Power Board in a bracelet form factor Perv Computi Existing Approach Wearable Sensing Phdipose et ai present a RFID wearable RFID reader Power Board in a bracelet form factor Perv Computi 00:00:32 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Kitchen Sink 1- Clothes Kitchen Sink 1- Clothes 00:00:47 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Kitchen Sink 1- Kitchen Sink 1- 00:00:26 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Kitten Sink Kitten Sink 00:00:29 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Prototype Sensor Mica2 Berkeley Mote B attery-po wered w wireless communication Microphone pressed directly against the pipe Decided to log the zero crossing rate and the root mean square of audio samples at 0.5 Hz Prototype Sensor Mica2 Berkeley Mote B attery-po wered w wireless communication Microphone pressed directly against the pipe Decided to log the zero crossing rate and the root mean square of audio samples at 0.5 Hz 00:01:00 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Logged Sensor Signal Sensor Logged Sensor Signal Sensor 00:00:41 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Logged Sensor Signal Sensor Stack Sensor Cold Water Sensor Logged Sensor Signal Sensor Stack Sensor Cold Water Sensor 00:00:44 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Logged Sensor Signal Sensor Stack Sensor Cold Water Sensor Entropy Transformation Logged Sensor Signal Sensor Stack Sensor Cold Water Sensor Entropy Transformation 00:00:15 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Data Collection Deployed for 40 days, we collected approximately 3.4 million sensor readings Designated first 7 days as our training data, using it to develop our recognition system Statistical model of whether a pipe is on Set of patterns for matching activities Remaining 33 days are used as test data Data Collection Deployed for 40 days, we collected approximately 3.4 million sensor readings Designated first 7 days as our training data, using it to develop our recognition system Statistical model of whether a pipe is on Set of patterns for matching activities Remaining 33 days are used as test data 00:00:52 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Data Overview Test data contains Clothes Washer activations Dishwasher activations Shower activations Toilet activations Kitchen Sink activations Bathroom Sink activations We are not dirty people, the detected number of Bathroom Sink activations is lower than actually occurred, see paper Data Overview Test data contains Clothes Washer activations Dishwasher activations Shower activations Toilet activations Kitchen Sink activations Bathroom Sink activations We are not dirty people, the detected number of Bathroom Sink activations is lower than actually occurred, see paper 00:00:58 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Recognition Patterns Developed an algorithm to match activities as patterns of sensor activations Mostly On. seconds Mostly Off Mostly On Mostly Off seconds seconds seconds Recognition Patterns Developed an algorithm to match activities as patterns of sensor activations Mostly On. seconds Mostly Off Mostly On Mostly Off seconds seconds seconds 00:00:38 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Recognition Algonthm Matched patterns from complex to simple Clothes Washer Dishwasher Shower Remaining water labeled as sink activation Bathroom sink if there is stack activity, a toilet flush, or a shower in close proximity Otherwise, kitchen sink activity Recognition Algonthm Matched patterns from complex to simple Clothes Washer Dishwasher Shower Remaining water labeled as sink activation Bathroom sink if there is stack activity, a toilet flush, or a shower in close proximity Otherwise, kitchen sink activity 00:01:10 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Structured Activity Results Actual Found Recall False Clothes Washer Dishwasher Shower Structured Activity Results Actual Found Recall False Clothes Washer Dishwasher Shower 00:01:17 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Sink Results Actual Found Recall False Lo o Bathroom bink Lono Kitchen hnk Short Bathroom hink Short Kitchen Sink Sink Results Actual Found Recall False Lo o Bathroom bink Lono Kitchen hnk Short Bathroom hink Short Kitchen Sink 00:00:43 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Results and Tradeoffs Home activity sensing is a set of tradeoffs Utility of information collected Installation cost Maintenance cost Intrusiveness into home life Our approach is obviously limited in what information it can collect, but water activity is very useful for our motivating applications and we are at a sweet spot in the tradeoffs Results and Tradeoffs Home activity sensing is a set of tradeoffs Utility of information collected Installation cost Maintenance cost Intrusiveness into home life Our approach is obviously limited in what information it can collect, but water activity is very useful for our motivating applications and we are at a sweet spot in the tradeoffs 00:00:42 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Future Work Explore more robust and automated methods Considering more than onoff Examine a larger variety of homes Combining our approach with other sensing Electricity, natural gas, etc. Solving the sink problem with hardware Location and object interaction systems Study minimal training requirements Future Work Explore more robust and automated methods Considering more than onoff Examine a larger variety of homes Combining our approach with other sensing Electricity, natural gas, etc. Solving the sink problem with hardware Location and object interaction systems Study minimal training requirements 00:02:23 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University Wed, 18 Oct 2006 00:00:00 +0200 Sensing from the Basement: A Feasibility Study of Unobtrusive and Low-Cost Home Activity Recognition SCIENCE > 2006 Thank You Questions. James Fogarty jfoga rtycs washington. edu Scott E. Hudson scott.h udsoncs.cmu.edu This material is based upon work supported by the Defense Advanced Research Projects Agency DARPA under Contract No. NBCHD030010, and by the National Science Foundation under grants IIS-0121560 and IIS-0325351. Thank You Questions. James Fogarty jfoga rtycs washington. edu Scott E. Hudson scott.h udsoncs.cmu.edu This material is based upon work supported by the Defense Advanced Research Projects Agency DARPA under Contract No. NBCHD030010, and by the National Science Foundation under grants IIS-0121560 and IIS-0325351. 00:06:52 no James Fogarty, Carolyn Au, Scott Hudson, University of Washington, Carnegie Mellon University