WO2000039964A1 - Home control system with distributed network devices - Google Patents

Home control system with distributed network devices Download PDF

Info

Publication number
WO2000039964A1
WO2000039964A1 PCT/EP1999/010197 EP9910197W WO0039964A1 WO 2000039964 A1 WO2000039964 A1 WO 2000039964A1 EP 9910197 W EP9910197 W EP 9910197W WO 0039964 A1 WO0039964 A1 WO 0039964A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
appliance
control
action
location
Prior art date
Application number
PCT/EP1999/010197
Other languages
French (fr)
Inventor
Doreen Y. Cheng
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Priority to JP2000591754A priority Critical patent/JP2002534841A/en
Priority to EP99965516A priority patent/EP1057302A1/en
Priority to KR1020007009560A priority patent/KR20010041425A/en
Publication of WO2000039964A1 publication Critical patent/WO2000039964A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2805Home Audio Video Interoperability [HAVI] networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2816Controlling appliance services of a home automation network by calling their functionalities
    • H04L12/282Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2823Reporting information sensed by appliance or service execution status of appliance services in a home automation network
    • H04L12/2827Reporting to a device within the home network; wherein the reception of the information reported automatically triggers the execution of a home appliance functionality
    • H04L12/2829Reporting to a device within the home network; wherein the reception of the information reported automatically triggers the execution of a home appliance functionality involving user profiles according to which the execution of a home appliance functionality is automatically triggered
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/40Bus networks
    • H04L12/40052High-speed IEEE 1394 serial bus
    • H04L12/40117Interconnection of audio or video/imaging devices
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles
    • H04L67/306User profiles
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/535Tracking the activity of the user
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/4104Peripherals receiving signals from specially adapted client devices
    • H04N21/4131Peripherals receiving signals from specially adapted client devices home appliance, e.g. lighting, air conditioning system, metering devices
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/441Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card
    • H04N21/4415Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213Monitoring of end-user related data
    • H04N21/44218Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4662Learning process for intelligent management, e.g. learning user preferences for recommending movies characterized by learning algorithms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L2012/2847Home automation networks characterised by the type of home appliance used
    • H04L2012/285Generic home appliances, e.g. refrigerators
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]

Definitions

  • This invention relates to the field of consumer devices, and in particular to the field of automated home control systems.
  • HANi is a software architecture using IEEE 1394
  • Home API is an open industry specification that defines a standard set of software services and application programming interfaces which enable software applications to monitor and control home device.
  • Jini is a distributed software architecture (network) wherein clients see devices and services as objects. Jini has a discovery service to enable a device or service to register itself with the network. Jini has a look-up service to enable determining what objects are available. Coherence among the distributed objects is enabled through leasing of objects, i.e., providing control over an object to another object for a predetermined time period. If the system fails, the system is brought back upon termination of the lease to a well defined state.
  • Typical home automation systems are configured to provide a central control station and a number of remote controllers.
  • the central control station may be a home computer
  • the remote controllers may be sub-controllers located in particular areas of the home, such as in a master bedroom, entry foyer, and the like.
  • Typical home automation systems may also include remote sensors that are used, for example, to automatically turn lights on or off when motion is detected, or to turn a television set on or off in response to a particular sound or voice command.
  • Some home automation systems allow the desired operations to be preprogrammed, so that, for example, lights or appliances are turned on or off at different preset times, televisions are tuned to different channels at different times, and so on.
  • a preprogrammed home automation system typically utilizes a central controller to effect the preprogrammed actions.
  • the automated control is directly dependent upon the reliability and availability of the central controller.
  • the central controller in most home automation systems is typically the home computer, which is often used for other applications as well. When used as the home automation controller, care must be taken to assure that the computer remains in operation and that the controller application continues to run independent of the other applications. A "crash" of the computer or the controller application will typically disable a significant portion, if not all, of the home automation system.
  • a dedicated central controller is often utilized as the central controller, eliminating the likelihood of a crash caused by another application, but the options available on such dedicated controllers are typically limited compared to those available on a general purpose home computer.
  • a distributed network of locator devices and control devices throughout a locale such as a home.
  • Devices within the locale are controlled by detecting the presence of an identified user within different areas of the locale, and coordinating the control of appliances within the area in response to this user's presence or absence.
  • the locating and control devices may be stand-alone devices, or integrated within other electronic devices, such as televisions, stereos, computers, and so on.
  • user task modules that suggest control actions based on the location of the user, the current context, and a profile of the user based upon the user's prior actions. Communications among the modules and devices are effected using standard network systems and protocols, including IEEE 1394, X10, HANi, and HomeAPI.
  • the determination of each user's location is facilitated by the use of a tracker module that the user carries about. The user may determine the degree of automation to be applied at any time.
  • FIG. 1 is an example block diagram of a home control system in accordance with this invention.
  • FIG. 2 is an example data flow diagram of a home control system in accordance with this invention.
  • FIG. 3 is an example flow chart for a control module in a home control system in accordance with this invention.
  • FIG. 1 illustrates an example block diagram of a home control system in accordance with this invention. Illustrated in FIG. 1 are four rooms, a family room 100, a kitchen 200, an office 300, and a utility room 400. In each of these rooms are an assortment of devices; in the vocabulary of home control systems, devices that perform a function other than control are termed appliances. Appliances may include control functions, but their "primary" appliance function is not control, per se.
  • a web-tv appliance 110 In the family room 100 are located in the family room 100 a web-tv appliance 110, a television appliance 120, a stereo appliance 130, and light appliances 140.
  • the example web-tv appliance 110 has an appliance function of providing web-access via a television, and also includes a control module CM 114, a user task module UTM 116, and a locator module LM 119. Another control module CM 134 is located in the example stereo appliance 130, and another locator module, LM 190 is located in the family room 100.
  • the illustrated devices and modules are interconnected via a home network 50. To avoid unnecessary clutter, the example connections to the network 50 are illustrated by "X" - labeled terminators. In a preferred embodiment, the communications via the network are in accordance with existing standards.
  • a HANi-compliant device has, as a minimum, enough functionality to communicate with other devices in the system.
  • devices may exchange control and data in a peer-to-peer fashion. This ensures that at the communication level, none of the devices is required to act as the master or controller of the system. On the other hand, it allows a logical master or controller to impose a control structure on the basic peer-to-peer communication model.
  • copending US application "Computer-controlled Home Theater With Independent User-control", serial number
  • 08/920,751 filed 8/29/97 for Jeroen Heuvelman et al., and incorporated herein by reference, presents a home entertainment system comprising a sub-system with audio/video functionalities, a computer and a device interface that interconnects the sub-system and the computer.
  • the computer controls the sub-system through a software application
  • the computer and the sub-system have separate user-controls. In case the computer crashes, the user can still control those audio/video functionalities of the sub-system that are independent of the computer.
  • each occupant of the home possesses a tracker 10 which is configured to enable the remote identification of the associated occupant.
  • the location of the tracker 10 is assumed to coincide with the location of the associated occupant, or "user"; the tracker 10, for example, may be a small token that the user carries or wears.
  • the locators 119 and 190 are configured to recognize it; in so doing, the locator modules 119 and 190 enables the identification of the associated occupant as being located in the family room 100.
  • the trackers and locator modules are coupled via a wireless connection, thereby allowing the user to move about freely.
  • the home network 50 may be wired or wireless, or a combination of both.
  • the locating modules 119, 190 communicate the presence of the user associated with tracker 10 to the home network 50; in particular, to control modules 114, 134, and others.
  • the control modules query one or more of the user task modules, including UTM 116 and others, for an appropriate control response. For example, if the user that is associated with tracker 10 normally prefers to hear classical music in the afternoon, user task module 116 will communicate a suggestion to control module 134 to play classical music, if the time of day is the afternoon; if the time of day is morning, the user task module 116 may send a suggestion to the control module 134 to tune to the local news-radio station.
  • the control module 134 communicates the appropriate commands, if any, to the appropriate appliances to effect a chosen response from among the suggestions and rules.
  • one of the rules may be to preclude a change of mode of an appliance in the family room 100 if another user (as indicated by the presence of another tracker) is in the family room 100.
  • priority schemes can be used to determine actions to be taken if multiple trackers are in the same area.
  • non-conflicting actions are effected, such as turning lights on if at least one of the users have indicated this preference and the other user has not explicitly expressed an opposite preference
  • conflicting actions such as turning a television and a stereo on
  • each rule or preference for a user is assigned a priority. For example, a priority can be established for a particular user for turning on a light whenever the user enters an area, regardless of conflicts. A lower priority for that same user can be established for turning on a television only if no conflicts exists.
  • control modules and user task modules employ knowledge based and machine learning techniques, and others, to determine an appropriate response upon each user's arrival to, or departure from, particular areas of the home.
  • the response is based on prior experiences and feedback from each user, as well as the user's recent history. For example, the response may differ depending upon whether the user arrives in the family room 100 after having been in the kitchen 200, or the office 300. These determinations will also be dependent upon the current context, such as the time of day, the day of the week, and other factors that are determined to be correlated to the user's past behavior, or assumed typical behavior.
  • the learning techniques are multi-variate in nature, so that a synergy of control is achieved.
  • a learned response may be to lower curtains, dim lights, play soft music, and place the telephone in autoanswer mode in response to a particular set of user patterns, and to keep the area well lit and turn on the radio in response to a different set of user patterns.
  • the user will typically effect an alternative control response. For example, if the control module 134 turns the stereo 130 on, and the user disagrees with this response, the user will typically turn the stereo 130 off, or effect some other change.
  • the user responses that immediately follow a control module's execution of a command to an appliance are weighted heavily for training, to effect a potentially different selection by the control module in the future.
  • Subsequent user selections are weighted less for training as the time interval between the control module's action and the user selection increases. For example, if the user listens to the classical music that the control module 134 selects for an hour or so, and then selects popular music, the selection of popular music may be processed by the control module 134 as an independent user preference, or as a sequential preference (play classics for one hour, then switch to popular), or as a disapproval of the control module's selection of classical music, or a combination of the three rationales.
  • each control module is allocated a position in a control hierarchy with respect to each appliance.
  • Each highest ranking control module that is currently available controls each appliance.
  • the suggestions from each user task module and from among the user task modules are given decision weights relative to each appliance, to ease the decision process of each control module.
  • the example kitchen 200 of FIG. 1 includes a control unit 210, a television appliance 220, speaker appliances 230, and a microwave appliance 240.
  • the example control unit 210 is illustrated as containing a control module 214, a user task module 216, and a locator module 219, while the appliances are devoid of control devices.
  • the locator module 219 notifies the appropriate devices in the home network 50, including the control module 214.
  • the control module 214 accesses user task modules within the network 50, including the user task module 216, to determine if any actions are suggested, and accesses its internal rules to determine an action. For example, if the tracker 10 had previously been in the family room 100, and the television 120 in the family room 100 had been tuned to a particular channel, the internal rules of control module 214 would suggest turning on the television 220 in the kitchen, and tuning it to the same channel as the television 120. In like manner, in this example, the internal rules of control module 114 will suggest that television 120 be turned off, if no other users are detected in the family room 100. Note that the above sequences of suggestions are based on rules that are independent of the particular user that departs the family room 100 and enters the kitchen 200.
  • control module that is responsible for controlling the particular appliances. For example, a particular user may prefer to listen to music while in the kitchen 200, and would not desire to have the television 220 turned on, even though television 120 had been turned on by or for this user.
  • the control module that is configured to control the television 220 and the control module that is configured to control the speaker appliances 230 apply these user-specific rules and preferences and will not turn the television 220 on, and will turn the speakers 230 on; if appropriate, based on the user specific rules and current contexts, the stereo 130 will also be tuned to provide the appropriate selection of music to the speakers 230.
  • the aforementioned rules are structured so as to anticipate a user's desire, based on experience or default assumptions.
  • a default assumption would be that when a user enters an area at a particular time of day, the lights 140, 340 should be turned on, and when room is unoccupied, the lights 140, 340 and entertainment appliances 120, 130, 220, 230, 320, 330 are turned off.
  • the default assumptions may effect sequential operations.
  • the control module 114 can be structured to wait for a predetermined time period before turning off the television 120 when the user departs the family room 100, to avoid an annoying shut-down, start-up sequence every time the user moves about the areas.
  • the control modules may determine the appropriate action based upon a user's recent activities.
  • the controller effects some actions, such as turning on lights, but not others, such as turning on stereo sets or televisions.
  • some actions such as turning on lights, but not others, such as turning on stereo sets or televisions.
  • the default assumption is that the user would prefer to have light for movement through adjoining rooms, but would not prefer to have other appliances turned on and off during each transit through a room.
  • the user task modules and control modules are preprogrammed with default rules that are subsequently updated based on user experience, or direct user modification.
  • the microwave appliance 240 may include a programmable device that allows a user to preprogram the microwave to start at a particular time, for a particular duration, and so on.
  • An alternative embodiment of the microwave appliance 240 could allow the user to preprogram the start based on the return of a person to the home, rather than on a preprogrammed time.
  • the control module 214 in this example signals the microwave appliance 240 when the tracker that is associated with the specified person is next detected within the home.
  • the start could also be triggered by other events as well, such as the entry of the person to a particular room within a particular time period.
  • a configuration module 312 in a computer 310 in the home office 300 Illustrated in FIG. 1 is a configuration module 312 in a computer 310 in the home office 300.
  • the configuration module 312 enables the system to be customized for each user directly.
  • the configuration module 312 is also used to enable a user or installer to record the system configuration in terms of the location of modules and appliances throughout the house, and coordinates the allocation of controlled appliances to control modules.
  • the configuration module 312 is also used to enable an automatic or manually initiated download of the rules and preferences to the control modules and user task modules as required.
  • the knowledge base and machine learning functions are preferably effected in the user task modules 116, 316, 416 that are distributed about the network 50, while the configuration module 312 effects actions required to coordinate these activities.
  • the configuration module allocates responsibilities to balance the computational load as devices are added to or removed from the network.
  • the configuration module dynamically allocates responsibilities for appliances among the control modules, and responsibilities for users among the user task modules.
  • the configuration module 312 also responds to queries by the individual modules regarding configuration status and information.
  • the rules regarding the operation of the external control modules 114, 134, 214, and 414 are configured so as not to be dependent upon the continuos operation of the computer 310 or the configuration module 312. In this manner, the reliability of the system in accordance with this invention is not dependent upon the reliability and availability of a central controller, such as computer 310.
  • the overall system reliability may also be enhanced by utilizing the principles of copending US application "Slave DSP Reboots Stalled Master CPU", serial number 08/880,387, filed 6/23/97 for Paul Foster et al., and herein incorporated by reference.
  • This document relates to a digital home entertainment system comprising one or more slave processors, for example, digital signal processors (DSPs), for processing specific tasks, and a master processor, for example, a central processing unit (CPU), for control of the system.
  • the slave processor is capable of rebooting the master processor if the master processor has stalled. This slave- controlled rebooting avoids manual cold rebooting of the system and is particularly advantageous in open-architecture multimedia systems with asynchronously cooperating components.
  • the user task modules 116, 316, 416 may each be designed to process particular information.
  • the user task module 116 in the web-tv appliance 110 may be designed to specifically facilitate the optimization of each user's access to the web.
  • copending U.S. patent application "Customized Upgrading of Internet-Enabled Devices Based on User-Profile", attorney docket PHA 23,500, serial number 09/160,490, filed 9/25/98 for Adrian Turner et al, and incorporated by reference herein, presents a method of retrieving information from the Internet based upon a user profile.
  • the locator module 119 notifies the other modules on the network of the presence of the user associated with tracker 10.
  • the control module 114 accesses the Internet and retrieves the information based on this user profile in accordance with the above mentioned copending application.
  • the user task module 416 in the environmental control appliance 410 may be designed to facilitate the optimization of the home environment in response to user preferences and experiences.
  • the user task module 416 may know nothing about web services, and the user task module 116 may know nothing about environmental services. That is, each user task module's function can be specialized to a particular knowledge domain.
  • the user task module 316 on the other hand, being located in a computer 310, may be designed to handle all available services.
  • the resolution of conflicting suggestions from multiple task modules can be resolved using the aforementioned priority scheme, wherein each task module has a ranking of control, and the suggestions from the currently highest ranking module is given priority.
  • the user task modules communicate among themselves and consistent rules are formulated.
  • the user task module 316 in the computer 310 periodically gathers information from each of the other user task modules and attempts to resolve conflicts using conventional decision theory techniques, such as voting, prioritize voting, and the like.
  • each module can vary, and less capable devices can rely upon other devices to varying degrees.
  • the aforementioned web-optimized user task module 116 may include an ability to store an activity history corresponding to the use of the television 120 and stereo 130, and provide suggestions to the control modules 114, 134 based on these histories and a primitive level of machine learning. Periodically, the user task module 316 obtains this activity history from the user task module 116, develops more sophisticated rules and suggestions based upon these histories, and communicates these new rules and suggestions to other modules throughout the system as appropriate.
  • Differing levels of control may be included in the home automation system in accordance with this invention.
  • a tracker 11 that is associated with a different user than the one associated with tracker 10.
  • the locator module 390 detects the presence of tracker 11, and notifies the network 50.
  • the computer 310 and the environment control 410 effect different control.
  • the device command module 414 in the environment control 410 may merely send a signal to an environment network 405, such as a furnace or air conditioning system, to adjust the level of the temperature in the office 300, based on the data in the user task module 416 corresponding to the user associated with tracker 11.
  • the user task module 416 in this example may merely contain the last temperature that this user selected, with no knowledge based or learning machine capabilities.
  • the control module 314 in the computer 310 may effect a number of complex actions, such as establishing contact to an external network 305 to retrieve the user's e-mail, initiating one or more of the applications, such as a word processor, that the user typically uses, reciting the received e-mail messages via the speakers 330, accepting voice commands from the user, and so on. These actions will have been determined based on the user's prior history, the user's explicit commands, or a set of default options, as discussed above.
  • the trackers 10, 11 and locator modules 119, 190, 290, 390 are designed to operate together so as to determine the relative or absolute location of each tracker 10 in the vicinity of each module. Any number of available technologies may be employed to facilitate this determination.
  • the tracker emits a signal that is only receivable by locator modules within its vicinity, using for example infrared signaling.
  • the locator module emits a radio frequency (RF) trigger signal, and the trackers that are within the vicinity of the locator module reflect a modulated form of the RF trigger signal back to the locator modules.
  • RF radio frequency
  • the tracker may be of varying complexity. In a single-user application, the tracker need only be a device that is detectable by a locator module. In a multiple-user application, each tracker includes a means of communicating a unique identifier. In a preferred implementation, the tracker also includes user selectable options, including the options to turn the tracker off, or to enter a manual mode, wherein the locator modules continue to determine the location of the tracker, but some or all of the automated actions by the control modules are inhibited. The tracker may also be configured with a means for the user to communicate commands directly. Copending application "Remote Control Device with Location Dependent
  • Kemink presents a method and apparatus for providing the user of the apparatus differing control capabilities in dependence upon where the user is located.
  • the tracker of this invention may include some or all of the capabilities described in this copending application, such that, for example, when the user selects a manual mode, the tracker provides user options based upon the appliances within the user's immediate area.
  • FIG. 2 illustrates an example data flow diagram of a home control system in accordance with this invention.
  • the control module 540 receives input from a variety of sources in order to determine the appropriate commands 541 to communicate to appliances 550, based upon a determination 591 of the location of one or more trackers 510.
  • a controller (control module 540) is a device that acts as a host for a controlled device (appliance 550).
  • the controller hosts the abstract representation for the controlled device.
  • the control interface is exposed via the API of the abstract representation. This API is the access point for applications to control the device.
  • the controller receives commands and information via the API, and determines an appropriate device command.
  • the controlled device assumes a controlled state in response to receipt of the determined command.
  • each locator 590 determines the identity and mode of each tracker 510 within its vicinity.
  • the mode of the tracker 510 includes, for example, a manual control mode, a standby mode, and an automated mode. In the manual control mode, no automated actions are taken by the control module 540.
  • control module 540 determines the appropriate actions to be taken based upon suggestions 561 from one or more user task modules 560, as well as the defined rules 530, the context 570, and the configuration 520 of the network 50, as discussed above.
  • the control module 540 communicates a query 546 to the user task module 560, requesting one or more suggestions based upon the location of a particular user.
  • the user task module 560 communicates zero or more suggested actions 561 to the control module 540.
  • the control module 540 determines the desired actions, if any, and communicates the appropriate commands 541 to appliances 550 to effect these actions.
  • each appliance 550 communicates its status 551 to the network 50, and in particular to the control module 540, as illustrated by the dashed line in FIG. 2.
  • the control module 540 uses this communicated status to verify that the desired actions have been effected, and to determine any subsequent adjustments to the appliances, such as those performed directly by the user. If the control module 540 includes a knowledge base or learning techniques, these subsequent adjustments will be used to modify the decision criteria used to determine subsequent automated actions, as discussed above.
  • the control module actions and any user controlled actions or adjustments, and the times that each occur, are also communicated to the user task module 560 as history updates 548.
  • the history of activities contained in each user task module 560 may be used by control modules and user task modules throughout the network to refine their suggestion and decision processes based on each user's actions at varying times and locations.
  • FIG. 3 illustrates an example flow chart for the control module 540 of FIG. 2 in a home control system in accordance with this invention.
  • the location and mode data associated with each user is updated.
  • the control module 540 may contain a data base that is continually updated as each locator module 590 reports an arrival or departure of a user/tracker 510.
  • the control module 540 may query each locator module 590 in the network 50 as required. If the mode of the tracker 510 that is associated with the user has changed, at 615, the status of the user is updated, at 620.
  • the tracker 510 may include a multitude of modes, including the aforementioned manual, standby, and automated modes, as well as other user selectable modes.
  • the tracker 510 may be configured to allow the user to specify a particular mood, such as active, quiet, romantic, and so on.
  • Not all control modules will contain the ability to process the variety of tracker modes; the block 620 effects a transformation from the possible tracker modes to the states that are supported by the particular control module 540.
  • the control module 540 queries the user task modules 560, at 630 and 640, for suggestions as to the preferred actions to be taken in response to this arrival or departure. Additionally, if the context 570 has changed, at 645, the user task modules 560 are again queried for suggested actions, at 650. In a preferred embodiment, all rules and user preferences are processed to identify time dependencies, and the context 570 is deemed to change at the time of each of the identified dependencies. The context 570 is also deemed to change whenever a external sensor, such as a light or motion sensor, reports a change.
  • a external sensor such as a light or motion sensor
  • the control module 540 After receiving the user specific suggestions from the user task modules 560, the control module 540 determines the appropriate actions to be taken, at 660, based upon these suggestions, as well as the context and rules associated with this user, as well as the status and location of this user and other users, as discussed above. If, at 665, the decided actions are applicable to appliances 550 within the immediate control of this control module 540, the actions are transformed into appliance commands to effect the actions, at 670, and communicated to the controlled appliances, at 672. If, at 665, the control module 540 is in a secondary control role, such as when used as a redundant controller in a distributed control system, the control module 540 communicates its decided action to each other control module that is in direct control of each of the affected appliances.
  • the configuration module 312 may redefine the functions available to the control modules as other appliances or capabilities are added to the network.
  • the rules for controlling appliances need not be restricted to the arrival or departure of tracker's in each area.
  • one of the appliances in the home network 50 may be a motion detector, and another appliance an alarm device.
  • a control module in the home network 50 can be configured to activate the alarm device if the motion detector reports motion in a particular area, and a locator module does not report the presence of a tracker in that same area.
  • the operation of a control system in accordance with this invention may be coupled to or integrated with other automated systems. For example, copending U.S.
  • the tracker 10 may be configured to communicate its location directly to the network 50, using for example a GPS or other location determining technology.
  • the functions of the control module and user task module may be integrated into a single module.
  • the principles of this invention may be embodied in hardware, software, or a combination of both.
  • the user task modules may be software applications and memory devices, while the control module may be implemented as special purpose hardware components.

Abstract

Devices within a locale, such as a home, are controlled by detecting the presence of an identified user within different areas of the locale. The devices within the areas are controlled in response to each identified user's preferences. The locating and control devices may be stand-alone devices, or integrated within other electronic devices, such as televisions, stereos, computers, and so on. Also provided in this invention are user task modules that suggest control actions based on the location of the user, the current context, and a profile of the user based upon the user's prior actions. The determination of each user's location is facilitated by the use of a tracker module that the user carries about. The user may determine the degree of automation to be applied at any time.

Description

Home control system with distributed, network devices.
This invention relates to the field of consumer devices, and in particular to the field of automated home control systems.
Home automation is becoming increasingly popular. Standards continue to be developed which will allow devices of varying types and varying vendors to be controlled by a common controller. Such standards include IEEE 1394, X-10, HANi, HomeAPI, Jini, and the like. IEEE 1394 and X-10 are communication protocols; HANi is a software architecture using IEEE 1394; Home API is an open industry specification that defines a standard set of software services and application programming interfaces which enable software applications to monitor and control home device. Jini is a distributed software architecture (network) wherein clients see devices and services as objects. Jini has a discovery service to enable a device or service to register itself with the network. Jini has a look-up service to enable determining what objects are available. Coherence among the distributed objects is enabled through leasing of objects, i.e., providing control over an object to another object for a predetermined time period. If the system fails, the system is brought back upon termination of the lease to a well defined state.
Typical home automation systems are configured to provide a central control station and a number of remote controllers. For example, the central control station may be a home computer, and the remote controllers may be sub-controllers located in particular areas of the home, such as in a master bedroom, entry foyer, and the like. Typical home automation systems may also include remote sensors that are used, for example, to automatically turn lights on or off when motion is detected, or to turn a television set on or off in response to a particular sound or voice command. Some home automation systems allow the desired operations to be preprogrammed, so that, for example, lights or appliances are turned on or off at different preset times, televisions are tuned to different channels at different times, and so on.
The preprogramming of a typical home automation system, however, becomes infeasible or impractical except for very routine and commonplace operations. This infeasibility is particularly apparent in a home with multiple occupants. The automated turning on and tuning of a television at a particular time of day is only feasible if all occupants of the home agree, or if it is known a priori that the person who will be in the room with the television at that time would want the television turned on and tuned to the preprogrammed channel. Even a single occupant may find preprogramming infeasible in a multi-room home, particularly if that occupant does not follow a well defined regiment of behavior. That is, for example, the occupant may have a favorite television program, but may view it in different rooms depending upon where the occupant happens to be at the time the program begins. As mentioned above, a preprogrammed home automation system typically utilizes a central controller to effect the preprogrammed actions. As such, the automated control is directly dependent upon the reliability and availability of the central controller. The central controller in most home automation systems is typically the home computer, which is often used for other applications as well. When used as the home automation controller, care must be taken to assure that the computer remains in operation and that the controller application continues to run independent of the other applications. A "crash" of the computer or the controller application will typically disable a significant portion, if not all, of the home automation system. Alternatively, a dedicated central controller is often utilized as the central controller, eliminating the likelihood of a crash caused by another application, but the options available on such dedicated controllers are typically limited compared to those available on a general purpose home computer.
It is an object of this invention to provide an infrastructure for the use of control modules that are distributed in a network of devices. It is a further object of this invention to provide a system and method to facilitate the automated control of devices within a home environment. It is another object of this invention to provide a system and method for creating a profile of use for each user of the system, and controlling the consumer electronic devices based on this profile. It is a further object of this invention to increase the reliability and versatility of home control systems. It is a further object of this invention to allow for an easy system configuration and setup.
These objects and others are achieved by providing a distributed network of locator devices and control devices throughout a locale, such as a home. Devices within the locale are controlled by detecting the presence of an identified user within different areas of the locale, and coordinating the control of appliances within the area in response to this user's presence or absence. The locating and control devices may be stand-alone devices, or integrated within other electronic devices, such as televisions, stereos, computers, and so on. Also provided in this invention are user task modules that suggest control actions based on the location of the user, the current context, and a profile of the user based upon the user's prior actions. Communications among the modules and devices are effected using standard network systems and protocols, including IEEE 1394, X10, HANi, and HomeAPI. The determination of each user's location is facilitated by the use of a tracker module that the user carries about. The user may determine the degree of automation to be applied at any time.
The invention is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:
FIG. 1 is an example block diagram of a home control system in accordance with this invention.
FIG. 2 is an example data flow diagram of a home control system in accordance with this invention. FIG. 3 is an example flow chart for a control module in a home control system in accordance with this invention.
FIG. 1 illustrates an example block diagram of a home control system in accordance with this invention. Illustrated in FIG. 1 are four rooms, a family room 100, a kitchen 200, an office 300, and a utility room 400. In each of these rooms are an assortment of devices; in the vocabulary of home control systems, devices that perform a function other than control are termed appliances. Appliances may include control functions, but their "primary" appliance function is not control, per se. In the family room 100 are located a web-tv appliance 110, a television appliance 120, a stereo appliance 130, and light appliances 140. The example web-tv appliance 110 has an appliance function of providing web-access via a television, and also includes a control module CM 114, a user task module UTM 116, and a locator module LM 119. Another control module CM 134 is located in the example stereo appliance 130, and another locator module, LM 190 is located in the family room 100. The illustrated devices and modules are interconnected via a home network 50. To avoid unnecessary clutter, the example connections to the network 50 are illustrated by "X" - labeled terminators. In a preferred embodiment, the communications via the network are in accordance with existing standards. Using the HANi architecture as an example, a HANi-compliant device has, as a minimum, enough functionality to communicate with other devices in the system. During interaction, devices may exchange control and data in a peer-to-peer fashion. This ensures that at the communication level, none of the devices is required to act as the master or controller of the system. On the other hand, it allows a logical master or controller to impose a control structure on the basic peer-to-peer communication model. In like manner, copending US application "Computer-controlled Home Theater With Independent User-control", serial number
08/920,751, filed 8/29/97 for Jeroen Heuvelman et al., and incorporated herein by reference, presents a home entertainment system comprising a sub-system with audio/video functionalities, a computer and a device interface that interconnects the sub-system and the computer. The computer controls the sub-system through a software application The computer and the sub-system have separate user-controls. In case the computer crashes, the user can still control those audio/video functionalities of the sub-system that are independent of the computer. Additionally, copending application "Low Data-Rate Network Representation on High Data-Rate HAVI-Network", attorney docket PHA 23,492, serial number 09/146,020, filed 9/2/98 for Yevgeny Shteyn, presents principles of a home network using the HAVi architecture, and is incorporated by reference herein. Although the paradigm of a home network is used herein, it will be evident to one of ordinary skill in the art that the principles of this invention may be employed in other environments as well, such as hospitals, offices, factories, and the like.
Illustrated in FIG. 1 is a tracker 10. In accordance with this invention, each occupant of the home possesses a tracker 10 which is configured to enable the remote identification of the associated occupant. By convention, the location of the tracker 10 is assumed to coincide with the location of the associated occupant, or "user"; the tracker 10, for example, may be a small token that the user carries or wears. When the tracker 10 enters the family room 100, the locators 119 and 190 are configured to recognize it; in so doing, the locator modules 119 and 190 enables the identification of the associated occupant as being located in the family room 100. As will be detailed further, the trackers and locator modules are coupled via a wireless connection, thereby allowing the user to move about freely. The home network 50 may be wired or wireless, or a combination of both. The locating modules 119, 190 communicate the presence of the user associated with tracker 10 to the home network 50; in particular, to control modules 114, 134, and others. In response, the control modules query one or more of the user task modules, including UTM 116 and others, for an appropriate control response. For example, if the user that is associated with tracker 10 normally prefers to hear classical music in the afternoon, user task module 116 will communicate a suggestion to control module 134 to play classical music, if the time of day is the afternoon; if the time of day is morning, the user task module 116 may send a suggestion to the control module 134 to tune to the local news-radio station. In response to one or more suggestions from user task module 116 and others, and in response to other potential rules, the control module 134 communicates the appropriate commands, if any, to the appropriate appliances to effect a chosen response from among the suggestions and rules. For example, one of the rules may be to preclude a change of mode of an appliance in the family room 100 if another user (as indicated by the presence of another tracker) is in the family room 100. As would be evident to one of ordinary skill in the art, priority schemes can be used to determine actions to be taken if multiple trackers are in the same area. In a straightforward embodiment, non-conflicting actions are effected, such as turning lights on if at least one of the users have indicated this preference and the other user has not explicitly expressed an opposite preference, whereas conflicting actions, such as turning a television and a stereo on, are not taken unless one of the users has been given an explicit priority of actions over the other. In a preferred embodiment, each rule or preference for a user is assigned a priority. For example, a priority can be established for a particular user for turning on a light whenever the user enters an area, regardless of conflicts. A lower priority for that same user can be established for turning on a television only if no conflicts exists.
In a preferred embodiment, the control modules and user task modules employ knowledge based and machine learning techniques, and others, to determine an appropriate response upon each user's arrival to, or departure from, particular areas of the home. The response is based on prior experiences and feedback from each user, as well as the user's recent history. For example, the response may differ depending upon whether the user arrives in the family room 100 after having been in the kitchen 200, or the office 300. These determinations will also be dependent upon the current context, such as the time of day, the day of the week, and other factors that are determined to be correlated to the user's past behavior, or assumed typical behavior. In a preferred embodiment, the learning techniques are multi-variate in nature, so that a synergy of control is achieved. For example, a learned response may be to lower curtains, dim lights, play soft music, and place the telephone in autoanswer mode in response to a particular set of user patterns, and to keep the area well lit and turn on the radio in response to a different set of user patterns. If a user disagrees with the selected response, the user will typically effect an alternative control response. For example, if the control module 134 turns the stereo 130 on, and the user disagrees with this response, the user will typically turn the stereo 130 off, or effect some other change. In a preferred embodiment, the user responses that immediately follow a control module's execution of a command to an appliance are weighted heavily for training, to effect a potentially different selection by the control module in the future. Subsequent user selections are weighted less for training as the time interval between the control module's action and the user selection increases. For example, if the user listens to the classical music that the control module 134 selects for an hour or so, and then selects popular music, the selection of popular music may be processed by the control module 134 as an independent user preference, or as a sequential preference (play classics for one hour, then switch to popular), or as a disapproval of the control module's selection of classical music, or a combination of the three rationales.
Conventional machine learning techniques, including Bayesian networks, fuzzy logic, and the like are used to modify or reaffirm the control module's selections, so as to improve the likelihood of the control modules and user task modules anticipating the user's desires, based upon an identification of the user within a given area. In a preferred embodiment, user specific actions affect the user task modules, whereas generally applicable experiences affect the control modules. One or more locator modules, control modules, and user task modules may be associated with the same location, or with the control of the same device. In this manner, the reliability of the system is enhanced, by reducing the dependencies on any one device. Techniques are commonly known in the art for resolving conflicts which may arise due to such a distributed control architecture. In a straightforward embodiment, each control module is allocated a position in a control hierarchy with respect to each appliance. Each highest ranking control module that is currently available controls each appliance. In like manner, the suggestions from each user task module and from among the user task modules are given decision weights relative to each appliance, to ease the decision process of each control module. The example kitchen 200 of FIG. 1 includes a control unit 210, a television appliance 220, speaker appliances 230, and a microwave appliance 240. The example control unit 210 is illustrated as containing a control module 214, a user task module 216, and a locator module 219, while the appliances are devoid of control devices. When the tracker 10 enters the kitchen 200, the locator module 219 notifies the appropriate devices in the home network 50, including the control module 214. The control module 214 accesses user task modules within the network 50, including the user task module 216, to determine if any actions are suggested, and accesses its internal rules to determine an action. For example, if the tracker 10 had previously been in the family room 100, and the television 120 in the family room 100 had been tuned to a particular channel, the internal rules of control module 214 would suggest turning on the television 220 in the kitchen, and tuning it to the same channel as the television 120. In like manner, in this example, the internal rules of control module 114 will suggest that television 120 be turned off, if no other users are detected in the family room 100. Note that the above sequences of suggestions are based on rules that are independent of the particular user that departs the family room 100 and enters the kitchen 200. These user- independent suggestions and rules are combined with any user-specific rules at the control module that is responsible for controlling the particular appliances. For example, a particular user may prefer to listen to music while in the kitchen 200, and would not desire to have the television 220 turned on, even though television 120 had been turned on by or for this user. The control module that is configured to control the television 220 and the control module that is configured to control the speaker appliances 230 apply these user-specific rules and preferences and will not turn the television 220 on, and will turn the speakers 230 on; if appropriate, based on the user specific rules and current contexts, the stereo 130 will also be tuned to provide the appropriate selection of music to the speakers 230. The aforementioned rules are structured so as to anticipate a user's desire, based on experience or default assumptions. For example, a default assumption would be that when a user enters an area at a particular time of day, the lights 140, 340 should be turned on, and when room is unoccupied, the lights 140, 340 and entertainment appliances 120, 130, 220, 230, 320, 330 are turned off. The default assumptions may effect sequential operations. For example, the control module 114 can be structured to wait for a predetermined time period before turning off the television 120 when the user departs the family room 100, to avoid an annoying shut-down, start-up sequence every time the user moves about the areas. In like manner, the control modules may determine the appropriate action based upon a user's recent activities. For example, in a preferred embodiment, if the user is determined to be traveling through rooms, without a significant pause within each room, the controller effects some actions, such as turning on lights, but not others, such as turning on stereo sets or televisions. In this example, the default assumption is that the user would prefer to have light for movement through adjoining rooms, but would not prefer to have other appliances turned on and off during each transit through a room. In a preferred embodiment, the user task modules and control modules are preprogrammed with default rules that are subsequently updated based on user experience, or direct user modification.
The rules may also be coupled to other control devices as well. For example, as is common in the art, the microwave appliance 240 may include a programmable device that allows a user to preprogram the microwave to start at a particular time, for a particular duration, and so on. An alternative embodiment of the microwave appliance 240 could allow the user to preprogram the start based on the return of a person to the home, rather than on a preprogrammed time. The control module 214 in this example signals the microwave appliance 240 when the tracker that is associated with the specified person is next detected within the home. As would be evident to one of ordinary skill in the art, the start could also be triggered by other events as well, such as the entry of the person to a particular room within a particular time period.
Illustrated in FIG. 1 is a configuration module 312 in a computer 310 in the home office 300. The configuration module 312 enables the system to be customized for each user directly. The configuration module 312 is also used to enable a user or installer to record the system configuration in terms of the location of modules and appliances throughout the house, and coordinates the allocation of controlled appliances to control modules. In accordance with this invention, the configuration module 312 is also used to enable an automatic or manually initiated download of the rules and preferences to the control modules and user task modules as required. The knowledge base and machine learning functions are preferably effected in the user task modules 116, 316, 416 that are distributed about the network 50, while the configuration module 312 effects actions required to coordinate these activities. In a preferred embodiment, for example, the configuration module allocates responsibilities to balance the computational load as devices are added to or removed from the network. In like manner, the configuration module dynamically allocates responsibilities for appliances among the control modules, and responsibilities for users among the user task modules. The configuration module 312 also responds to queries by the individual modules regarding configuration status and information. In a preferred embodiment, the rules regarding the operation of the external control modules 114, 134, 214, and 414 are configured so as not to be dependent upon the continuos operation of the computer 310 or the configuration module 312. In this manner, the reliability of the system in accordance with this invention is not dependent upon the reliability and availability of a central controller, such as computer 310. The overall system reliability may also be enhanced by utilizing the principles of copending US application "Slave DSP Reboots Stalled Master CPU", serial number 08/880,387, filed 6/23/97 for Paul Foster et al., and herein incorporated by reference. This document relates to a digital home entertainment system comprising one or more slave processors, for example, digital signal processors (DSPs), for processing specific tasks, and a master processor, for example, a central processing unit (CPU), for control of the system. The slave processor is capable of rebooting the master processor if the master processor has stalled. This slave- controlled rebooting avoids manual cold rebooting of the system and is particularly advantageous in open-architecture multimedia systems with asynchronously cooperating components.
The user task modules 116, 316, 416 may each be designed to process particular information. In the example configuration of FIG. 1, the user task module 116 in the web-tv appliance 110 may be designed to specifically facilitate the optimization of each user's access to the web. For example, copending U.S. patent application "Customized Upgrading of Internet-Enabled Devices Based on User-Profile", attorney docket PHA 23,500, serial number 09/160,490, filed 9/25/98 for Adrian Turner et al, and incorporated by reference herein, presents a method of retrieving information from the Internet based upon a user profile. In an example embodiment, the locator module 119 notifies the other modules on the network of the presence of the user associated with tracker 10. In response to this notification, the control module 114 accesses the Internet and retrieves the information based on this user profile in accordance with the above mentioned copending application. The user task module 416 in the environmental control appliance 410, on the other hand, may be designed to facilitate the optimization of the home environment in response to user preferences and experiences. The user task module 416 may know nothing about web services, and the user task module 116 may know nothing about environmental services. That is, each user task module's function can be specialized to a particular knowledge domain. The user task module 316, on the other hand, being located in a computer 310, may be designed to handle all available services. The resolution of conflicting suggestions from multiple task modules can be resolved using the aforementioned priority scheme, wherein each task module has a ranking of control, and the suggestions from the currently highest ranking module is given priority. Alternatively, in a preferred embodiment, the user task modules communicate among themselves and consistent rules are formulated. In the example embodiment of FIG. 1, the user task module 316 in the computer 310 periodically gathers information from each of the other user task modules and attempts to resolve conflicts using conventional decision theory techniques, such as voting, prioritize voting, and the like.
The level of complexity of each module can vary, and less capable devices can rely upon other devices to varying degrees. For example, the aforementioned web-optimized user task module 116 may include an ability to store an activity history corresponding to the use of the television 120 and stereo 130, and provide suggestions to the control modules 114, 134 based on these histories and a primitive level of machine learning. Periodically, the user task module 316 obtains this activity history from the user task module 116, develops more sophisticated rules and suggestions based upon these histories, and communicates these new rules and suggestions to other modules throughout the system as appropriate.
Differing levels of control may be included in the home automation system in accordance with this invention. For example, illustrated in FIG. 1 is a tracker 11 that is associated with a different user than the one associated with tracker 10. When the tracker 11 enters the home office 300, the locator module 390 detects the presence of tracker 11, and notifies the network 50. In response to this notification, the computer 310 and the environment control 410 effect different control. The device command module 414 in the environment control 410 may merely send a signal to an environment network 405, such as a furnace or air conditioning system, to adjust the level of the temperature in the office 300, based on the data in the user task module 416 corresponding to the user associated with tracker 11. The user task module 416 in this example may merely contain the last temperature that this user selected, with no knowledge based or learning machine capabilities. The control module 314 in the computer 310, on the other hand, may effect a number of complex actions, such as establishing contact to an external network 305 to retrieve the user's e-mail, initiating one or more of the applications, such as a word processor, that the user typically uses, reciting the received e-mail messages via the speakers 330, accepting voice commands from the user, and so on. These actions will have been determined based on the user's prior history, the user's explicit commands, or a set of default options, as discussed above. The trackers 10, 11 and locator modules 119, 190, 290, 390 are designed to operate together so as to determine the relative or absolute location of each tracker 10 in the vicinity of each module. Any number of available technologies may be employed to facilitate this determination. In a preferred embodiment, the tracker emits a signal that is only receivable by locator modules within its vicinity, using for example infrared signaling. In an alternative embodiment, the locator module emits a radio frequency (RF) trigger signal, and the trackers that are within the vicinity of the locator module reflect a modulated form of the RF trigger signal back to the locator modules. These and other methods of remotely determining the presence of an object within an area or region are common in the art.
The tracker may be of varying complexity. In a single-user application, the tracker need only be a device that is detectable by a locator module. In a multiple-user application, each tracker includes a means of communicating a unique identifier. In a preferred implementation, the tracker also includes user selectable options, including the options to turn the tracker off, or to enter a manual mode, wherein the locator modules continue to determine the location of the tracker, but some or all of the automated actions by the control modules are inhibited. The tracker may also be configured with a means for the user to communicate commands directly. Copending application "Remote Control Device with Location Dependent
Interface", attorney docket PHA 23,522, serial number , filed 12/11/98 for Joost
Kemink, and incorporated herein by reference, presents a method and apparatus for providing the user of the apparatus differing control capabilities in dependence upon where the user is located. The tracker of this invention may include some or all of the capabilities described in this copending application, such that, for example, when the user selects a manual mode, the tracker provides user options based upon the appliances within the user's immediate area. FIG. 2 illustrates an example data flow diagram of a home control system in accordance with this invention. The control module 540 receives input from a variety of sources in order to determine the appropriate commands 541 to communicate to appliances 550, based upon a determination 591 of the location of one or more trackers 510. In the HAVi environment, for example, a controller (control module 540) is a device that acts as a host for a controlled device (appliance 550). The controller hosts the abstract representation for the controlled device. The control interface is exposed via the API of the abstract representation. This API is the access point for applications to control the device. The controller receives commands and information via the API, and determines an appropriate device command. The controlled device assumes a controlled state in response to receipt of the determined command. In a preferred embodiment, each locator 590 determines the identity and mode of each tracker 510 within its vicinity. The mode of the tracker 510 includes, for example, a manual control mode, a standby mode, and an automated mode. In the manual control mode, no automated actions are taken by the control module 540. In the standby mode, only primitive actions, such as the turning on of lights when the tracker is detected in each area, are effected by the control module 540. In the automated mode, all determinable actions are effected by the control module 540. The control module 540 determines the appropriate actions to be taken based upon suggestions 561 from one or more user task modules 560, as well as the defined rules 530, the context 570, and the configuration 520 of the network 50, as discussed above. In a preferred embodiment, the control module 540 communicates a query 546 to the user task module 560, requesting one or more suggestions based upon the location of a particular user. In response to this query, the user task module 560 communicates zero or more suggested actions 561 to the control module 540. The control module 540 determines the desired actions, if any, and communicates the appropriate commands 541 to appliances 550 to effect these actions. In a preferred embodiment, each appliance 550 communicates its status 551 to the network 50, and in particular to the control module 540, as illustrated by the dashed line in FIG. 2. The control module 540 uses this communicated status to verify that the desired actions have been effected, and to determine any subsequent adjustments to the appliances, such as those performed directly by the user. If the control module 540 includes a knowledge base or learning techniques, these subsequent adjustments will be used to modify the decision criteria used to determine subsequent automated actions, as discussed above. The control module actions and any user controlled actions or adjustments, and the times that each occur, are also communicated to the user task module 560 as history updates 548. The history of activities contained in each user task module 560 may be used by control modules and user task modules throughout the network to refine their suggestion and decision processes based on each user's actions at varying times and locations.
For completeness, FIG. 3 illustrates an example flow chart for the control module 540 of FIG. 2 in a home control system in accordance with this invention. At block 610, the location and mode data associated with each user is updated. A variety of techniques may be employed to effect this update. For example, the control module 540 may contain a data base that is continually updated as each locator module 590 reports an arrival or departure of a user/tracker 510. Alternatively, the control module 540 may query each locator module 590 in the network 50 as required. If the mode of the tracker 510 that is associated with the user has changed, at 615, the status of the user is updated, at 620. In general, the tracker 510 may include a multitude of modes, including the aforementioned manual, standby, and automated modes, as well as other user selectable modes. For example, the tracker 510 may be configured to allow the user to specify a particular mood, such as active, quiet, romantic, and so on. Not all control modules will contain the ability to process the variety of tracker modes; the block 620 effects a transformation from the possible tracker modes to the states that are supported by the particular control module 540.
If the identified user is either newly arrived 625 or newly departed 635 to or from the reported location, the control module 540 queries the user task modules 560, at 630 and 640, for suggestions as to the preferred actions to be taken in response to this arrival or departure. Additionally, if the context 570 has changed, at 645, the user task modules 560 are again queried for suggested actions, at 650. In a preferred embodiment, all rules and user preferences are processed to identify time dependencies, and the context 570 is deemed to change at the time of each of the identified dependencies. The context 570 is also deemed to change whenever a external sensor, such as a light or motion sensor, reports a change. After receiving the user specific suggestions from the user task modules 560, the control module 540 determines the appropriate actions to be taken, at 660, based upon these suggestions, as well as the context and rules associated with this user, as well as the status and location of this user and other users, as discussed above. If, at 665, the decided actions are applicable to appliances 550 within the immediate control of this control module 540, the actions are transformed into appliance commands to effect the actions, at 670, and communicated to the controlled appliances, at 672. If, at 665, the control module 540 is in a secondary control role, such as when used as a redundant controller in a distributed control system, the control module 540 communicates its decided action to each other control module that is in direct control of each of the affected appliances. As discussed above, techniques for resolving disputes among distributed controllers are common in the art, as are protocols for assigning direct control to secondary controllers when it is determined that a primary controller is inoperative. After communicating the decided actions or corresponding appliance commands, the history data is updated, at 690, as discussed above. This process is continually repeated, as indicated by the loop from block 690 back to block 610 in FIG. 3.
The foregoing merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are thus within its spirit and scope. For example, copending U.S. patent application "Upgrading of Synergetic Aspects of Home Networks", attorney docket PHA23,527, serial number 09/189,535, filed 11/10/98 for Yevgeny Shteyn, and incorporated by reference herein, presents a method of enhancing the synergy of the appliances that are located on the network, based on the inventory of appliances on the network. Using this copending application, the configuration module 312 may redefine the functions available to the control modules as other appliances or capabilities are added to the network. In like manner, the rules for controlling appliances need not be restricted to the arrival or departure of tracker's in each area. For example, one of the appliances in the home network 50 may be a motion detector, and another appliance an alarm device. A control module in the home network 50 can be configured to activate the alarm device if the motion detector reports motion in a particular area, and a locator module does not report the presence of a tracker in that same area. In like manner, the operation of a control system in accordance with this invention may be coupled to or integrated with other automated systems. For example, copending U.S. patent application "Distributed Software Controlled Theft Detection", attorney docket PHA 23,503, serial number 09/176,171, filed 10/21/98 for Doreen Cheng, and incorporated herein by reference, discloses a method of protecting property within a home or office via a distributed network. This same network may be utilized to provide location dependent security and control capabilities in accordance with the principles presented herein.
The structure and functional partitions of the home control system of FIGs. 1 and 2 are presented herein for illustrative purposes only. As would be evident to one of ordinary skill in the art, for example, the tracker 10 may be configured to communicate its location directly to the network 50, using for example a GPS or other location determining technology. In like manner, the functions of the control module and user task module may be integrated into a single module. The principles of this invention may be embodied in hardware, software, or a combination of both. For example, the user task modules may be software applications and memory devices, while the control module may be implemented as special purpose hardware components. These alternative configurations and other optimization techniques would be evident to one of ordinary skill in the art in view of the principles presented in this specification.

Claims

CLAIMS:
1. A method for enabling control of an appliance (550), comprising the steps of: enabling a determination of a location of a user (591), enabling a determination of an action in dependence upon the location of the user (591), enabling a determination of an appliance command (541) associated with the action, and, enabling the appliance (550) to assume a controlled state in response to the determined appliance command (541).
2. The method of claim 1, wherein the appliance command (541) is communicated to the appliance (550) via a network.
3. The method of claim 1 , further comprising the step of enabling a selection of the appliance (550) from a plurality of appliances based on the location of the user (591).
4. The method of claim 1, wherein the determination of the action is based upon at least one of: a context (570), a time, a rule, a mode (591), and a history of activities of the user (548).
5. The method of claim 1, further comprising the step of enabling a determination of a suggestion of action (561) based on the location of the user (591), and wherein the determination of the action is further based on the suggestion of action (561).
6. The method of claim 1, further comprising the step of enabling a creation of a set of rules (530) associated with the user, the location of the user (591), and the appliance (550), and wherein the determination of the action is further based on the set of rules
(530).
7. A control module comprising: means for determining a location of a user (591), means for determining an action based upon the location of the user (591), means for determining an appliance (550) and an appliance command (541) based on the action, and means for communicating the appliance command (541) to the appliance (550) to effect the action.
8. The control module of claim 7, further including means for receiving a task suggestion (561) from a user task module (560) based on the location of the user (591), and wherein the means for determining the action is operably coupled to the means for receiving the task suggestion (561) such that the action is further based on the task suggestion (561).
9. The control module of claim 7, further including means for determining rules (530) based on the location of the user (591), and wherein the means for determining the action is operably coupled to the means for determining rules (530) such that the action is further based on the rules (530).
10. The control module of claim 7, wherein the means for determining an action is further dependent upon a knowledge domain associated with the appliance (550).
11. An appliance (110, 130) for use in a control system, the appliance (110, 130) comprising: an appliance component for effecting a primary appliance function; and a control component (114, 134) that determines a control command (541) based upon a location of a user (591) and communicates the control command (541) to a second appliance (120, 140).
12. The appliance (110, 130) of claim 11, wherein the control command (541) is determined based further upon at least one of: a context (570), a time, a rule (530), and a history of user activities (548).
PCT/EP1999/010197 1998-12-29 1999-12-17 Home control system with distributed network devices WO2000039964A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2000591754A JP2002534841A (en) 1998-12-29 1999-12-17 Home control device with distributed network device
EP99965516A EP1057302A1 (en) 1998-12-29 1999-12-17 Home control system with distributed network devices
KR1020007009560A KR20010041425A (en) 1998-12-29 1999-12-17 Home control system with distributed network devices

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US22240398A 1998-12-29 1998-12-29
US09/222,403 1998-12-29

Publications (1)

Publication Number Publication Date
WO2000039964A1 true WO2000039964A1 (en) 2000-07-06

Family

ID=22832035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP1999/010197 WO2000039964A1 (en) 1998-12-29 1999-12-17 Home control system with distributed network devices

Country Status (4)

Country Link
EP (1) EP1057302A1 (en)
JP (1) JP2002534841A (en)
KR (1) KR20010041425A (en)
WO (1) WO2000039964A1 (en)

Cited By (63)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002017566A1 (en) * 2000-08-23 2002-02-28 Koninklijke Philips Electronics N.V. Communication system and device
WO2003058575A1 (en) * 2002-01-08 2003-07-17 Koninklijke Philips Electronics N.V. Controlling application devices simultaneously
WO2003105406A1 (en) * 2002-06-07 2003-12-18 Koninklijke Philips Electronics N.V. System and method for adapting the ambience of a local environment according to the location and personal preferences of people in the local environment
EP1388985A1 (en) * 2002-08-06 2004-02-11 Sony International (Europe) GmbH Method and system for handling e-mail tasks within a HAVi system based home network
EP1458140A1 (en) * 2003-03-12 2004-09-15 Samsung Electronics Co., Ltd. Private network system having a ubiquitous service function and method for operating the same
EP1483597A2 (en) * 2002-02-14 2004-12-08 Avaya Technology Corp. Presence tracking and name space interconnection techniques
EP1496650A1 (en) * 2003-07-07 2005-01-12 Lg Electronics Inc. Control system and method for home network system
EP1471692A3 (en) * 2003-04-16 2005-01-12 Samsung Electronics Co., Ltd. Method and system for constructing and utilizing home-state information in a home network
EP1507361A2 (en) * 2003-08-15 2005-02-16 Microsoft Corporation Context-sensitive remote control
WO2005050923A1 (en) * 2003-11-14 2005-06-02 Resinetics, Inc. Methods, systems and computer program products for providing resident aware home management
WO2005076619A1 (en) * 2004-01-09 2005-08-18 Koninklijke Philips Electronics N.V. Using a presence status in a media-on-demand system
EP1589699A2 (en) * 2004-04-19 2005-10-26 Broadcom Corporation Systems and methods for integrated control within a home entertainment system
EP1606724A2 (en) * 2003-03-07 2005-12-21 Openpeak Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
EP1615445A2 (en) * 2004-07-06 2006-01-11 Thomson Licensing Method of encoding and playing back audiovisual or audio documents and device for implementing the method
JP2006079509A (en) * 2004-09-13 2006-03-23 Kyocera Mita Corp Power saving management device
EP1679828A1 (en) * 2005-01-07 2006-07-12 Samsung Electronics Co., Ltd. Method and system for prioritizing tasks made available by devices in a network
EP1679831A2 (en) 2005-01-10 2006-07-12 Samsung Electronics Co.,Ltd. Contextual task recommendation system and method for determining user's context and suggesting tasks
US7215760B2 (en) 2002-12-04 2007-05-08 Avaya Technology Corp. Outbound dialing decision criteria based
KR100743963B1 (en) 2005-12-08 2007-07-30 한국전자통신연구원 System and Method for Context-Aware Service
WO2007144020A1 (en) * 2006-06-12 2007-12-21 T-Cos Solutions Gmbh Method and modular system for controlling and/or testing of distributed resources
US7519612B2 (en) 2004-07-20 2009-04-14 Sony Corporation Information processing system, information processing method, and computer program used therewith
WO2009153708A1 (en) * 2008-06-18 2009-12-23 Koninklijke Philips Electronics N. V. Programming system and method for programming an interactive system
EP1729517A3 (en) * 2005-05-23 2010-10-13 Sony Corporation Content display-playback system, content display-playback method, and recording medium and operation control apparatus used therewith
WO2011001320A1 (en) 2009-06-30 2011-01-06 Koninklijke Philips Electronics, N.V. Systems and methods for managing interaction with controllable lighting networks
US7933945B2 (en) 2002-06-27 2011-04-26 Openpeak Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
US7987489B2 (en) 2003-01-07 2011-07-26 Openpeak Inc. Legacy device bridge for residential or non-residential networks
US8028283B2 (en) 2006-03-20 2011-09-27 Samsung Electronics Co., Ltd. Method and system for automated invocation of device functionalities in a network
US8056077B2 (en) 2006-05-16 2011-11-08 Samsung Electronics Co., Ltd. Apparatus and method for performing a task
US8099313B2 (en) 2004-09-22 2012-01-17 Samsung Electronics Co., Ltd. Method and system for the orchestration of tasks on consumer electronics
US8185427B2 (en) 2004-09-22 2012-05-22 Samsung Electronics Co., Ltd. Method and system for presenting user tasks for the control of electronic devices
US8196064B2 (en) 2002-06-27 2012-06-05 Id8 Group R2 Studios, Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
US8205013B2 (en) 2005-05-02 2012-06-19 Samsung Electronics Co., Ltd. Method and system for aggregating the control of middleware control points
WO2013020970A1 (en) * 2011-08-08 2013-02-14 tado GmbH User status- and user behavior-based control system and method for building-installation systems and components
US8412554B2 (en) 2004-09-24 2013-04-02 Samsung Electronics Co., Ltd. Method and system for describing consumer electronics using separate task and device descriptions
WO2014060920A1 (en) * 2012-10-17 2014-04-24 Koninklijke Philips N.V. Granting control of a shared system
US8842818B2 (en) 2006-06-30 2014-09-23 Avaya Inc. IP telephony architecture including information storage and retrieval system to track fluency
EP2735123A4 (en) * 2011-07-19 2015-03-04 Nokia Corp Method, apparatus, and computer program product for recommending interaction between devices in a local environment
CN104981963A (en) * 2013-02-05 2015-10-14 三菱电机株式会社 Home-use power command device and home-use power control system
CN105518652A (en) * 2013-09-06 2016-04-20 慧与发展有限责任合伙企业 Managing a sensory factor
WO2016064128A1 (en) * 2014-10-24 2016-04-28 Samsung Electronics Co., Ltd. Control device, method of controlling the same, and integrated control system
US9398152B2 (en) 2004-02-25 2016-07-19 Avaya Inc. Using business rules for determining presence
US9704113B2 (en) 2014-05-02 2017-07-11 Qualcomm Incorporated Method and apparatus for setting user preferences or device configuration
WO2017146402A1 (en) 2016-02-24 2017-08-31 Samsung Electronics Co., Ltd. Server and method of controlling user environment by server
US9860076B2 (en) 2014-05-07 2018-01-02 Vivint, Inc. Home automation via voice control
EP3297218A1 (en) * 2012-08-28 2018-03-21 Delos Living, LLC Environmental control system and method of operation such system
US9948476B2 (en) 2014-11-21 2018-04-17 Facebook, Inc. Techniques to associate user data with a mobile device
EP3373525A3 (en) * 2017-03-08 2018-09-19 LG Electronics Inc. Electronic device and method using machine learning for identifying characteristics of users located within a specific space
WO2019055090A1 (en) * 2017-09-18 2019-03-21 Microsoft Technology Licensing, Llc Notifications to all devices to update state
US10353576B2 (en) 2016-06-12 2019-07-16 Apple Inc. User interface for managing controllable external devices
US10477640B2 (en) 2009-10-08 2019-11-12 Delos Living Llc LED lighting system
EP3024199B1 (en) * 2014-11-21 2019-12-11 Facebook, Inc. Method, storage media, system and program product for associating user data with a mobile device
US10599116B2 (en) 2014-02-28 2020-03-24 Delos Living Llc Methods for enhancing wellness associated with habitable environments
US10671600B1 (en) 2007-07-24 2020-06-02 Avaya Inc. Communications-enabled dynamic social network routing utilizing presence
US10779085B1 (en) 2019-05-31 2020-09-15 Apple Inc. User interfaces for managing controllable external devices
US10820058B2 (en) 2018-05-07 2020-10-27 Apple Inc. User interfaces for viewing live video feeds and recorded video
US10923226B2 (en) 2015-01-13 2021-02-16 Delos Living Llc Systems, methods and articles for monitoring and enhancing human wellness
US11079913B1 (en) 2020-05-11 2021-08-03 Apple Inc. User interface for status indicators
US11338107B2 (en) 2016-08-24 2022-05-24 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
US11363071B2 (en) 2019-05-31 2022-06-14 Apple Inc. User interfaces for managing a local network
US11649977B2 (en) 2018-09-14 2023-05-16 Delos Living Llc Systems and methods for air remediation
US11668481B2 (en) 2017-08-30 2023-06-06 Delos Living Llc Systems, methods and articles for assessing and/or improving health and well-being
US11844163B2 (en) 2019-02-26 2023-12-12 Delos Living Llc Method and apparatus for lighting in an office environment
US11898898B2 (en) 2019-03-25 2024-02-13 Delos Living Llc Systems and methods for acoustic monitoring

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002281574A (en) * 2001-03-15 2002-09-27 Sharp Corp Equipment control method, equipment controller, operating situation acquiring device, equipment control system, program and computer-readable recording medium recorded with program
KR100437796B1 (en) * 2002-04-01 2004-06-30 엘지전자 주식회사 united control kit of home network system
JP4470650B2 (en) * 2004-08-30 2010-06-02 パナソニック株式会社 Network control terminal equipment
KR100754521B1 (en) * 2005-02-22 2007-09-03 삼성전자주식회사 Home network system and method for providing information therein
JP2006345139A (en) * 2005-06-08 2006-12-21 Matsushita Electric Ind Co Ltd Network control information equipment system for home electric appliance
KR101056028B1 (en) * 2005-07-14 2011-08-10 주식회사 케이티 Service management device for knowledge-based home network service
JP4872360B2 (en) * 2006-01-23 2012-02-08 パナソニック電工株式会社 Remote control system
JP4986739B2 (en) * 2007-06-27 2012-07-25 キヤノン株式会社 Content reproduction apparatus and information processing method
KR20100065812A (en) * 2008-12-09 2010-06-17 한국전자통신연구원 Apparatus for operating home network by using building construction drawing and method thereof

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4321304A1 (en) * 1993-06-26 1995-03-02 Domarkas Brigitte System for controlling and/or monitoring devices
US5555376A (en) * 1993-12-03 1996-09-10 Xerox Corporation Method for granting a user request having locational and contextual attributes consistent with user policies for devices having locational attributes consistent with the user request
US5630159A (en) * 1994-12-29 1997-05-13 Motorola, Inc. Method and apparatus for personal attribute selection having delay management method and apparatus for preference establishment when preferences in a donor device are unavailable

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4321304A1 (en) * 1993-06-26 1995-03-02 Domarkas Brigitte System for controlling and/or monitoring devices
US5555376A (en) * 1993-12-03 1996-09-10 Xerox Corporation Method for granting a user request having locational and contextual attributes consistent with user policies for devices having locational attributes consistent with the user request
US5630159A (en) * 1994-12-29 1997-05-13 Motorola, Inc. Method and apparatus for personal attribute selection having delay management method and apparatus for preference establishment when preferences in a donor device are unavailable

Cited By (115)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002017566A1 (en) * 2000-08-23 2002-02-28 Koninklijke Philips Electronics N.V. Communication system and device
WO2003058575A1 (en) * 2002-01-08 2003-07-17 Koninklijke Philips Electronics N.V. Controlling application devices simultaneously
US8966561B2 (en) 2002-01-08 2015-02-24 Koninklijke Philips N.V. Controlling application devices simultaneously
KR101138857B1 (en) 2002-01-08 2012-05-14 코닌클리케 필립스 일렉트로닉스 엔.브이. A method and a systme for controlling application devices, a computer system, and a computer-readable recording medium
EP1483597A2 (en) * 2002-02-14 2004-12-08 Avaya Technology Corp. Presence tracking and name space interconnection techniques
EP1483597A4 (en) * 2002-02-14 2006-03-22 Avaya Technology Corp Presence tracking and name space interconnection techniques
WO2003105406A1 (en) * 2002-06-07 2003-12-18 Koninklijke Philips Electronics N.V. System and method for adapting the ambience of a local environment according to the location and personal preferences of people in the local environment
US8196064B2 (en) 2002-06-27 2012-06-05 Id8 Group R2 Studios, Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
US7933945B2 (en) 2002-06-27 2011-04-26 Openpeak Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
EP1388985A1 (en) * 2002-08-06 2004-02-11 Sony International (Europe) GmbH Method and system for handling e-mail tasks within a HAVi system based home network
US7215760B2 (en) 2002-12-04 2007-05-08 Avaya Technology Corp. Outbound dialing decision criteria based
US9578140B2 (en) 2003-01-07 2017-02-21 Microsoft Technology Licensing, Llc Legacy device bridge for residential or non-residential networks
US8793746B2 (en) 2003-01-07 2014-07-29 Id8 Group R2 Studios, Inc. Legacy device bridge for residential or non-residential networks
US10432756B2 (en) 2003-01-07 2019-10-01 Microsoft Technology Licensing, Llc Legacy device bridge for residential or non-residential networks
US7987489B2 (en) 2003-01-07 2011-07-26 Openpeak Inc. Legacy device bridge for residential or non-residential networks
EP1606724A4 (en) * 2003-03-07 2007-08-22 Openpeak Inc Method, system, and computer program product for managing controlled residential or non-residential environments
EP1606724A2 (en) * 2003-03-07 2005-12-21 Openpeak Inc. Method, system, and computer program product for managing controlled residential or non-residential environments
EP1458140A1 (en) * 2003-03-12 2004-09-15 Samsung Electronics Co., Ltd. Private network system having a ubiquitous service function and method for operating the same
EP1471692A3 (en) * 2003-04-16 2005-01-12 Samsung Electronics Co., Ltd. Method and system for constructing and utilizing home-state information in a home network
CN1316792C (en) * 2003-04-16 2007-05-16 三星电子株式会社 Method and system for constructing and utilizing home-state information in home network
US7366498B2 (en) 2003-07-07 2008-04-29 Lg Electronics Inc. Control system and method for home network system
EP1496650A1 (en) * 2003-07-07 2005-01-12 Lg Electronics Inc. Control system and method for home network system
US7064675B2 (en) 2003-08-15 2006-06-20 Microsoft Corporation Context-sensitive remote controls
EP1507361A3 (en) * 2003-08-15 2005-03-23 Microsoft Corporation Context-sensitive remote control
EP1507361A2 (en) * 2003-08-15 2005-02-16 Microsoft Corporation Context-sensitive remote control
US7492277B2 (en) 2003-08-15 2009-02-17 Microsoft Corporation Context-sensitive remote controls
US7492278B2 (en) 2003-08-15 2009-02-17 Microsoft Corporation Context-sensitive remote controls
WO2005050923A1 (en) * 2003-11-14 2005-06-02 Resinetics, Inc. Methods, systems and computer program products for providing resident aware home management
WO2005076619A1 (en) * 2004-01-09 2005-08-18 Koninklijke Philips Electronics N.V. Using a presence status in a media-on-demand system
US9398152B2 (en) 2004-02-25 2016-07-19 Avaya Inc. Using business rules for determining presence
EP1589699A2 (en) * 2004-04-19 2005-10-26 Broadcom Corporation Systems and methods for integrated control within a home entertainment system
EP1615445A3 (en) * 2004-07-06 2011-07-06 Thomson Licensing Method of encoding and playing back audiovisual or audio documents and device for implementing the method
EP1615445A2 (en) * 2004-07-06 2006-01-11 Thomson Licensing Method of encoding and playing back audiovisual or audio documents and device for implementing the method
US7519612B2 (en) 2004-07-20 2009-04-14 Sony Corporation Information processing system, information processing method, and computer program used therewith
EP1619877B1 (en) * 2004-07-20 2014-01-29 Sony Corporation Information processing system, information processing method, and computer program used therewith
JP2006079509A (en) * 2004-09-13 2006-03-23 Kyocera Mita Corp Power saving management device
US8099313B2 (en) 2004-09-22 2012-01-17 Samsung Electronics Co., Ltd. Method and system for the orchestration of tasks on consumer electronics
US8185427B2 (en) 2004-09-22 2012-05-22 Samsung Electronics Co., Ltd. Method and system for presenting user tasks for the control of electronic devices
US8412554B2 (en) 2004-09-24 2013-04-02 Samsung Electronics Co., Ltd. Method and system for describing consumer electronics using separate task and device descriptions
EP1679828A1 (en) * 2005-01-07 2006-07-12 Samsung Electronics Co., Ltd. Method and system for prioritizing tasks made available by devices in a network
KR100728021B1 (en) 2005-01-07 2007-06-13 삼성전자주식회사 Method and system for prioritizing tasks made available by devices in a network
US8510737B2 (en) 2005-01-07 2013-08-13 Samsung Electronics Co., Ltd. Method and system for prioritizing tasks made available by devices in a network
EP1679831A2 (en) 2005-01-10 2006-07-12 Samsung Electronics Co.,Ltd. Contextual task recommendation system and method for determining user's context and suggesting tasks
US8069422B2 (en) 2005-01-10 2011-11-29 Samsung Electronics, Co., Ltd. Contextual task recommendation system and method for determining user's context and suggesting tasks
KR100754194B1 (en) 2005-01-10 2007-09-03 삼성전자주식회사 Contextual task recommendation system and method for determining user's context and suggesting tasks
EP1679831A3 (en) * 2005-01-10 2007-10-17 Samsung Electronics Co.,Ltd. Contextual task recommendation system and method for determining user's context and suggesting tasks
US8205013B2 (en) 2005-05-02 2012-06-19 Samsung Electronics Co., Ltd. Method and system for aggregating the control of middleware control points
EP1729517A3 (en) * 2005-05-23 2010-10-13 Sony Corporation Content display-playback system, content display-playback method, and recording medium and operation control apparatus used therewith
US8321898B2 (en) 2005-05-23 2012-11-27 Sony Corporation Content display-playback system, content display-playback method, and recording medium and operation control apparatus used therewith
KR100743963B1 (en) 2005-12-08 2007-07-30 한국전자통신연구원 System and Method for Context-Aware Service
US8028283B2 (en) 2006-03-20 2011-09-27 Samsung Electronics Co., Ltd. Method and system for automated invocation of device functionalities in a network
US8056077B2 (en) 2006-05-16 2011-11-08 Samsung Electronics Co., Ltd. Apparatus and method for performing a task
WO2007144020A1 (en) * 2006-06-12 2007-12-21 T-Cos Solutions Gmbh Method and modular system for controlling and/or testing of distributed resources
US8842818B2 (en) 2006-06-30 2014-09-23 Avaya Inc. IP telephony architecture including information storage and retrieval system to track fluency
US10671600B1 (en) 2007-07-24 2020-06-02 Avaya Inc. Communications-enabled dynamic social network routing utilizing presence
WO2009153708A1 (en) * 2008-06-18 2009-12-23 Koninklijke Philips Electronics N. V. Programming system and method for programming an interactive system
WO2011001320A1 (en) 2009-06-30 2011-01-06 Koninklijke Philips Electronics, N.V. Systems and methods for managing interaction with controllable lighting networks
US9775218B2 (en) 2009-06-30 2017-09-26 Philips Lighting Holding B.V. Systems and methods for managing interaction with controllable lighting networks
US9213324B2 (en) 2009-06-30 2015-12-15 Koninklijke Philips N.V. Systems and methods for managing interaction with controllable lighting networks
CN102483614B (en) * 2009-06-30 2015-12-16 皇家飞利浦电子股份有限公司 For managing the mutual system with controllable lighting networks
CN102483614A (en) * 2009-06-30 2012-05-30 皇家飞利浦电子股份有限公司 Systems and methods for managing interaction with controllable lighting networks
US10952297B2 (en) 2009-10-08 2021-03-16 Delos Living Llc LED lighting system and method therefor
US10477640B2 (en) 2009-10-08 2019-11-12 Delos Living Llc LED lighting system
US11109466B2 (en) 2009-10-08 2021-08-31 Delos Living Llc LED lighting system
EP2735123A4 (en) * 2011-07-19 2015-03-04 Nokia Corp Method, apparatus, and computer program product for recommending interaction between devices in a local environment
US9804578B2 (en) 2011-08-08 2017-10-31 tado GmbH User status- and user behavior-based control system and method for building technology systems and components
WO2013020970A1 (en) * 2011-08-08 2013-02-14 tado GmbH User status- and user behavior-based control system and method for building-installation systems and components
US10691148B2 (en) 2012-08-28 2020-06-23 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
US10845829B2 (en) 2012-08-28 2020-11-24 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
EP3297218A1 (en) * 2012-08-28 2018-03-21 Delos Living, LLC Environmental control system and method of operation such system
US10928842B2 (en) 2012-08-28 2021-02-23 Delos Living Llc Systems and methods for enhancing wellness associated with habitable environments
US11587673B2 (en) 2012-08-28 2023-02-21 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
WO2014060920A1 (en) * 2012-10-17 2014-04-24 Koninklijke Philips N.V. Granting control of a shared system
US9947156B2 (en) 2012-10-17 2018-04-17 Philips Lighting Holding B.V. Granting control of a shared system
CN104981963A (en) * 2013-02-05 2015-10-14 三菱电机株式会社 Home-use power command device and home-use power control system
CN104981963B (en) * 2013-02-05 2018-04-27 三菱电机株式会社 Home-use electric power command device and household electricity Force control system
CN105518652A (en) * 2013-09-06 2016-04-20 慧与发展有限责任合伙企业 Managing a sensory factor
EP3042309A4 (en) * 2013-09-06 2017-04-26 Hewlett-Packard Enterprise Development LP Managing a sensory factor
US10599116B2 (en) 2014-02-28 2020-03-24 Delos Living Llc Methods for enhancing wellness associated with habitable environments
US10712722B2 (en) 2014-02-28 2020-07-14 Delos Living Llc Systems and articles for enhancing wellness associated with habitable environments
US11763401B2 (en) 2014-02-28 2023-09-19 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
US9704113B2 (en) 2014-05-02 2017-07-11 Qualcomm Incorporated Method and apparatus for setting user preferences or device configuration
US9860076B2 (en) 2014-05-07 2018-01-02 Vivint, Inc. Home automation via voice control
US10554432B2 (en) 2014-05-07 2020-02-04 Vivint, Inc. Home automation via voice control
WO2016064128A1 (en) * 2014-10-24 2016-04-28 Samsung Electronics Co., Ltd. Control device, method of controlling the same, and integrated control system
US10439838B2 (en) 2014-10-24 2019-10-08 Samsung Electronics Co., Ltd. Control device, method of controlling the same, and integrated control system
US9948476B2 (en) 2014-11-21 2018-04-17 Facebook, Inc. Techniques to associate user data with a mobile device
EP3024199B1 (en) * 2014-11-21 2019-12-11 Facebook, Inc. Method, storage media, system and program product for associating user data with a mobile device
US10923226B2 (en) 2015-01-13 2021-02-16 Delos Living Llc Systems, methods and articles for monitoring and enhancing human wellness
EP3420706A4 (en) * 2016-02-24 2019-04-17 Samsung Electronics Co., Ltd. Server and method of controlling user environment by server
WO2017146402A1 (en) 2016-02-24 2017-08-31 Samsung Electronics Co., Ltd. Server and method of controlling user environment by server
US10809679B2 (en) 2016-02-24 2020-10-20 Samsung Electronics Co., Ltd. Server and method of controlling user environment by server
CN108713314B (en) * 2016-02-24 2021-03-16 三星电子株式会社 Server and method for controlling user environment by server
CN108713314A (en) * 2016-02-24 2018-10-26 三星电子株式会社 Server and method by server controls user environment
US10353576B2 (en) 2016-06-12 2019-07-16 Apple Inc. User interface for managing controllable external devices
US10635303B2 (en) 2016-06-12 2020-04-28 Apple Inc. User interface for managing controllable external devices
US11338107B2 (en) 2016-08-24 2022-05-24 Delos Living Llc Systems, methods and articles for enhancing wellness associated with habitable environments
CN108572729A (en) * 2017-03-08 2018-09-25 Lg 电子株式会社 Use the electronic equipment and method of machine learning
US10464570B2 (en) 2017-03-08 2019-11-05 Lg Electronics Inc. Electronic device and method using machine learning for identifying characteristics of users located within a specific space
EP3373525A3 (en) * 2017-03-08 2018-09-19 LG Electronics Inc. Electronic device and method using machine learning for identifying characteristics of users located within a specific space
US11668481B2 (en) 2017-08-30 2023-06-06 Delos Living Llc Systems, methods and articles for assessing and/or improving health and well-being
WO2019055090A1 (en) * 2017-09-18 2019-03-21 Microsoft Technology Licensing, Llc Notifications to all devices to update state
US11005900B2 (en) 2017-09-18 2021-05-11 Microsoft Technology Licensing, Llc Notifications to all devices to update state
US10820058B2 (en) 2018-05-07 2020-10-27 Apple Inc. User interfaces for viewing live video feeds and recorded video
US10904628B2 (en) 2018-05-07 2021-01-26 Apple Inc. User interfaces for viewing live video feeds and recorded video
US11649977B2 (en) 2018-09-14 2023-05-16 Delos Living Llc Systems and methods for air remediation
US11844163B2 (en) 2019-02-26 2023-12-12 Delos Living Llc Method and apparatus for lighting in an office environment
US11898898B2 (en) 2019-03-25 2024-02-13 Delos Living Llc Systems and methods for acoustic monitoring
US11363071B2 (en) 2019-05-31 2022-06-14 Apple Inc. User interfaces for managing a local network
US10904029B2 (en) 2019-05-31 2021-01-26 Apple Inc. User interfaces for managing controllable external devices
US10779085B1 (en) 2019-05-31 2020-09-15 Apple Inc. User interfaces for managing controllable external devices
US11785387B2 (en) 2019-05-31 2023-10-10 Apple Inc. User interfaces for managing controllable external devices
US11824898B2 (en) 2019-05-31 2023-11-21 Apple Inc. User interfaces for managing a local network
US11513667B2 (en) 2020-05-11 2022-11-29 Apple Inc. User interface for audio message
US11079913B1 (en) 2020-05-11 2021-08-03 Apple Inc. User interface for status indicators

Also Published As

Publication number Publication date
JP2002534841A (en) 2002-10-15
EP1057302A1 (en) 2000-12-06
KR20010041425A (en) 2001-05-15

Similar Documents

Publication Publication Date Title
EP1057302A1 (en) Home control system with distributed network devices
EP1851596B1 (en) Home appliance identification and control in a networked environment
US7627098B2 (en) Intelligent management apparatus and method of digital home network system
US9613526B2 (en) System and method for controlling device location determination
EP2151723B1 (en) Home appliance control system and methods in a networked environment
EP1496650B1 (en) Control system and method for home network system
US7729282B2 (en) Home network system and its configuration system
WO2004061701A1 (en) Apparatus and methods for communication among devices
US10952306B2 (en) Methods, systems, and products for control of electrical loads

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): JP KR SG

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

WWE Wipo information: entry into national phase

Ref document number: 1999965516

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 1020007009560

Country of ref document: KR

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWP Wipo information: published in national office

Ref document number: 1999965516

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1020007009560

Country of ref document: KR

WWW Wipo information: withdrawn in national office

Ref document number: 1999965516

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1020007009560

Country of ref document: KR