BR112015032568B1 - METHOD FOR CONTROLLING AT LEAST ONE COMPONENT DEVICE IN A HOME ENTERTAINMENT SYSTEM - Google Patents

Abstract

SYSTEM AND METHOD FOR USER MONITORING AND DETERMINING INTENTION The present invention relates to sensing interfaces associated with a home entertainment system that are used to automate a system response to events that occur in a display area associated with the system. home entertainment. Data derived from such detection interfaces can also be used to increase the responsiveness of one or more system components. Furthermore, user presence derived from such detection interfaces can be used to capture and report user viewing habits and/or viewing preferences.

Description

RELATED ORDER INFORMATION

[0001] This application claims the benefit of, and is a continuation in part of, US Application No. 13/758,307, filed February 4, 2013, the description of which is incorporated herein by reference in its entirety.

FUNDAMENTALS OF THE INVENTION

[0002] Home entertainment systems composed of several appliances and/or the control devices used to issue commands to such appliances may be provided with devices to detect the presence of the user and/or interaction with the user through methods such as recognition of gestures, spoken and facial voice, spatial analysis, etc., as is known in the art. Furthermore, the increasing use of personal communication devices such as smartphones, tablet computers, etc., may provide additional means for identifying user presence through detection of such personal communication devices on a wireless local area network such as a wireless network. Wi-Fi, a Bluetooth network, etc. While multiple media sources and multiple media rendering devices can be coupled into many of these home entertainment systems via a centrally routed device such as an AV receiver, internet connection TV device, smart TV, etc., there is no there are currently existing systems or methods for using user presence detection and/or interaction detection alone or in conjunction with a central routing apparatus to provide enhanced home entertainment system functionality.

SUMMARY OF THE INVENTION

[0003] This invention relates generally to home entertainment systems and control methods therefor, and in particular to enhanced functionality for such home entertainment systems that are enabled by the availability of additional user-related input methods for such systems. For example, in one aspect of the invention, sensing interfaces such as an image sensing interface (e.g., an interface associated with a camera), a sound sensing interface (e.g., an interface associated with a microphone), and/or an interface to detect the presence of an RF device, such as a smartphone, can be used to fully or partially automate a system response to common events that may occur during a TV viewing session, such as a user or users leaving or entering the viewing area, a user answering a phone call, detection of a ringing doorbell or a baby monitor alarm, etc. In another aspect of the invention, data derived from such sensing interfaces can be used to improve the responsiveness of one or more system components, for example, detecting when a user is arriving at a physical remote control unit or preparing a component for issue a voice command or gesture. In yet another aspect of the invention, user presence data derived from such sensing interfaces can be used by a central routing apparatus in conjunction with media stream information in order to capture and report viewing habits and/or preferences. of user.

[0004] A better understanding of the objects, advantages, characteristics, properties and relationships of the invention will be obtained from the following detailed description and attached drawings that present illustrative embodiments and that are indicative of the various ways in which the principles of the invention can be employed .

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] For a better understanding of the various aspects of the invention, reference can be made to the preferred embodiments shown in the attached drawings in which:

[0006] Figure 1 illustrates an exemplary system in which the teachings of the present invention can be used.

[0007] Figure 2 illustrates a further exemplary system in which the teachings of the present invention can be used.

[0008] Figure 3 illustrates, in block diagram form, an exemplary hardware architecture for a device that can be a component part of the systems illustrated in Figures 1 and 2.

[0009] Figure 4 illustrates, in block diagram form, an exemplary software architecture for the device shown in Figure 3.

[0010] Figure 5 illustrates, in flowchart form, the operation of an exemplary event processing module of the software architecture illustrated in Figure 4.

DETAILED DESCRIPTION OF THE INVENTION

[0011] Referring to Figure 1, an exemplified home entertainment system in which the methods of the present invention can be applied may comprise an AV receiver 100, which serves as a hub for directing a selected video and/or audio media stream from a source device, such as, for example, a cable or satellite Internet connection device and/or DVR ("STB") device 104, a DVD player 106, a CD 108, or a game console 110 to a target device, such as a TV set 102, where the selected audio and/or video media stream will be rendered. In a preferred embodiment, connections 130, 132 between appliances 102 to 110 and AV receiver 100 may generally comprise connections for carrying HDMI-compatible digital signals, although it is appreciated that other interface standards such as component video, PCM audio , etc., can be replaced where necessary by the limitations of a particular device. In some embodiments, additional AV content streams may also be available from a streaming service 118, such as, for example, Netflix, Vudu, YouTube, NBC online, etc., via a wide area network such as such as the Internet 116, to which the final AV receiver 100 may be provided with a connection 112 to an Internet gateway device, such as, for example, the router 114. As will be appreciated, the connection between the AV receiver 100 and the Internet gateway device 114 may be wired as illustrated, or may be wireless, for example, a WiFi local area network, as the case may be. To support audio and/or video telephone calling, conferencing, etc., and in accordance with certain teachings of the present invention, an exemplified home entertainment system may also be provided with one or more sensing interfaces, such as interfaces associated with the microphone 120 and camera 122, suitable for capturing audible and/or visible events within the environment of the home entertainment system. Users 124, 126 of the illustrative entertainment system can select the media stream currently being viewed by any convenient method such as through use of a remote control as is known in the art, a voice command, a gesture, etc. In some embodiments, data with respect to these selections, including, without limitation, media source, channel, track, title, etc., along with viewing duration, user presence, etc. may be accumulated and reported to a database server 128 for the aggregation and analysis of user viewing habits and preferences as discussed in more detail below.

[0012] Returning now to Figure 2, in a second illustrative embodiment, a "smart" TV device 200 can incorporate both content rendering functionality and source stream selection. In this configuration, local devices 104 to 110 can be directly connected to various TV input ports via 200, for example HDMI connections 130. As is characteristic of the smart TV genre, the TV 200 can also support a 112 connection to a network area such as the Internet 116 over which streaming AV content and other data can be received. The means for user command input to TV 200 and appliances 104 to 110 may take the form of a control device 204, for example a conventional remote control or a smartphone application communicating with the appliances via any convenient infrared protocol ( IR), radio frequency (RF), wired, point-to-point, or network protocol, as necessary to cause the respective target devices to perform the desired operational functions. Additionally, in certain embodiments, user input may also comprise speech and/or gesture commands in lieu of or in addition to controlling device signals, sounds and gestures that may be received by microphone 120 and camera 122, processed and decoded by one of the devices, for example the TV 200, and when necessary transmitted to other target devices via, for example, CEC HDMI commands, IR or RF signals, etc., as described, for example, in Copending Patent Application US 13/ 657,176 “System and Method for Optimized Appliance Control,” commonly owned and incorporated herein by reference in its entirety.

[0013] For brevity, the discussions that follow will generally be in relation to the configuration of the exemplary equipment in Figure 2, it being understood, however, that in other modalities, for example, those illustrated in Figure 1, the steps of the methods here presented can be performed, with necessary changes, by various apparatus or combinations of apparatus, as appropriate for a particular equipment configuration.

[0014] As illustrated in Figure 3, an exemplified core routing appliance, such as smart TV appliance 200, may include, as needed for a particular application, rendering capabilities, e.g., media processor and TV engine 300 ( it being appreciated that this may comprise one or more than one physical processor, depending on the particular embodiment); memory 302 which may comprise any type of read/write media, such as RAM, ROM, FLASH, EEPROM, hard disk, an optical disk, etc., or a combination thereof; a USB interface 304; 306 digital AV interface and input ports, eg DVI or HDMI; 308 analog AV interface and input ports, eg, composite or component video with associated analog audio; an Ethernet interface and/or WiFi 310; a Bluetooth interface 328; a digital camera interface 312 with associated camera 122 which may be externally connected or built into the TV cabinet 200; a microphone interface 314 with associated microphone 120, which may be externally connected or built into the TV cabinet 200; a remote control interface 316 for receiving user-initiated operational commands via IR or RF signals 326; a screen output 318 connected to the TV screen 322; and an audio output 320 connected to internal or external speakers 324.

[0015] To cause the smart TV set 200 to perform an action, the appropriate programming instructions can be stored in memory 302 (hereinafter "TV programming") for execution by the media processor(s) and streaming engine. TV 300. An exemplary architecture for such TV programming is shown in Figure 4. As illustrated, the exemplary TV programming may include, as needed for a particular application, an underlying operating system 402, such as, for example, LINUX, which can support a set of software modules implementing the various smart TV device functionalities. These software modules may include a hardware abstraction layer 404 to provide a device-independent interface between the various application software modules and hardware-dependent software modules, such as video output trigger 406, video output trigger audio 408, HDMI interface 410, analog ADC/DAC input/output 412, WiFi and/or Ethernet interface 414, Bluetooth interface 416; USB interface 418, remote control interface 420, and camera and microphone triggers 422 and 423. The exemplified application modules that reside above abstraction layer 404 may include, as needed for a particular modality of layer interface and protocol management session and transport 428; AV output management 424; input/output processing and routing 440; a miscellaneous services module 426 to support closed captioning, display configuration, OSD, etc.; remote control command decoder 430; and device resource management 442. Additionally, in accordance with the teachings of this invention, the exemplified TV programming may include audio and visual event detector modules 432, 434, e.g., image and/or voice recognition mechanisms or similar; user event processing 436; and user statistics gathering and reporting module 438.

[0016] Under the control of such TV programming, the smart TV set 200 can, for example, receive an incoming AV media stream from one of the input ports 306, 308 to be processed, stored, sorted into audio and video components, and routed to outputs 318, 320 for rendering to TV screen 322 and speaker(s) 324; can receive commands from remote control interface 316 that are decoded and acted upon, for example, to select an input media stream, adjust audio volume, etc.; can manage an Internet connection via WiFi or Ethernet interface 310 to enable content browsing, download of software updates, video telephony using inputs from camera 122 and microphone 120; etc. Additionally, in accordance with the teachings provided herein, the exemplified TV programming can receive and process input signals from the control device 204, camera camera 122 and/or microphone 120 in order to detect user presence, identify users individual files, and/or receive user command input, as will be described below. As will be appreciated, while in the illustrative embodiment, the source of audio input signals may comprise a microphone 120 and associated interface 314 provided as part of a smart TV set 200, in alternative embodiments, the audio input signals may be captured by any other apparatus in the system and forwarded to apparatus 200 for processing, or may originate from a microphone provided in a control device, such as remote control or apparatus 204, the output of which microphone may, by way of example, be scanned and/or processed by the control device 204 wirelessly forwarded to the smart TV set 200 via the remote control interface 326, the WiFi interface 310, the Bluetooth interface 328, or any other means suitable for a particular implementation. In an exemplary embodiment, the TV programming user event processing module 436 of the TV set 200 (hereinafter "event processing") can act as illustrated in the flowchart of Figure 5 upon the occurrence of a user related event. . First, at step 502, it may be determined whether the event constitutes the receipt of a remote control command, as may be reported by remote control command decoder 430. As will be appreciated, remote control commands may be received via any or the entire RC 420 interface (eg infrared signals or RF4CE or similar), Ethernet/Wi-Fi interface 414, or Bluetooth interface 416, depending on the particular modality of the particular control device currently in use. If the event is determined to constitute the receipt of a remote control command, at step 540, the requested functional operation may be performed. By way of example, without limitation, such operations may include adjusting the output audio volume to be performed by the AV management module 424, selecting a new media input stream to be performed by the routing module, and I /O 440, etc. In some embodiments, received remote control commands may also include requests to direct the functional operation of other connected apparatus, for example, control DVD player 106 or STB 104 via CEC commands to be issued by interface management module 428 through HDMI 130 connections; or, alternatively, via a connected IR burner or a LAN, as described, for example, in copending application US Patent Application 13/657,176 “System and Method for Optimized Appliance Control”, commonly owned and incorporated herein by reference in its entirety. Upon completion of the command function requested in step 542, it can then be determined whether the command function just executed comprised a change in the media stream being rendered by the TV 200, for example, the selection of a new port input from 306, 308; a switch to the selected broadcast channel or DVR playback from STB 104; a switch to an internet media source, etc. If so, at step 544, data relating to this event may be piped to user statistics module 438 for logging and ultimately reporting to database server 128. As will be appreciated, data logged about the new content stream may comprise some or all of the command parameters themselves, for example, an STB channel number and timestamp; metadata items can be retrieved from the content source device or channel, such as a DVD title, video URL, etc.; a sample of the audio or video content itself for analysis, as known in the art and described, for example, in US Patents 7,986,913, 7,627,477 or 7,346,512; or any other data that may be suitable for identification purposes. If it is determined that the executed function did not comprise any change in the current media stream, at step 546 it can next be determined whether the executed function comprised a reportable event, such as, for example, turning on or off TV 200 (or any connected device), issue a fast forward command during DVR playback, etc. If so, this event can also be reported to user statistics module 438 for logging, after which remote control command processing is complete. In this regard, it should be appreciated that reportable device events, such as turning a docked device on or off, may also be initiated separately, for example, by directly communicating with a device from its own remote control. Consequently, although not illustrated, when such events are detectable, for example via HDMI status, absence or presence of audio or video signals, etc., these events can also be reported and logged.

[0017] If the reported event is not a remote control command, in step 504, event processing may next determine whether the reported event constitutes an image change event reported by the visual event detection module 432 in response to the analysis of image data received from camera 122 via camera trigger 422. Such image processing may utilize, for example, the techniques described in US Patents 5,534,917, 6,829,384, 8,274,535, Publication Application for WIPO patent (PCT) WO 2010/057683A1, or similar, and may, for example, periodically monitor an image comprising the field of view of camera 122, in order to initiate image analysis, in response to detection of any variation in the image data that exceeds a certain threshold value. If it is determined that the event is a report of a detected image change, at step 518, it is next determined whether the event comprises a user's exit or imminent exit from the viewing environment of television 200. If so, various actions may be performed by the 436 event processor, as appropriate. By way of example, if the starting user is the only viewer (or, in some embodiments, the primary user, for example, the user who initiated the current viewing session, the user who is provided within the system with a higher priority high relative to remaining users, or similar) in event processing can be adapted to issue a “pause” command to the source of the current media stream. Other actions may include, but are not limited to, activating a DVR's recording function, logging off a web site, etc., as appropriate for a particular modality and configuration. If such an action is performed, at step 520 in the illustrative embodiment, event processing may cause a request for confirmation to be displayed on TV screen 322, for example, “Would you like to pause this show? (Y/N)”. If confirmed by the user in step 522, that confirmation may take the form of a gesture, spoken command, remote control input, etc., or, in embodiments where the default is to perform an action, a timeout, in step 528 the indicated action can be performed. In this context it should be noted that, in modalities where voice or gesture responses are expected, the performance accuracy of audio and/or visual event detection modules 432, 434 can be improved by indicating a range of possible responses ( e.g. “yes” or “no” in this case) for these modules in advance, thus limiting the number of sound models or gestures that need to be combined. Also, in some embodiments where voice input may be used, the sound level of TV audio output 320 may be temporarily reduced to reduce background noise. Then, at step 530, the data relating to the user change, including the identity of the user where this can be determined, for example, through the use of techniques as described in US Patents 7,551,756, 7,702,599, or similar, may be transmitted to statistics gathering module 438 for recording, after which processing is complete.

[0018] If the detected image change event is not a user departure, in step 524, the event processor may next determine whether the reported event comprises the arrival of a new or additional user in the TV viewing environment and if so, take appropriate action. For example, some modalities can be adapted to allow a viewer to invoke a “private view” state that can cause the current content to be automatically muted, paused, swapped, etc. in case an additional user enters the preview environment. In yet another embodiment, a user's entry into a viewing environment may trigger an offer to resume playback of previously paused content; or in a multi-room, multi-device home environment where appliances are networked and equipped with viewer awareness, a user entering a viewing environment may cause the event processor in that environment to query other viewer processors. events and/or statistics modules in other parts of the home environment to determine if that user recently left another environment, and if so, offer to resume playback of a content stream that was paused in that other environment.

[0019] It is noted that the action(s) to be performed upon a user entering the viewing environment can be defined as being specific to the identity of the arriving person, for example, to be performed only when a specifically recognized individual or type /category of recognized individual, such as a child, enters the viewing environment. Similarly, the action(s) to be performed upon a user's arrival in the viewing environment can be defined as being specific to the identity and/or type of user or users currently watching. In this case, an action to be taken might be one to inhibit the performance of any further actions by the currently watching user or users, for example to inhibit a child from changing a channel that is currently being watched to thereby allow a parent to have the opportunity to see what the child is currently watching. In this last example, the inhibition of any action(s) can be carried out after a certain period of time has expired, through the new user leaving the visualization area again, through the new user replacing the action (for example, through a gesture, voice, new action, or similar - to the extent the user is authorized to perform such action), etc. In still other cases, the actions to be taken when a user enters the viewing area can be prioritized based on the identities of the newly arrived user and the user or users currently watching. Furthermore, actions to be taken by the system upon a detected “user input event” can be used to provide one or more devices within the system with desired states, for example to set volume level settings, closed caption settings, business skip settings, SAP tuning, light level settings, and/or similar type of user preference settings without limitation. Thus, from the previous examples, it is appreciated that various combinations of actions can be specified for the system to perform the detection of a “user input event”, actions that may or may not consider user identities and/or priorities. of user identity, and as such, the examples provided herein are not intended to be limiting in any way.

[0020] As yet another example, the actions to be performed by the system upon the detected arrival of a new user to the viewing area may cause the system to combine the lists of favorite channels (as shown in an electronic program guide ) that have been established for the various viewers in a single list or to provide a single list that will only include those programs and/or channels that are commonly found within the favorite channel lists that have been established for each of the various viewers. Likewise, access to videos, games, programs, channels or the like (as defined through the use of a parental control chip) may be limited to only those videos, games, programs, channels or the like that are normally accessible to each of the multiple viewers. Furthermore, to the extent that the preferences established for recognized users do not conflict, the system may take the appropriate action(s) to establish within the system one or more of these preferences upon detection of an "event". user input”.

[0021] It is for the purposes mentioned above in step 526 that it can be determined whether such “user input event” action preferences have been set and, if so, in step 528 the appropriate action(s) can be taken, after which data relating to the arrival of the user, including the identity of the user where it can be determined, can be conveyed to statistics gathering module 438 for recording, and event processing is complete.

[0022] It is also clear that, while the above describes various actions that can be performed upon detection of a “user input event”, it is understood that similar actions can be performed when the system is started for the first time in the presence from multiple viewers.

[0023] In the event that it is detected that a user responsible for a “user entry event” leaves or is in the process of leaving the visualization area, that is, a “user exit event” is detected, the actions The above actions taken in conjunction with the “user input event” can be automatically reverted (or the remaining user(s) can be warned if a reversal action is about to occur). For example, video that has been paused can be restarted, electronic program guides/favorite channel lists can be restored, etc. Similarly, a “user exit event” indicative of one or more users exiting or preparing to leave the viewing area can be used by the system to take actions that would remove consideration of the existing user preference, to put the system into a state that is appropriate for the remaining users, etc. As before, the actions to be taken by the system upon detection of a "logout event" can be defined to be specific to the identity of the user or users leaving, the identity of the user or users remaining, and/or may be prioritized based on the identities of the leaving user or users and the remaining user or users without limitation. It is again for these purposes that at step 518 it may be determined whether such "user exit events" action preferences have been set and, if so, at step 520 the appropriate action(s) may be taken, after which data about the exiting user, including the identity of the user where it can be determined, can be passed to statistics gathering module 438 for recording, and event processing is complete.

[0024] If the detected image change event is not a user arrival or departure, in steps 532, 534 and 536, it can then be determined whether the reported event comprises a detectable gesture, a detectable gesture in this context comprises an action or user movement which either by pre-programming or learning process has been identified to the visual event detection module 432 as having significance as user input. If the reported event is determined to be associated with an operational command function, eg, "pause", "mute", "channel up", etc., processing may continue at step 540 to execute the command as previously described. If the reported event is determined to be a readiness gesture, appropriate anticipatory action may be taken at step 538. In this context, a readiness gesture may comprise, without limitation, any preliminary movement or gesture by a user that could be interpreted as a possible indication of that user's intention to perform an action, e.g. standing, picking up or using a remote control device, waving a person into the room, picking up a phone, etc. Anticipatory actions may include, for example, pre-conditioning visual and/or audio detection modules 432,434 to favor certain subsets of models for matching purposes; modify an on-screen menu from a format optimized for gesture control to one optimized for navigation using a keyboard-based device such as a remote control, or vice versa; reduce or mute audio volume; signal a remote control device coming out of a quiescent state, turn its backlight on or off, or the like; etc.

[0025] If the reported event is not an image change event, in step 506, the event processing can next determine whether the reported event constitutes a sound recognition event reported by the audio event detection module 434. Sound and speech recognition by module 434 can use, for example, the techniques described in US Patent 7,603,276, WIPO Patent Application Publication (PCT) WO 2002/054382A1, or the like. If the event is a sound recognition event, at step 512, it may be determined whether the decoded sound constitutes a voice command issued by the user. If so, processing can continue at step 540 to execute the desired command as previously described. If it is not a voice command, at step 514, it may be determined whether the reported event constitutes a trigger sound. In this context, a trigger sound may be an occurrence of a non-vocal audio signal received through microphone 120 which, either by pre-programming or via a learning process, has been assigned a command or anticipatory action. By way of example, without limitation, trigger sounds may include a ringing telephone or doorbell, a baby monitor, smoke alarm, microwave chime, etc. If the reported sound event sound is a trigger sound, the appropriate action, such as muting the television, etc., can be performed at step 516. The system can also be programmed to recognize various sounds and/or phrases/spoken words as being indicative of a preparatory event whereby one or more of the system components will be ready via an anticipatory action as described above. By way of example, a spoken phrase such as “let's see what else is on” can be recognized as a preparatory event whereby an anticipatory action can be taken to place a remote control device in a state where the control device remote is ready to receive input in anticipation of its use or a spoken phrase such as “come here”, the sound of a doorbell or similar can be recognized as a preparatory event whereby an anticipatory action can be taken to prepare the system to look for an anticipated event, for example a specific gesture, such as a user standing up, leaving the preview area, etc. whereby the appropriate response action for the anticipated captured event, for example pausing the media, can be performed.

[0026] In the case that an anticipated event is not realized within a predetermined period of time, the system can execute, if necessary, other actions, such as repair action, to put the system in a state as desired, for example , to return one or more components of the home entertainment system to a state where the component(s) is no longer looking for the anticipation event to occur.

[0027] If the reported event is not a sound recognition event, at step 508, the event processing can next determine whether the reported event comprises a wireless device, such as, for example, a smartphone, tablet computer, game controller, etc., by joining or leaving a LAN or PAN associated with the smart TV 200 or other device in the system. If these devices were previously registered with TV programming, such activity can be used to infer the presence or absence of particular users. Such information may then be processed in a manner similar to user image detection (e.g., processed as a user being added or leaving) by continuing to step 518 as described above.

[0028] Finally, in step 510, the event processor may process any other event reporting activity as consistent with a particular modality. For example, in some embodiments, users may be provided with personal remote control devices that incorporate user identification data into their command transmissions, as described in the Copending US Patent Application. 13/225635 "Controlling Devices Used to Provide an Adaptive User Interface," commonly owned and incorporated herein by reference in its entirety, in which case user presence reporting events may be generated by remote control interface 420. In other embodiments, technologies, such as infrared body heat detection as proposed in the art for use in automatic autonomous shutdown applications, may further be adapted for the purposes described herein. Additional sources of activity events can also include data received from other home equipment such as security, lighting, HVAC control systems equipped with occupancy sensors; entrance cameras, garage sensors, etc., where appropriate.

[0029] The statistics gathering module 438 can be adapted to report the data conducted to it during the event processing steps described above to a centralized service, for example hosted on the Internet connected server device 128, for aggregation and analysis of user viewing habits and preferences. Depending on the particular embodiment, such reports may be performed on an event-by-event basis, or, alternatively, data may be accumulated and reported at predetermined time intervals or only upon receipt of a request from the server device.

[0030] By way of example, in an illustrative embodiment, the data reported to the statistics gathering module 438 may be formatted into various classes and different event record types for loading into the server device 128. The exemplified record classes may include user events, for example, as may be reported in step 530 of Figure 5; apparatus events, for example, as may be reported in step 548 of Figure 5; and content events, for example, as may be reported in step 544 of Figure 5. As will be appreciated, different or additional event classes may be appropriate in alternative embodiments, and therefore the above classifications are presented by way of example only and without limitation.

[0031] Referring to Table 1 below, user event record types can include addition (ie, arrival) of the user to the preview area and deletion (ie, departure) of a user from the preview area. As illustrated, each of these record types can include timestamp data and a user ID. The illustrated timestamp is suitable for use in applications where the server device already has knowledge of the geographic location of the reporting system, for example as a result of an initial setup process, by URL decoding, etc. In embodiments where this is not the case, the timestamp field can include additional data such as a time zone, postal code, etc., whenever necessary. The user identity can be any data item that serves to uniquely identify individual users to facilitate visualization of habits and preference analysis at server 128. By way of example, the user ID data can comprise identities explicitly assigned during a process definition/configuration; random numbers assigned by the system as each distinct user is initially detected; a check value generated by a facial or voice recognition algorithm; a MAC address or serial number assigned to a smartphone or tablet computer; etc. as appropriate for a particular modality. Table 1 : User event log

[0032] Referring now to Table 2, device event records may comprise record types indicative of events reported to, functional commands issued to, and/or operations performed by various controlled devices, for example, as reported in step 548 of Figure 5. These events may include, without limitation, device on/off commands, playback control commands, etc., as necessary for a complete understanding of user habits and preferences. For example, in addition to capturing the power state for a DVR set, fast forward commands can also be logged in order to monitor commercial skipping activity. As illustrated, each type of device event record may also include a timestamp field as described above and a device type/ID field that comprises an indicator of device type (e.g., STB/DVR, DVD, Internet stream, etc.), together with a unique ID or subtype value that may be assigned by the monitoring and/or statistics gathering module in order to distinguish between multiple appliances of the same type, for example a domestic appliance with several DVD players, or with cable or satellite STBs. Table 2: Device event log

[0033] Referring now to Table 3, the content event record types may include, without limitation, a track or channel change information record type; a title information record type containing, for example, a show title retrieved from program guide data, or video-on-demand title information, etc.; a type of metadata record containing metadata values obtained from a DVD or CD, streaming video service, or the like; a content sample record type containing a sample audio and/or video content clip for comparison to a content identification database; or in alternative embodiments any other data that can be used to determine the identity of a particular content stream. Each record type may comprise timestamp fields and source apparatus, as described above, along with a field containing identity data, which may comprise numeric, text, or binary data, as required. As will be appreciated, additional record and/or field types may be used in other embodiments, where necessary, to allow reliable identification of media content streams. Table 3: Content event log

[0034] It is clear that although the exemplified data structures of Tables 1 to 3 are presented here using a tabular format for ease of reference, in practice these can be implemented in various forms using any convenient data representation, for example, a structured database, XML file, cloud-based service, etc., as appropriate for a particular modality. Furthermore, it is also clear that, although the functionality of gathering and recording statistics of the illustrative embodiment is implemented as part of the programming of an exemplified apparatus 200, that is, in the software module 438, in other embodiments this functionality can be provided in a different location, for example, on one of the other devices forming part of an entertainment system, on a local PC, on a remote server or cable system head office, etc., or in any other convenient location to which the particular programming apparatus may be able to report user events.

[0035] While various concepts have been described in detail, it is clear to those skilled in the art that various modifications and alternatives to these concepts could be developed in light of the teachings of the description. For example, in alternative embodiments, the method steps described above may advantageously be performed on various other apparatus, as appropriate, for example, an AV receiver or a cable/satellite STB. Furthermore, in an interconnected system as illustrated in Figures 1 or 2, especially when such interconnection is digital, it should be understood that the various steps of the methods may be carried out by different devices. For example, without limitation, image analysis and/or speech recognition can be performed by a smart TV device, a locally connected personal computer, a home security system, etc., or even “in the cloud”, i.e. , by an Internet-based service, with the results reported to a user statistics gathering module and/or an event processing module residing in a connected AV receiver or STB.

[0036] Furthermore, although described in the context of functional modules and illustrated using the block diagram format, it is understood that, unless otherwise indicated, one or more of the functions and/or features described may be integrated into a single physical device and/or a software module, or one or more functions and/or features may be implemented in separate physical devices, or software modules. It is also clear that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the invention. Rather, the actual implementation of such modules would be well within an engineer's routine, given here a description of the attributes, functionality, and interrelationship of the various functional modules in the system. Therefore, one skilled in the art, applying his knowledge, will be able to practice the invention presented in the claims without undue experimentation. It will be further appreciated that the particular concepts described are illustrative only and not limiting to the scope of the invention which will be given the full breadth of the appended claims and their equivalents.

[0037] All patents cited herein are hereby incorporated by reference in their entirety.

Claims (1)

1. A method for controlling at least one component device (300) in a home entertainment system comprised of a plurality of component devices, comprising: receiving event data via an image detection interface (312) while the at least one of the plurality of component devices (200) is in a tuning state; determining, from the received event data, at least a number and identity of viewers in a viewing area associated with the home entertainment system; and when it is determined from the received event data that the number and identity of viewers in the viewing area associated with the home entertainment system, in accordance with a predetermined rule, have changed causing a command action to be performed wherein the at least one of the plurality of component devices (200) is caused to perform an operational function to thereby prevent a change of setting state of the at least one of the plurality of component devices (200); characterized in that the at least one of the plurality of component devices (200) is temporarily rendered unresponsive to adjustment commands provided to it for at least a predetermined period of time in connection with executing the command action and wherein, after Upon expiration of the predetermined period of time, the at least one of the plurality of component devices (200) automatically returns to being responsive to adjustment commands provided thereto.