BR112015028858B1 - SYSTEM AND METHOD FOR OPTIMIZED APPLIANCE CONTROL - Google Patents

Abstract

system and method for optimized device control. a device receives a request from a control device, such as a remote control, smart phone, or the like, where the request is intended to cause one or more target devices to perform one or more functional operations. the device responds to the request by applying the optimal methodology to propagate one or more commands to each targeted target device to cause each targeted target device to perform one or more targeted functional operations.

Description

[0001] This application claims benefit and is a continuation in part of US Application No. 13/657,176, filed October 22, 2012, which application claims the benefit of US Interim Application No. 61/552,857, filed October 28, 2012. October 2011, and US Interim Application No. 61/680,876, filed August 8, 2012, the disclosures of which are incorporated herein by reference in their entirety.

[0002] This application is also related to US Patent Application No. 12/621,277, filed November 18, 2009 and entitled “System and Method for Reconfiguration of an Entertainment System Controlling Device”, which, in turn, is a continuation in part of US Patent Application No. 12/569,121 (now US Patent 8,243,207), filed September 29, 2009 and entitled "System and Method for Activity Based Configuration of an Entertainment System", the disclosures of which are incorporated herein by reference in their entirety.

[0003] This application is also related to US Patent Application No. 13/198,072, filed August 4, 2011 and entitled "System and Method for Configuring the Remote Control Functionality of a Portable Device", the disclosure of which is incorporated herein by reference in its entirety.

[0004] This application is also related to U.S. Patent Application No. 13/240,604, filed September 22, 2011 and entitled "System and Method for Configuring Controlling Device Functionality", the disclosure of which is incorporated herein by reference in its entirety.

FUNDAMENTALS

[0005] Control devices, e.g., remote controls, for use in issuing commands to entertainment and other devices, and the features and functionality provided by such control devices are well known in the art. In order to facilitate such functionality, various communication protocols, command formats and interface methods have been implemented by device manufacturers to allow operational control of entertainment devices and more, also known in the art. In particular, the recent proliferation of wireless and wired communication and/or digital interconnection methods such as WiFi, Bluetooth, HDMI, etc., together and between devices has resulted in a corresponding proliferation of such communication protocols and command formats. While many of these more modern methods may offer better performance and/or reliability when compared to previous control protocols, appliance manufacturer adoption of such more modern methods remains inconsistent and fragmented. This, along with the large installed base of previous generation devices, can cause confusion, faulty operation, or other problems when a user or manufacturer of a control device, such as a remote control, tries to take advantage of the improved features and functionality of those devices. new control methods.

SUMMARY OF THE INVENTION

[0006] This invention relates generally to improved methods for controlling apparatus by using a control device, such as a remote control, smart phone, tablet computer, etc. and, in particular, to methods for taking advantage of improved device control communication methods and/or command formats in a reliable manner that is largely transparent to a user and/or seamlessly integrated with legacy device control technology.

[0007] To this end, the present invention comprises a modular hardware and software solution, hereinafter referred to as a Universal Control Engine (UCE), which is adapted to provide device control through a variety of control methodologies and media. available communications, such as, for example, various infrared (IR) remote control protocols; Consumer Electronic Control (CEC) that can be implemented over a wired HDMI connection; Internet Protocol (IP), wired or wireless; Wireless RF4CE; Bluetooth (BT) wireless personal area network(s); UPnP protocol using wired USB connections; or any other available device or patented device control methodology. Since each individual control paradigm may have its own strengths and weaknesses, the UCE can be adapted to combine various control methods in order to realize the best control option for each individual command for each individual device.

[0008] The UCE itself can be adapted to receive commands from a control device, for example, a conventional remote control or a remote control app residing in a smart device such as a telephone or graphics tablet, etc., using any protocol. and convenient command structure (IR, RF4CE, BT, proprietary RF, etc.). As will become apparent, the control device can range from a very simple unidirectional IR device to a fully functional WiFi-enabled smart phone or the like. The UCE can receive command requests from such a control device and apply the optimal methodology to propagate the command function(s) to each targeted target device, such as, for example, a TV, AV receiver, player DVD, etc. In this way, the UCE can allow a single control device to command the operation of all the devices in a home theater system, while still coordinating available methods of controlling each particular device in order to select the best and most reliable method to emit each command for each given device. By way of example, and without limitation, a UCE may use IR commands to turn on an AV receiver set, while CEC commands or another method can be used to select inputs or turn off the same AV receiver set; or CEC commands can be used to turn on and select inputs on a TV set, while IR commands can be used to control the volume on the same TV set.

[0009] As will become apparent, a UCE may comprise modular hardware and software that may be incorporated into a standalone device suitable for use in an existing home theater equipment configuration, or may be incorporated into any of the apparatus such as a Receiver. STB, TV, AV, HDMI key, etc. Additionally, when embedded in a device, UCE functionality may be provided as a separate hardware module or may be incorporated together with other hardware functionality, e.g. as part of an HDMI interface IC or chipset, etc.

[0010] A better understanding of the objects, advantages, features, properties and relationships of the invention will be gained from the following detailed description and accompanying drawings which present illustrative embodiments and which are indicative of the various ways in which the principles of the invention may be employed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] For a better understanding of the various aspects of the invention, reference may be made to preferred embodiments shown in the accompanying drawings, in which:

[0012] Figures 1 and 2 illustrate exemplary systems in which an independent UEC device can be used to command the operation of several devices;

[0013] Figures 3 and 4 illustrate exemplary systems in which UEC functionality can be incorporated into a device that is part of a home entertainment system;

[0014] Figure 5 illustrates a block diagram of an exemplary UEC device;

[0015] Figure 6 illustrates a graphical representation of an exemplary UCE-based control environment;

[0016] Figure 7 illustrates an exemplary preferred command matrix for use in a UCE-based control environment, for example, as illustrated in Figure 6;

[0017] Figure 8 illustrates a block diagram of an exemplary smart device that can support a remote control app and a configuration method for use in configuring an ECU;

[0018] Figure 9 illustrates an exemplary series of steps that can be performed in order to organize and configure an exemplary UCE;

[0019] Figure 10 illustrates an exemplary series of steps that can be performed in order to define for a UCE a device configuration that corresponds to a user activity;

[0020] Figure 11 illustrates exemplary activity configuration matrices such as those that can be defined during the steps of Figure 10;

[0021] Figure 12 illustrates an exemplary current appliance state matrix that can be maintained in an ECU for use in determining the commands needed to invoke one of the states defined by the matrix of Figure 11;

[0022] Figure 13 illustrates an exemplary series of steps that can be performed by an ECU in issuing a function command to a device;

[0023] Figure 14 illustrates an exemplary series of steps that can be performed by a UCE in establishing device states that match a desired activity defined in one of the matrices of figure 11; and

[0024] Figure 15 illustrates an exemplary series of steps that can be performed by a smart device to configure command control macros.

DETAILED DESCRIPTION

[0025] Referring to Figure 1, an exemplary system is illustrated in which an ECU device 100 can be used to issue commands to control various controllable apparatus, such as a television 106, an external digital cable converter apparatus combined with a video recorder. digital video (“STB/DVR”) 110, a DVD player 108 and an AV receiver 120. Although illustrated in the context of a television 106, STB/DVR 110, a DVD player 108 and an AV receiver 120, it should be noted understand that controllable appliances may include, but are not limited to, televisions, VCRs, DVRs, DVD players, cable or satellite external digital converter boxes (“STBs”), amplifiers, CD players, game consoles, home lighting, wall hangings, fans, HVAC systems, thermostats, personal computers, etc. In the illustrative example of Figure 1, apparatus commands may be issued by the ECU 100 in response to infrared ("IR") request signals 116 received from a remote control device 102, radio frequency ("RF") request signals 118 received from an app 124 residing on a smart device 104, or any other device from which the UCE 100 can be adapted to receive requests, using any appropriate communication method. As illustrated, transmission of requested device commands from the UCE to devices 106, 108, 112, 120 may take the form of wireless IR signals 114 or CEC commands issued by a wired HDMI interface 112, as appropriate for the capabilities of the device. particular device to which each command can be directed. In particular, in the exemplary system illustrated, AV receiver 120 may not support HDMI inputs, being connected to audio source devices 108, 110, for example, via S/PDIF interfaces 122. In this way, UCE 100 may be limited to transmit all commands intended for AV receiver 120 exclusively as IR signals, while commands intended for other apparatus 106 to 110 may take the form of either CEC or IR signals in the manner appropriate for each command. By way of example, without limitation, certain TV manufacturers may prefer not to support volume adjustment through CEC. If the illustrative TV 106 is of such manufacture, the UCE 100 can relay volume adjustment requests to the TV 106 as IR signals 114, while other requests such as on/off or input selections can be relayed in the form of CEC commands by the HDMI 112 connection.

[0026] However, it should be noted that, although illustrated in the context of wired IR, RF, and CEC signal transmissions, in general, transmissions to and from the UCE device 100 can take the form of any IR protocol. , RF, hardware, point-to-point, or conveniently networked in the manner required for a particular modality. Additionally, although wireless communications 116, 118, etc. between exemplary devices are illustrated here as direct links, it should be noted that in some cases such communication may take place via a local area network or personal area network and, as such, may involve various intermediary devices such as routers. , bridges, access points, etc. Since these items are not necessary for an understanding of the present invention, they are omitted from this and subsequent figures for the sake of clarity.

[0027] Since smart device remote control apps such as the one contemplated in the illustrative device 104 are well known and, for the sake of brevity, their operation, features and functions will not be described in detail here. However, if a more complete understanding of the nature of such apps is desired, the interested reader may refer, for example, to US Patent Application No. 12/406,601 or the aforementioned US Patent Application No. 13/329,940, entitled “Graphical User Interface and Data Transfer Methods in a Control Device”, both of which are common property and incorporated herein by reference in their entirety.

[0028] Returning now to Figure 2, in an additional illustrative embodiment, ECU 100 can receive wireless request signals from a remote control 200 and/or a device residing on a tablet computer 202. As before, command transmissions to apparatus 106, 108, 110 may take the form of wired CEC commands or wireless IR commands. However, in this example, remote control 200 can be in bidirectional communication 208 with UCE 100, and in this way, the UCE can delegate the transmission of IR commands 210 to remote control device 200, i.e., use remote control 200 as a device relay for those commands determined to be best performed via IR transmissions. As also generally illustrated in Figure 2, a configuration apparatus 214 running on a smart device such as a tablet computer 202 may be used in conjunction with an Internet accessible server (212, 204) 206 and associated database 207 to initially configure the UCE 100 for operation with the specific group of devices to be controlled, i.e. communicate to the UCE 100 a set of conjugated command codes and capacity profile for each particular device to be controlled, for example, based on the type , manufacturer, model number, etc., as described in more detail below.

[0029] With reference to figure 3, in also an additional illustrative embodiment, the UCE functionality 100' can be built into a device, for example, STB/DVR 310. In this example, the remote control 102 and/or smart device 104 can transmit wireless request signals directly to the STB/DVR 310 for action by the built-in 100' ECU function, whose actions may, as before, comprise command transmissions from the CEC via HDMI connection 112 or IR command transmissions 114, coming from this case of an IR emitter provided in the device STB/DVR 310. In this configuration, a configuration application residing in STB/DVR 310 can be used to configure UEC 100', using, for example, an Internet connection 304 accessible through a modem cable and/or central cable distribution system.

[0030] In the additional illustrative embodiment of figure 4, UCE 100' functionality can be built into an AV receiver 420 which can serve as an HDMI switch between various content sources such as an STB/DVR 110 or a DVD player 108 and a rendering device such as TV 106. In addition to HDMI inputs, AV receiver 420 may also support various other input formats, for example, analog inputs such as illustrative 404 of CD player 408; composite or component video; S/PDIF coaxial or fiber optic; etc. In this embodiment, request signals 406 can be directed to AV receiver 420, for example, by remote control 402, for action by ECU function 100'. As before, resulting device commands can be transmitted using CEC signals transmitted over HDMI connections 112, or via IR signals 114 transmitted from an associated IR emitter. As appropriate for a particular modality, the initial configuration of the UCE 100' to match the equipment to be controlled can be done by an internet-connected app residing on the AV receiver 420, or by an app residing on the tablet computer 202 or other smart device. , as previously mentioned in association with Figure 2.

[0031] As can be seen, several other configurations are also possible without departing from the fundamental UCE concept, for example, the UCE 100' function can be incorporated into an Internet capable TV, an HDMI key, a game console, etc.; device command set and capability database 207 may be located on a central cable system, may be stored locally (in whole or in part), which local storage may take the form of internal memory in the ECU itself or in an instrument such as a TV, STB or AV Receiver, or may take the form of a memory card or the like attachable to a smart device or appliance; etc.

[0032] With reference to Figure 5, an exemplary UCE device 100 (which independently of whether an apparatus supporting UCE functionality) may include, according to the need of a particular application, a processor 500 coupled to a memory 502, whose memory can comprise a combination of ROM memory, RAM memory and/or non-volatile read/write memory and may take the form of a chip, a hard disk, a magnetic disk, an optical disk, a memory card, etc., or any other combination of these. It should also be noted that some or all of the memories illustrated may be physically incorporated into the same IC chip as the processor 500 (a so-called "microcontroller") and, as such, are shown separately in Fig. 5 for clarity only. Interface Hardware provided as part of the exemplary UCE platform may include IR receiver circuitry 504 and IR transmitter circuitry 506; an HDMI 508 interface; a transceiver and WiFi 510 interface; an Ethernet interface 512; and any other wired or wireless I/O interface(s) 514 appropriate for a particular modality, by way of example, without limitation, Bluetooth, RF4CE, USB, Zigbee, Zensys, X10/Insteon, HomePlug, HomePNA, etc. The electronic components comprising the exemplary UCE device 100 may be powered by an external power supply 516. In the case of a stand-alone UCE device such as illustrated in Figures 1 or 2, this may comprise, for example, a compact "socket" AC adapter. wall”, while integrated ECU devices as illustrated in figures 3 or 4 can extract operating power from the device in which they are integrated. It should also be appreciated that, in the latter case, in certain embodiments, the processor 500 and/or memory 502 and/or certain portions of interface hardware items 504 to 514 may be shared with other functionality of the host device.

[0033] As those skilled in the art understand, part or all of the memory 502 may include executable instructions that are intended to be executed by the processor 500 to control the operation of the UCE device 100 (collectively, the programming of the UCE) as well as data that serve to define the control protocols and command values required for use in transmitting command signals to controllable devices (collectively, the command data). In this way, the processor 500 can be programmed to control the various electronics in the exemplary UCE device 100, for example, to monitor the communication device 504, 510 for request messages arriving from control devices, to cause the transmission of signals. device control, etc. To make the UCE device 100 perform an action, the UCE device 100 can be adapted to be responsive to events, such as a request message received from the remote control 102 or smart device 104, changes in the state of the connected device reported by the interface HDMI 508, WiFi interface 510, or Ethernet interface 512, etc. In response to an event, appropriate instructions in the UCE schedule can be executed. For example, when a command request is received from a smart phone 104, the UCE device 100 may retrieve, from command data stored in memory 502, a preferred command transmission medium (e.g., IR, CEC over HDMI, IP over WiFi, etc.) and a corresponding command value and control protocol to be used in transmitting that command to a targeted target device, e.g. TV 106, in a format recognizable by that device to thereby control one or more operations functionality of this device. As a further example, the status of connected devices, e.g. on or off, input currently selected, playing or paused, etc., as can be discerned by interfaces 508 to 514, can be monitored and/or tabulated by programming. of the ECU to make it easier to adjust the device settings to match user-defined activity profiles, e.g. “Watch TV”, “Watch a movie”, etc.

[0034] An overview of an exemplary UCE control environment is presented in Figure 6. The UCE programming of an exemplary UCE device 100 may comprise a Universal Control Engine core 650 along with a series of scalable software modules 652 to 660, each module supporting a particular device command protocol or method and provisioned appropriately for a particular embodiment. By way of example, the illustrative embodiment of Figure 6 may include an Internet Protocol (IP) module 652, a CEC over HDMI module 654, a Bluetooth module 656, an IR module 660, and other module(s) 658, in a manner appropriate for the particular application. Devices to be controlled may include an IP 620 enabled AV receiver, an IP 610 enabled STB/DVR, TV 106, DVD player 108, and CD player 408. As illustrated, certain of these devices can be interconnected via the HDMI 112 and/or Ethernet 670 interfaces. (In this regard, it should be noted that the illustrative interconnects 112 and 670 of Figure 6 are intended to represent only logical topography and thus details of the exact physical cabling structure and/or the presence of any switch, router, hub, repeater, required interconnect, etc. are omitted for clarity.)

[0035] The preferred method/protocol/means for issuing commands for the exemplary devices in figure 6 may vary both by device and by the function to be performed. By way of example, 622 analog input selection and volume control commands targeted at the 620 AV receiver may have to be output via IR transmissions, while HDMI 624 input and on/off select functionality commands may be better communicated. through CEC commands, and 626 advanced functionality commands, such as sound field configuration, may be better communicated through an Ethernet connection. In a similar way, the various operational functions of the other devices can best be commanded through a mixture of means, methods and protocols, as illustrated. As can be seen, in some cases, a particular device may support reception of an operational command through more than one path, for example, the on/off function of the AV 620 receiver may be available not only as a command from the CEC , but also via an IR command. In such cases, the preferred UCE command format may be the one that has been determined to provide the greatest reliability, e.g., in the cited case, the CEC command may be preferred, as this command form is not line-of-sight dependent. and also allows confirmation that the action was performed by the target device.

[0036] In order to determine the optimal method for each type of device and controller configured, the exemplary UCE core program 650 may be provisioned with a preferred command matrix 700, as illustrated in Figure 7. Exemplary preferred command matrix 700 may comprise a series of cells or data elements, for example cells 712, each corresponding to a specific command 702 and a specific one of the devices to be controlled 704. The data content of a cell or element like this may comprise identification of a command/transmission form to use and a pointer to the required data value and formatting information for the specific command. By way of example, the data element 712 corresponding to the "Input 2" command 706 for the configured TV set 708 may comprise an indicator that a CEC command is to be used, i.e. an indicator of the transmitting device, or that is, to be used to communicate the command to the target device, along with a pointer to the appropriate command value data and HDMI-CEC bus address; while data element 714 corresponding to the same command function for configured AV receiver 710 may comprise an indicator that an IR command is to be used, along with a pointer to appropriate command data and formatting information in a stored IR code library anywhere in UCE 502 memory. In certain embodiments, one or more secondary command arrays 716 may also be provisioned, allowing the use of alternative command methods in the event that it is determined by UCE programming that a preferred command has gone well successful. Command matrix 700 may also contain null entries, for example 718, where a particular function is not available or not supported by a specific device. In an exemplary embodiment, command matrix 700 may be created and loaded into memory 502 of ECU 100 during an initialization and configuration process, as will now be described in further detail.

[0037] In order to perform initial configuration of an ECU device, a configuration application can be provided. In some embodiments, a configuration application may take the form of programming to be executed on any convenient device with a suitable user interface capable of communicating with the ECU, such as, without limitation, a smart phone, tablet computer, personal computer, external digital converter box, TV, etc., in a manner appropriate for a particular embodiment. In other embodiments, a configuration application such as this can be incorporated into the UCE's own programming, using, for example, a connected TV screen and an associated control device as the user interface. Regardless of the exact shape and location of the programming device and user interface, the series of steps that can be performed by an ECU configuration application when configuring an ECU device for operation with a specific set of fixtures remains similar. In this way, it can be seen that the methods comprising the illustrative UCE configuration application presented below in association with Figures 8 and 9 can be generally applied, MUTATIS MUTANDIS, to the various alternative configuration application modalities.

[0038] With reference to figure 8, as known in the art, a tablet computer such as the exemplary device 202 of figure 2 can comprise, according to the need of a particular application, a memory 802 of the processor 800, whose memory can comprise a combination of ROM memory, RAM memory and/or non-volatile read/write memory and may take the form of a chip, a hard disk, a magnetic disk, an optical disk, a memory card, etc., or any other combination of these. In some embodiments, provision may also be made for attaching external memory 804 which may take the form of an SD card, memory card, or the like. Hardware provided as part of an exemplary tablet computer platform may include an LCD touch screen 810 with associated display controller 806 and touch interface 808; hard keys 812 such as, for example, an on/off key; a USB 816 port; transceiver and WiFi 818 interface; a transceiver and Bluetooth 820 interface; an 822 camera; and various other features 824 appropriate for a particular modality, for example, an accelerometer, GPS, ambient light sensor, near-field communicator; etc. The electronic components comprising the exemplary tablet computer device 202 may be powered by an internal battery-based power supply 814, rechargeable, for example via the USB interface 816.

[0039] Memory 802 may include executable instructions that are intended to be executed by processor 800 to control the operation of tablet computer device 202 and implement various functionality such as web browsing, gaming, video streaming, etc. As is known in the art, programming comprising additional functionality (referred to as "apps") can be transferred to tablet computer 202, for example, via WiFi interface 818, USB 816, external memory 804, or any other convenient method. As previously discussed, such an app may comprise a remote control app, for example, as described in the same applicant's copending US Patent Application No. 13/329,940 and incorporated herein by reference in its entirety, which app may be for use in the operating command of devices 106, 108, 110 and/or 120 via the UCE device 100. In order to initially configure the UCE device 100 to correspond to the devices to be controlled and establish an appropriate command matrix, the tablet computer 202 can also be provided with a configuration app 214, either as part of a remote control app or as a separately downloadable item.

[0040] Referring now to figure 9, a configuration app like this, when invoked in step 902, may initially request that the user put all the devices to be controlled in a known state, for example, on, in order to allow device detection and/or test steps to follow. Then, in step 904, the setup app can determine the identity of those devices that are CEC-enabled. This can be done by communicating a request to the associated ECU which, in step 906, can have the ECU programming scan connected HDMI devices looking for devices that are CEC enabled and/or identifiable through interaction through the HDMI interface, for example as described in copending US Patent Application No. 13/198,072, by the same applicant and incorporated herein by reference in its entirety, and communicate such device identities to the configuration application. Then, in step 904, the setup application can determine whether additional non-CEC devices are connected to the UCE device via the HDMI interface. This can be done by asking the UCE programming to examine any additional HDMI connections in step 910 and communicate the findings back to the configuration application. Although not illustrated, it is understood that, where appropriate for a particular modality, UCE programming can conduct similar examinations in order to discover devices connected via Ethernet, USB, Bluetooth, RF4CE, WiFi, etc., where such interfaces can be provisioned to an ECU.

[0041] Next, in step 912, the configuration application can display a list of detected devices (both identified and not yet identified) to the user. In step 914, the user can be prompted to enter device identification information for that HDMI or otherwise connected devices that were detectable but not identified, as well as identifying information regarding any additional devices that may form part of the system to be be controlled, but which cannot be discovered, as described above (for example, devices such as the AV receiver 120 or CD player 408 that can only be responsive to unidirectional IR commands). Without limitation, such identifying information may take the form of user input data such as a device type, make and model number, or a configuration code from a listing in a user guide; or may take the form of scanned or electronic information, such as a digital photo of the device itself or a barcode, QR code, or the like associated with the device; near-field acquisition of RFID tag data; etc.; or any combination thereof as appropriate for a particular embodiment.

[0042] Once appropriate identification information has been acquired, in step 916, the configuration app may communicate this information to a database server, for example, server 206, for carrying out step 918, comprising identification and retrieving a set of command codes and capability data corresponding to the identified devices from a database 207, and providing this data to the configuration application for processing and final transfer to the UCE device. As can be seen, the transferred codeset data may comprise complete command data values and formatting information, may comprise pointers to command data values and formatting information already stored in memories 502 and/or 802/804 of the ECU or the device on which the configuration application is currently residing, or a combination of these. Where necessary, for example, when the database 207 may contain alternative code sets for an identified device, or where uncertainty exists regarding a particular device model number, etc., at steps 920, 922, and 924, various Control paradigms and/or command data sets can be tested against the devices to be controlled. Such testing may take the form of prompting the user to respond to the effects of observable commands, monitoring changes in the state of the HDMI interface, as described, for example, in US Patent Application No. 13/240,604, by the same applicant and incorporated herein. by reference in its entirety, or any other method convenient for a particular application. Once appropriate code sets have been fully determined, at steps 926, 928, and 930, a suitable preferred command matrix, for example, as illustrated in Figure 7, can be constructed and stored in memory 502 of exemplary UCE device 100, the matrix being built considering the communication capabilities and functionalities of the devices identified through the processes described above.

[0043] In order to select the optimal command method for each function of each configured device, any suitable method can be used, e.g. a system-oriented prioritization of command means and methods by desirability (e.g. apply IP , CEC, IR in descending order); device-specific command maps by brand and/or model; function-specific preference and/or priority maps (eg, all volume function commands via IR, where available); etc.; or any combination thereof. The exact priority selection or command method mapping can take into account factors such as connection reliability, eg wired versus wireless, bidirectional versus unidirectional communication, etc.; transmission speed or command execution; internal priorities in a device, eg incoming IP packets processed before CEC packets, etc.; type of protocol support (eg error correction versus error detection; ack/nak, etc.); or any other factor that can be applied in order to achieve optimal performance of a particular modality.

[0044] As can be seen, the construction of said preferred command matrix can be done on the database server or within the configuration application, or a combination of these, depending on the particular modality. Once a preferred command matrix has been finalized and stored on the UCE device, in step 932, a series of desired device configurations associated with specific user activities can be configured and stored on the UCE device, as will now be described.

[0045] Upon completion and storage of a preferred command matrix, an exemplary configuration application can subsequently guide a user through a series of steps in order to establish the desired device configurations for a number of possible activities. Referring to Figure 10, in step 1002, the user may be presented with a list of possible activities, e.g. “Watch TV”, “Watch a movie”, “Listen to music ” (Listening to music), etc. In some embodiments, the user may also be able to edit activity titles and/or create additional user-defined activities. In step 1004, a user can select a particular activity to configure, for example, “Watch TV”. In step 1006, the user can be prompted to identify the content source for the activity being configured, for example, STB/Cable DVR 110 for the exemplary “Watch TV” activity. An alert like this can take the form of a listing of eligible devices, determined during the presented device configuration steps; explicit user input of a device type; etc. Then, in steps 1008, the user can be prompted in a similar manner to select audio and video rendering apparatus for use in this activity, eg TV receiver 106 and AVR 120 respectively. Depending on the topography of the system and the interfaces in use (i.e. HDMI/CEC, IP, analog, etc.) the configuration application in combination with UCE programming may be able to certify which input port of each rendering device is attached to the content source device identified for this activity and/or if any intermediate switching device is in use (eg AV receiver 420 of the system illustrated in Figure 4). Where such information can be obtained, the configuration application can automatically create all or part of a rendering device input selection appropriate for the activity being configured. If not, in steps 1008 and 1010 the user may be additionally asked to identify the applicable content route(s) in the rendering appliances, e.g. input port numbers, presence of intermediate keys, etc. During or upon completion of steps 1004 to 1010, the configuration application can build an activity matrix, for example, as illustrated in Figure 11. By way of example, activity matrix 1100 for a “Watch TV” activity may comprise a series of cells, for example 1110 or 1112, each corresponding to a desired configuration of a particular state 1106 or function 1108 of a specific apparatus 1104 during the specified activity. By way of example, cell 1110 may indicate that the input of AV receiver 120 should be set to “S/PDIF2”, while cells 1112 and 1114 may indicate that transport function commands (e.g., “play” ), “pause”, “fast forward”, etc.) should be directed to STB/DVR 110, and not to DVD 114. association of functions such as, for example, volume control, with specific devices during a particular activity can be performed within an individual control device, i.e. the control device can determine the device for which volume control commands should be targeted, in a preferred modality this association can be done within the ECU, thereby ensuring consistency across each activity when multiple control devices are present in an environment, e.g. devices 102 and 104 of the environment and illustrated in figure 1.

[0046] Returning now to figure 10, in steps 1014 and 1016, the newly constructed activity matrix 1100 can be tested by having the ECU programming, using preferred command matrix 700, issue the commands necessary to place the identified devices in the desired state and then receive verification in step 1018 that the desired activity has started successfully. It is understood that such verification may comprise, for example, detection and reporting of HDMI or other content streams and/or device status by programming the UCE directly monitoring the CEC status or using methods such as described, for example, in the Request for US Patent No. 13/240,604; user input request confirming correct operation; monitoring the presence or absence of analog input signals; device status log or error messages; etc.; or any combination thereof as appropriate for a particular embodiment.

[0047] If the test is unsuccessful, at step 1018, the configuration application may return to step 1002 to allow reconfiguration of this activity and/or definition of alternative activities. If the test is successful, in steps 1020 and 1022, the completed activity matrix, for example, 1100 as illustrated in figure 11, can be transferred to the UCE 100 for storage in the memory of the UCE 502. Next, in step 1024, the user may be presented with the opportunity to return to step 1002 to configure additional activity settings, for example 1101, 1102, as illustrated in Figure 11, or exit the activity configuration process.

[0048] Referring now to Figure 13, the series of steps performed by ECU programming in order to transfer a function command to an apparatus in accordance with a command request 1300 received from a control device such as remote control 102 or 200, smart device 104 or 202, etc., or in accordance with an internally generated requirement resulting from the receipt of an activity request (as will be described below), may initially comprise retrieving a preferred command matrix of which element of data corresponding to the requested command and target device. As a specific example, receiving a “TV power on” request from remote control 102 or the like on a UEC provisioned with the preferred command matrices illustrated in Figure 7 may cause data element 720 to be recovered, indicating that the command must be communicated to the TV set, eg television 106, using an HDMI CEC command. In step 1304, the UCE programming can determine whether the retrieved value constitutes a null element. If it does, the referenced device does not support the requested command and thus, in step 1314, an error message can be generated and the process then terminated. As you can see, the exact nature of an error message like this may depend on the particular modality and/or the requesting control device: for example, if the request originates from a control device that is in bidirectional communication with the ECU, the error can be communicated back to the device requesting the action, i.e. display to the user, illuminate an LED, activate a bell, etc. properly. Alternatively, in those embodiments where an ECU is embedded in a device, the front panel display of that device can be used.

[0049] If the retrieved preferred command array element data is valid, in step 1306 the UCE may communicate the corresponding function command to the target apparatus using the indicated command value and transmission method, e.g. to the element 720 this may comprise issuing a command to “power on” the CEC to address zero of the logical device of the CEC (TV) via the HDMI interface of the UCE 508. Once the command has been issued, in step 1308, the UCE programming can determine whether the interface and communication protocol used in issuing the command provides any acknowledgment mechanism, i.e. explicit acknowledgment of receipt, monitoring the state of HDMI on an interface, detection of a media stream or HDCP communication establishment signal, etc. If not, for example, the command was issued using a unidirectional IR signal and no other acknowledgment devices such as no power monitoring or input signal are available, the ECU programming can simply assume that the command was successful and processing It's complete. If, however, means of confirmation exist, at step 1310 the UCE schedule can wait to determine if the command was successfully executed. Once the positive acknowledgment is received, processing is completed. If no acknowledgment is received, or if a negative acknowledgment is received, in step 1312, the ECU programming may determine if an alternative method is available to communicate the command to the target apparatus. Returning to the specific example presented earlier, this may comprise accessing a secondary command matrix 716 in order to determine if an alternative communication method is available for the specific function, for example, “TV power on.” If an alternative exists, at step 1316, the command value and surrogate transmission method may be retrieved and processing may return to step 1306 to initiate an alternative attempt. Back again to the specific example, if the command “power on” the CEC corresponding to data element 720 of matrix 700 issued to TV 106 cannot be confirmed, a command “power on” the encoded IR according to SIRCS (Sony Infrared Control Systems) in correspondence with the equivalent data element in the secondary matrix 716 can be tried as a substitute.

[0050] In addition to relaying individual command requests, as previously described, an exemplary ECU may also support activity selection, whereby receiving a single user request from a control device can cause a series of commands be issued to various devices in order to properly configure a system for a particular user activity, such as, for example, watching television. For this purpose, a set of matrices defining desired equipment states suitable for various activities, for example, as illustrated in 1100 to 1102 of Figure 11, can be stored in the memory of the UCE 502 for access by the UCE programming during execution of such a request. . As illustrated in Figure 12, in some embodiments, programming an exemplary ECU may maintain an additional matrix 1200 representative of the current state of the controlled devices, arranged, for example, by device 1202 and operating state 1204. By way of example, elements data sets 1206 and 1208 in the illustrative table 1200 may indicate that the TV 106 is currently on (1208) with the HDMI 2 port number selected as the input (1206). The data contents of elements in such a table can be maintained in any convenient manner appropriate for a particular modality, eg, without limitation, HDMI/CEC state recovery; monitoring input media streams and/or HDCP status; measurement of energy consumption; construction of a simulated state apparatus as described, for example, in U.S. Patent 6,784,805; etc.; or any combination thereof. In the case of certain devices, such as, for example, AV receiver 120, which may be controllable only via unidirectional IR, the current state of the device may not be discernible. In such cases, a null data element 1210 may be fed into the exemplary array 1200 to indicate that this apparatus may require configuration using only discrete commands and/or user interaction. As can be seen, in some modalities, the data contents of the illustrative table can be kept in memory 502 based on the progress by the UCE programming, while, in other modalities, this data can be collected “during the flight” at the moment where the activity request is being processed. Combinations of these methods can also be used, for example, “in-flight” collection for devices connected via an HDMI bus combined with maintenance of a simulated state for devices controlled via IR signals.

[0051] In order to configure a group of fixtures for a desired activity, UCE programming can compare a desired state matrix, eg 1100, with a current state array, eg 1200, element by element, by issuing commands needed to bring the devices to the desired state. By way of example, an exemplary series of steps that can be performed by programming an ECU in order to affect a “Watch TV” activity setting will now be presented in association with Figure 14. For purposes of this example, the reader may also wish to refer to the equipment configuration of figure 1 and the activity and current state matrices 1100 and 1200 of figures 11 and 12.

[0052] Upon receipt of a “Watch TV” request 1400, in step 1402, the exemplary UCE schedule can access an applicable appliance state matrix 1100. Then, in step 1404, it can be determined by the schedule of the ECU if the present “power” state of the TV 106 indicated by the current state matrix 1200 matches the desired stored state in the corresponding matrix data element 1100. If the states match, processing can continue at step 1408. do not match, in step 1406 a "power on" command may be communicated to the TV 106. As can be seen from the foregoing discussion in association with Figure 13 and inspection of the preferred exemplary command matrix 700, n illustrative system , communication of the “power on” command to the TV 106 may comprise a CEC command issued over the HDMI connection 112. Then, in step 1408 a “mute” command may be communicated to the TV 106, since the element 1116 of the matrix Illustrative iz 1100 indicates that TV106 is not the primary audio rendering device. According to the preferred command matrix 700, communication of the “mute” command to the TV 106 may comprise an IR transmission 114. Next, in steps 1410, 1412, the active input of the TV 106 may be set to “ HDMI1” via a CEC command and in steps 1414, 1416 a “power on” command the CEC can be communicated to the STB/DVR 110 if that device is not already on. In step 1418, the exemplary UCE programming can establish an internal state to indicate that future transport command requests (e.g., play, pause, FF, etc.) should be routed to the STB/DVR 110, as indicated by the element 1112 of array 1100. Next, in steps 1420,1422, a command “power off” the CEC can be communicated to the STB/DVR 108 if that device is not already off. Next, in steps 1424 and 1426, “power on” and “S/PDIF2 input” commands can be communicated to the AV 120 receiver via IR signals. As can be seen, it may not be possible to determine the current state of AV receiver 120, as indicated, for example, by elements 1210 and 1220 of matrix 1200, and in this way so-called "discrete" or explicit function commands may be issued which can establish the desired state independently of the current state of the device. Finally, at step 1428, the exemplary UCE programming can establish an internal state to indicate that future volume control command requests (e.g., volume up/down, mute) should be routed to AV receiver 120, as indicated by element 1118 of array 1100, after which activity request processing is completed.

[0053] As previously noted, the exemplary UCE may also support activity selection, whereby receiving a single user request from a smart device can cause a series of commands to be issued to multiple devices in order to configure a system properly for one or more user activities, such as “watch TV”, “watch movie”, “listen to music”, etc. To configure the smart device user interface to support such macro command functionality, an exemplary method is illustrated in Fig. 15. More particularly, with reference to Fig. 15, upon invoking a configuration app in step 1502, a user can be asked to place all devices to be controlled in a known state, e.g. on, in order to allow for the detection and/or test steps of the device to follow. Then, in step 1504, the configuration app can determine the identity of those devices that are CEC-enabled. This can be done by communicating a request to the associated ECU, which, in step 1506, can cause the ECU schedule to scan connected HDMI devices for devices that are CEC-enabled and/or identifiable through interaction through the HDMI interface, for example , as described in co-pending US Patent Application No. 13/198,072, by the same applicant and incorporated herein by reference in its entirety, and communicate such device identities to the configuration application. Then, in step 1508, the configuration app can also determine whether the devices contain any icon and/or icon information (metadata), as well as information - such as information regarding interface connection types, for example, input /HDMI output - for use in creating supported macros. If icon and/or icon information is available, icon and/or information data can be transmitted to the smart device, as shown in step 1526. An icon provided for the smart device can be automatically added to the user interface of the smart device. smart device, whereby an activation of the added icon can be used to provide access to the command and control functionality associated with the corresponding device, including a listing of macros available for that device, as described below. Similarly, icon information provided to the smart device can be used in connection with information stored on the smart device and/or on a remote server to automatically add an icon to the user interface of the smart device, whereby an activation of the added icon can be used to provide access to the command and control functionality associated with the corresponding device, including a listing of macros available for that device.

[0054] The configuration application then continues at step 1510 (after examining devices connected to the CEC as previously discussed) whereby the configuration application can then determine whether additional non-CEC devices are connected to the UCE device via the HDMI interface . This can be done by asking the UCE programming to examine any additional HDMI connections in step 1512 and communicate the observations back to the configuration application. Although not illustrated, it is understood that, where appropriate for a particular modality, UCE programming can conduct similar examinations in order to discover devices connected via Ethernet, USB, Bluetooth, RF4CE, WiFi, etc., where such interfaces can be used. be provisioned to an ECU.

[0055] Then, in step 1514, the configuration application can display a list of detected devices (both identified and not yet identified) to the user. In step 1516, the user may then be prompted to enter device identification information for those HDMI or otherwise connected devices that have been detected but not identified, as well as identification information relating to any additional devices that may form part of the device. system to be controlled, but which have not been discovered, as described above (for example, devices such as AV receiver 120 or CD player 408 that can be responsive only to unidirectional IR commands). Without limitation, such identifying information may take the form of user input data, such as a device type, make and model number, or a configuration code from a listing in a user guide; or may take the form of scanned or electronic information, such as a digital photo of the device itself or a barcode, QR code, or the like associated with the device; near-field acquisition of RFID tag data; etc.; or any combination thereof suitable for a particular embodiment.

[0056] Once appropriate identification information has been acquired, in step 1518, the configuration app can communicate this information to a database server, e.g., server 206, to perform step 1520 in which the server of the database uses the identification information to retrieve icons and/or icon metadata as needed (e.g. when such data cannot be obtained by the device), command information as previously discussed, and, in step 1522, automatically generate macros which correspond to the device or a plurality of devices considering their capacity data maintained in a database 207 and/or retrieved from the devices. Any such data collected and/or created by the server 206 will then be provisioned to the configuration application for processing and final transfer to the smart device and/or ECU as necessary. As can be seen, the transferred information and/or metadata may comprise complete command data values, device input/output data and current status, formatting information, pointers to command data values and formatting information already stored. in the UCE's 502 and/or 802/804 memories or the device on which the configuration application is currently residing, etc. Where necessary, for example, when the database 207 may contain alternative code sets, icon metadata, or macro information for an identified device, or where uncertainty exists with respect to a particular device model number, etc., in steps 1528, 1530 and 1522, various control paradigms and/or command data sets can be tested against the devices to be controlled. Such testing may take the form of prompting the user to respond to the effects of observable commands, monitoring changes in the state of the HDMI interface described, for example, in US Patent Application No. 13/240,604, by the same applicant and incorporated herein by reference in its entirety, or any other method convenient for a particular application. Once appropriate code sets and macro operations have been completely determined, at steps 1528 and 1530, a suitable preferred user profile 1524 can be constructed and stored in memory 502 of exemplary UCE device 100, user profile 1524 being constructed considering the communication capabilities and functionalities of the devices identified through the processes described above.

[0057] In order to select the optimal command method for each function of each configured appliance, any suitable method can be used, e.g. a system-oriented prioritization of the environment and command methods by desire (e.g. apply IP , CEC, IR in descending order); device-specific command maps by brand and/or model; function-specific preference and/or priority maps (eg, all volume function commands via IR, where available); etc.; or any combination thereof. The exact selection of command or mapping method priorities can take into account factors such as connection reliability, eg wired versus wireless, bidirectional versus unidirectional, etc.; transmission speed or command execution; internal priorities in a device, eg incoming IP packets processed before CEC packets, etc.; type of protocol support (eg error correction versus error detection; ack/nak, etc.); or any other factor that can be applied in order to achieve optimal performance of a particular modality.

[0058] As can be seen, the construction of said user profile 1524 can be done on the database server or within the configuration application, or a combination of these, depending on the particular modality.* * * * *

[0059] While several concepts have been described in detail, those skilled in the art should realize that various modifications and alternatives to these concepts could be developed in light of the general precepts of revelation. For example, in an alternative embodiment of the UCE functionality, in place of a preferred command matrix as illustrated in Figure 7, the programming of an exemplary UCE may utilize a command prioritization list, e.g. an "IP" prioritization list , CEC, IR” can cause the UCE programming to first determine whether the requested command can be issued using Internet Protocol only, if not, then determine whether the requested command can be issued using a CEC command via the HDMI interface, and then only if not, then it attempts to issue the requested command via an infrared signal. A prioritization like this reflects an exemplary preference to use two-way communication protocols over unidirectional communication protocols over line-of-sight communication protocols, eg IR, when supported by the targeted target device.

[0060] Additionally, although described in the context of functional modules and illustrated using block diagram format, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and/or features described may be integrated. on a single physical device and/or a software module, or one or more functions and/or features may be implemented on separate physical devices or software modules. It should also be noted that a detailed discussion of the actual implementation of each module is not necessary for a capacitive understanding of the invention. Rather, the actual implementation of such modules would be well within an engineer's routine capability, given the revelation here of the attributes, functionality, and interrelationship of the various functional modules in the system. Therefore, those skilled in the art, applying ordinary knowledge, will be able to practice the invention presented in the claims without undue experimentation. It should further be understood that the particular concepts disclosed are intended to be illustrative only and not limiting of the scope of the invention which is to be assigned with the full scope of the appended claims and any equivalents thereof.

[0061] All patents cited herein are hereby incorporated by reference in their entirety.

Claims (8)

1. Method for configuring a user interface of an application of the control device of a smart device (104), characterized in that it comprises: receiving in the smart device (104) from a Universal Control Engine (100) in communication with smart device (104) and a controllable device (106), information retrieved from the controllable device (106) by the Universal Control Engine (100), the information comprising data for use to make an icon representative of the controllable device (106) ) is added in the control device application user interface; Using the information received from the Universal Control Engine (100) by the smart device (104) to cause an icon representative of the controllable device (106) to be automatically added in the application user interface of the control device, the added icon being selectable to provide additional access by use of the control device application's user interface to user interface elements that are selectable to initiate control of one or more controllable functions of the controllable device through one or more communications emitted by the smart device ( 104) for the Universal Control Engine (100); wherein user interface elements that are selectable to initiate control of one or more controllable functions of the controllable apparatus (106) via one or more communications emitted by the smart device (104) for Universal Control Motor (100) made available in response to a selection of the icon added in the user interface of the smart device control device application, comprises user interface elements to place at least the controllable apparatus (106) in a desired state for use with respect to an activity; wherein the retrieved information of the controllable apparatus (106) by the Universal Control Engine (100) further comprises data representative of a capability supported by the controllable apparatus (106), wherein the data representative of a capability supported by the controllable apparatus (106) comprises data representative of a type controllable device interface connection (106); wherein the smart device (104) causes a user interface element for an activity that uses a controllable device interface connection type (106) to be automatically added to the smart device control device application user interface; and wherein the smart device (104) causes a sequence of macro commands to be automatically created for the activity that uses controllable device (106) interface connection type automatically added in the user interface of the smart device control device application ( 104), the sequence of macro commands being executed in response to a user interface element selection for the activity. 2. Method according to claim 1, characterized in that the data for use to cause an icon representative of the controllable device to be added in the user interface of the application of the control device comprises metadata. 3. Method according to claim 2, characterized in that the smart device (104) uses the metadata to create the icon representative of the controllable device (106) added in the user interface of the control device application. 4. Method, according to claim 3, characterized in that the smart device (104) uses data stored in a memory of the smart device to create, from the metadata, the icon representing the controllable device (106) added in the Control device application user interface. 5. Method according to claim 3, characterized in that the smart device (104) communicates the metadata to a server device (206), the server device (206) uses the metadata to create the icon representative of the controllable device (106), the server device (206) communicates the created icon to the smart device (104), and the smart device (104) automatically adds the created icon in the user interface of the smart device (104) remote control application. 6. Method according to claim 1, characterized in that the sequence of macro commands is executed by the Universal Control Engine (100). 7. Method according to claim 1, characterized in that the smart device (104) communicates data representative of a capability supported by the controllable device (106) to a server device (206), the server device (206) uses data representative of a capability supported by the controllable device to automatically create the macro-command sequence for the activity that uses the capability of the controllable device (106), and the server device (206) communicates the created macro-command sequence to the smart device (104). ) whereby the sequence of macro commands created is available for execution in response to a selection of the user interface element. 8. Method according to claim 7, characterized in that the sequence of macro commands is executed by the Universal Control Engine (100).

Images