US20220103428A1 - Automatic determination of display device functionality - Google Patents
Automatic determination of display device functionality Download PDFInfo
- Publication number
- US20220103428A1 US20220103428A1 US17/494,567 US202117494567A US2022103428A1 US 20220103428 A1 US20220103428 A1 US 20220103428A1 US 202117494567 A US202117494567 A US 202117494567A US 2022103428 A1 US2022103428 A1 US 2022103428A1
- Authority
- US
- United States
- Prior art keywords
- signal
- remote control
- display device
- command
- system remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims description 34
- 230000004044 response Effects 0.000 abstract description 13
- 238000012544 monitoring process Methods 0.000 abstract description 12
- 230000009471 action Effects 0.000 abstract description 11
- 238000004519 manufacturing process Methods 0.000 abstract description 5
- 238000004590 computer program Methods 0.000 abstract description 4
- 238000003860 storage Methods 0.000 description 24
- 238000004891 communication Methods 0.000 description 10
- 238000012360 testing method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003116 impacting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/085—Retrieval of network configuration; Tracking network configuration history
- H04L41/0853—Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/189—Automatic justification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
- H04B10/114—Indoor or close-range type systems
- H04B10/1143—Bidirectional transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0233—Object-oriented techniques, for representation of network management data, e.g. common object request broker architecture [CORBA]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0811—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking connectivity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C2201/00—Transmission systems of control signals via wireless link
- G08C2201/40—Remote control systems using repeaters, converters, gateways
- G08C2201/42—Transmitting or receiving remote control signals via a network
Definitions
- This disclosure is generally directed to automatically determining functionality supported by electronic devices.
- a user's entertainment system includes components that have different functionality and capabilities.
- CEC Consumer Electronics Control
- a user's entertainment system may have some components that support CEC, and others that do not support CEC.
- even components that are CEC-enabled may support only a portion of the CEC command set.
- the user may be able to use the remote control to perform only some functions. Users may be frustrated by such inconsistencies and limits, thereby negatively impacting enjoyment of their entertainment systems.
- system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for automatically determining the functionality and capabilities of electronic components For illustrative purposes, embodiments are described with respect to an example of controlling display devices in a media streaming environment. However, this disclosure is not limited to these example embodiments. Instead, the functions and structures described herein are applicable to automatically determining the functionality and capabilities of any electronic components in any environment or application.
- Some embodiments operate by transmitting a command to the display device in question (sometimes called the device under test—DUT—herein) and monitoring the display device. Then, it is determined whether a subsequent action by the display device was one of a set of proper responses to the command. If the action was proper, then it is determined that the display device supports the command set associated with the command.
- the command set may be the Consumer Electronics Control (CEC) command set, although this disclosure is not limited to that example.
- CEC Consumer Electronics Control
- the command transmitted to the display device may be an audible control command (that is, a command that impacts the audio of the display device), such as Volume Up or Down, Mute, Power Off, etc.
- a microphone or similar means may be used to detect volume changes made by the display device in response to the audible control command.
- the command may be a CEC command transmitted over a HDMI link.
- traffic on the HDMI link may be monitored for messages transmitted by the display device.
- This disclosure also includes embodiments for programming a system remote control for a display device, wherein the display device does not natively support the system remote control.
- Such embodiments operate by capturing infrared (IR) signals transmitted by a remote control that is supported by the device, and then monitoring for and determining one or more actions performed by the display device.
- the captured IR signals are associated with the determined actions, and the system remote control is programmed with the captured IR signals. Thereafter, the system remote control can be used by users to command the display device to perform the determined actions using the captured IR signals.
- IR infrared
- FIG. 1 illustrates a block diagram of a multimedia environment, according to some embodiments.
- FIG. 2 illustrates a block diagram of a media device, according to some embodiments.
- FIG. 3 illustrates a block diagram of a system remote control, according to some embodiments.
- FIG. 4 illustrates a method for automatically determining the functionality and capabilities of a device under test (DUT) by monitoring how the DUT responds to commands, according to some embodiments.
- DUT device under test
- FIG. 5 illustrates a method for automatically determining the functionality and capabilities of a DUT by detecting and processing audible outputs of the DUT generated in response to commands, according to some embodiments.
- FIG. 6 illustrates a method for automatically determining the functionality and capabilities of a DUT by monitoring traffic on a bus to determine how the DUT responds to commands, according to some embodiments.
- FIG. 7A illustrates a method for automatically determining the functionality and capabilities of a DUT by fingerprinting and crowdsourcing the DUT, according to some embodiments.
- FIG. 7B illustrates an example fingerprint data structure, according to some embodiments.
- FIG. 7C illustrates an example device remote control database, according to some embodiments.
- FIGS. 8 and 9 illustrate methods for determining infrared signals transmitted by a remote control and associated with commands of interest, according to some embodiments.
- FIG. 10 illustrates an example computer system useful for implementing various embodiments.
- FIG. 1 illustrates a block diagram of a multimedia environment 102 , according to some embodiments.
- multimedia environment 102 is directed to streaming media.
- this disclosure is not limited to that example embodiment.
- the multimedia environment 102 may include one or more media systems 104 , one or more content servers 122 and/or one or more crowdsource servers 114 communicatively coupled via a network 120 .
- the network 120 can include, without limitation, wired and/or wireless intranet, extranet, Internet, cellular, Bluetooth and/or any other short range, long range, local, regional, ad hoc, and/or global communications network, as well as any combination thereof.
- Media system 104 may include a display device 106 , media device 108 , system remote control 110 and/or display device remote control 140 .
- Display device 106 may be a monitor, television, computer, smart phone, tablet, and/or projector, to name just a few examples.
- Media device 108 may be a streaming media device, DVD device, audio/video playback device, cable box, set top box, and/or digital video recording device, to name just a few examples.
- the media device 108 can be a part of, integrated with, operatively coupled to, and/or connected to display device 106 .
- the media device 108 may be configured to communicate with network 120 .
- Content servers 122 may each include databases to store content 124 and metadata 126 .
- Content 124 may include any combination of music, videos, movies, TV programs, multimedia, images, still pictures, text, graphics, gaming applications, advertisements, software, and/or any other content or data objects in electronic form.
- metadata 126 comprises data about content 124 .
- metadata 126 may include associated or ancillary information indicating or related to writer, director, producer, composer, artist, actor, summary, chapters, production, history, year, trailers, alternate versions, related content, applications, and/or any other information pertaining or relating to the content 124 .
- Metadata 126 may also or alternatively include links to any such information pertaining or relating to the content 124 .
- Metadata 126 may also or alternatively include one or more indexes of content 124 , such as but not limited to a trick mode index.
- FIG. 2 illustrates an example block diagram of the media device 108 , according to some embodiments.
- Media device 108 may include a streaming module 202 , processing module 204 , user interface module 206 , database 208 , microphone 212 , camera 214 and/or IR transceiver 216 .
- FIG. 3 illustrates an example block diagram of the system remote control 110 , according to some embodiments.
- System remote control 110 may include a user interface 302 , storage module 304 , microphone 305 , IR transceiver 306 , camera 308 and/or IR processing module 310 .
- user 112 may use system remote control 110 to interact with the user interface module 206 of media device 108 to select content, such as a movie, TV show, music, book, application, game, etc.
- the streaming module 202 of media device 108 may request the selected content from content server(s) 122 over the network 120 .
- Content server(s) 122 may transmit the requested content to the streaming module 202 .
- Media device 108 may transmit the received content to display device 106 for presentation to user 112 .
- the streaming module 202 may transmit the content to display device 106 in real time or near real time as it receives such content from content server(s) 122 .
- media device 108 may buffer or store the content received from content server(s) 122 in database 208 for later playback on display device 106 .
- media system 104 may include a display device remote control 140 in addition to system remote control 110 .
- Display device remote control 140 may be specific to (i.e., native to) the display device 106 , to control display device 106 (and may not be capable of controlling other devices such as media device 108 ).
- the display device remote control 140 may be the remote control that was included when the user 112 purchased the television.
- the display device remote control 140 may transmit commands from a command set supported by the display device 106 , to control the display device 106 .
- an infrared (IR) transmitter 142 in the display device remote control 140 may transmit such commands as IR signals to an IR receiver 107 in the display device 107 .
- IR infrared
- other devices in multimedia environment 102 may have IR transmitters, receivers and/or transceivers, such as media device 108 , system remote control 110 , etc.
- system remote control 110 can be configured to control multiple devices in media system 104 , such as but not limited to media device 108 and display device 106 .
- System remote control 110 is able to achieve this functionality even though some devices, such as display device 106 (as well as other devices in media system 104 ) may support different (or at least partially different) command sets. At times herein, system remote control 110 is said to be non-native to the display device 106 .
- user 112 may be required to manually program system remote control 110 to work with different devices in media system 104 (such as display device 106 , for example). Assume display device 106 is a television, for example. In some embodiments, user 112 may be required to identify the command set supported by the television (by looking in the user manual or searching on-line, for example), and then input into the system remote control 110 the code associated with that command set. However, this can be a difficult and frustrating process for many users, especially users who are not technically inclined.
- media device 108 and/or crowdsource server 114 may automatically determine the functionality and capabilities of the display device 106 and/or other devices in media system 104 . This may include determining the respective command sets of the devices in media system 104 . Then, media device 108 may automatically configure system remote control 110 to operate with and control display device 106 and/or other devices in media system 104 . Accordingly, embodiments solve the technical problem of enabling a single remote control to operate with and control multiple non-native devices in a media environment, where the command sets supported by the devices may differ, and/or may be unknown to users 112 . Such features and advantages are achieved by causing media device and/or crowdsource server 114 to operate according to instructions representative of the operations described herein (such as shown in the flowcharts of FIGS. 4-9 ).
- FIG. 4 illustrates a method 402 for automatically determining the functionality and capabilities of a device under test (DUT) by monitoring how the DUT responds to commands, according to some embodiments.
- Method 402 may be useful when it is not known what command set(s), if any, are supported by the DUT.
- Method 402 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 4 , as will be understood by a person of ordinary skill in the art.
- processing logic can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 4 , as will be understood by a person of ordinary skill in the art.
- Method 402 shall be described with reference to FIGS. 1 and 2 . However, method 402 is not limited to that example embodiment.
- a command from a command set is transmitted to a DUT.
- media device 108 may transmit a command to the display device 106 over a link 105 .
- link 105 may be an HDMI (High-Definition Multimedia Interface) link and the command may be from the Consumer Electronics Control (CEC) control set.
- CEC Consumer Electronics Control
- CEC is a feature of HDMI that allows users to command and control devices connected through HDMI using a single remote control.
- signals transmitted by the DUT are monitored.
- media device 108 may monitor link 105 for signals transmitted by the display device 106 .
- media device 108 may determine if signals received over link 105 from display device 106 are appropriate given the CEC command transmitted in 404 and the syntax of the associated command set (that is, the CEC command set).
- the DUT responded as expected to the command
- the media device 108 may program or otherwise configure the system remote control 110 to interact with the display device 106 using the command set.
- the media device 108 may transmit the command set to the system remote control 110 using its IR transceiver 216 .
- the system remote control 110 may receive this command set using its IR transceiver 306 , and store the received command set in the storage module 304 .
- the received command set may be in the form of information identifying one of a number of command sets previously stored in the storage module 304 of the system remote control 110 , or may be information specifying IR signals corresponding to commands in the command set.
- the system remote control 110 is enabled to directly command and control the display device 106 by using its IR transceiver 306 to transmit IR signals to the display device 106 .
- the media device 108 commands and controls the DUT using commands from that command set. For example, assume it is determined in 410 that the DUT supports CEC. Then in 410 , the media device 108 may thereafter control the display device 106 using CEC commands transmitted over the HDMI link 105 , where such CEC commands correspond to commands issued by the system remote control 110 , and where such commands issued by the system remote 110 are received and understood by the media device 108 . For example, assume the user 112 uses the system remote control 110 to transmit a “volume up” command to the display device 106 .
- the media device 108 may receive and recognize the “volume up” command. In response, the media device 108 may transmit a CEC “volume up” command to the display device 106 over the HDMI link 105 , to thereby cause the display device 106 to increase volume.
- step 412 it is determined whether there are more tests to perform to help identify a command set supported by the DUT. If there are more tests, then those tests are performed in step 414 . For example, if there are other commands from the same or different command sets to interrogate the DUT with, then step 414 is implemented by returning control to step 404 . Alternatively or additionally, if there are different tests or methods to perform—such as those described herein—then step 414 is implemented by performing those tests/methods.
- 416 it is determined the DUT does not support a command set currently recognized by media system 104 . In this case, it is not possible to program or use system remote control 110 to control the display device 106 . Instead, currently, the user 112 will be required to use the display device remote control 140 to control the display device 106 .
- the command set supported by the display device 106 may be recognized in the future. Accordingly, the media device 108 may perform flowchart 402 in the future to attempt to identify the command set supported by the display device 106 and enable the system remote control 110 to control the display device 106 .
- FIG. 5 illustrates an example implementation of the method 402 of FIG. 4 .
- FIG. 5 illustrates a method 502 for automatically determining the functionality and capabilities of a DUT by detecting and processing audible outputs of the DUT generated in response to commands, according to some embodiments.
- audible includes sounds capable of being heard by humans, and sounds not capable of being heard by humans.
- Method 502 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein.
- Method 502 shall be described with reference to FIGS. 1-3 . However, method 502 is not limited to that example embodiment.
- media device 108 may transmit a CEC “volume up” command to the display device 106 over link 105 .
- link 105 is an HDMI link.
- CEC and the CEC “volume up” command are used herein for illustrative purposes only, and this disclosure is not limited to that example embodiment.
- Method 502 is applicable to any other command set and any other command within such other command set, as long as there is a finite set of valid, audible responses to the command made by the DUT (that is, the display device 106 ).
- Step 504 generally corresponds to step 404 in FIG. 4 .
- media device 108 may monitor the audio outputs of display device 106 using its microphone 212 , or microphone 305 in the system remote control 110 .
- Step 506 generally corresponds to step 406 in FIG. 4 .
- media device 108 may determine if the display device 106 responded in a proper or valid way to the command issued in step 504 , by analyzing the audio outputs of the display device 106 . For example, display device 106 may determine if the display device 106 increased its volume after the CEC “volume up” command was issued in step 504 . Step 508 generally corresponds to step 408 in FIG. 4 .
- step 510 the media device 108 may conclude that the display device 106 supports CEC. Thereafter, the system remote control 110 and/or the media device 108 may command and control the display device 106 using CEC commands, as discussed above with reference to step 410 of FIG. 4 .
- steps 512 , 514 and 516 correspond to steps 412 , 414 and 416 of FIG. 4 .
- FIG. 6 illustrates an example implementation of the method 402 of FIG. 4 .
- FIG. 6 illustrates a method 602 for automatically determining the functionality and capabilities of a DUT by monitoring traffic on a bus to determine how the DUT responds to commands, according to some embodiments.
- Method 602 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 6 , as will be understood by a person of ordinary skill in the art. Method 602 shall be described with reference to FIGS. 1 and 2 . However, method 602 is not limited to that example embodiment.
- media device 108 may transmit a CEC “sleep” command to the display device 106 over link 105 . Then, the media device 108 may transmit a CEC “give device power status” command to the display device 106 over link 105 .
- link 105 is an HDMI link. It is noted that CEC and the CEC “sleep” and “give device power status” commands are used herein for illustrative purposes only, and this disclosure is not limited to that example embodiment.
- Method 602 is applicable to any other command set and any other command(s) within such other command set, as long as there is a finite set of valid responses to the command(s) that are transmitted by the DUT (that is, the display device 106 ) over the link 105 .
- Step 604 generally corresponds to step 404 in FIG. 4 .
- media device 108 may monitor the link 105 for responses by the display device 106 to the sequence of CEC “sleep” and “give device power status” commands.
- Step 606 generally corresponds to step 406 in FIG. 4 .
- media device 108 may determine if the display device 106 responded in a proper or valid way to the command(s) issued in step 604 , by analyzing the traffic over the link 105 . For example, media device 108 may determine that the display device 106 responded properly if it transmitted a CEC “report power status” message over the HDMI link 105 . As will be appreciated by persons skilled in the art, in some embodiments, the “report power status” message may be a proper response to the “give device power status” when the interrogated device has been put to sleep pursuant to the “sleep” command. Step 608 generally corresponds to step 408 in FIG. 4 .
- step 610 the media device 108 may conclude that the display device 106 supports CEC. Thereafter, the system remote control 110 and/or the media device 108 may command and control the display device 106 using CEC commands, as discussed above with reference to step 410 of FIG. 4 .
- steps 612 , 614 and 616 correspond to steps 412 , 414 and 416 of FIG. 4 .
- FIG. 7A illustrates a method 702 for automatically determining the functionality and capabilities of a DUT by fingerprinting and crowdsourcing the DUT.
- Method 702 may be part of flowchart 402 of FIG. 4 —as step 414 , for example—or may be stand alone and distinct from flowchart 402 .
- Method 702 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 7A , as will be understood by a person of ordinary skill in the art. Method 702 shall be described with reference to FIGS. 1 and 2 . However, method 702 is not limited to that example embodiment.
- data may be collected on the DUT for purpose of generating an identification (sometimes called a fingerprint herein) of the DUT.
- the data collected in step 704 represents attributes, features and/or characteristics of the DUT.
- the media device 108 may collect such data on the display device 106 .
- the media device 108 may query or otherwise interact with the display device 106 over link 105 using well known standards or interfaces to collect data on the display device 106 .
- Such well known standards or interfaces may include CEC and extended display identification data (EDID), for example.
- CEC was discussed above.
- EDID is a standard published by the Video Electronics Standards Association (VESA) that defines an interface/data structure that can be used by a digital display (such as display device 106 ) to describe its capabilities to a content source (such as media device 108 ).
- Data collected in step 704 may include responses to CEC and/or EDID commands/messages and/or timing data of the latency of responding to such commands/messages.
- data collected in step 704 may include timing relating to the HDCP negotiation process between the media device 108 and the display device 106 .
- HDCP high-bandwidth digital content protection
- HDCP is a form of digital copy protection to prevent copying of digital audio and video content as it travels across connections.
- data collected in step 704 may include the model number of the display device 106 .
- the user 112 may be prompted to enter the model number of the display device 106 using the user interface 302 (i.e., a keypad or touch screen) of the system remote control 110 .
- the user 112 may be prompted to take a picture of the model number of the display device 106 using the camera 308 of the system remote control 110 (or any other handheld computing device, such as a smart phone).
- the media device 108 and/or crowdsource server 114 may then identify the model number of the display device 106 by applying character recognition techniques to the picture.
- the media device 108 may transmit this collected data to the crowdsource server 114 via network 120 .
- the media device 108 may also transmit to the crowdsource server 114 information indicating whether or not the display device 106 supports CEC (as determined in steps 410 , 510 and/or 610 , as described above), and/or information specifying IR signals corresponding to remote control commands supported by the display device 106 (examples for determining such IR signals are described below with reference to FIGS. 8 and 9 ).
- a fingerprint of the DUT may be generated using the data collected in 704 .
- the data collected in step 704 represents attributes, features and/or characteristics of the display device 106 .
- the collected data may be ordered and/or combined to generate a fingerprint of the display device 106 .
- FIG. 7B illustrates an example data structure 720 of a fingerprint, according to some embodiments.
- Fingerprint data structure 720 may include a model number field 722 , a CEC data field 724 , an EDID data field 726 , a HDCP data field 728 , as well as other fields corresponding to data collected in step 704 .
- the fingerprint of display device 106 is generated by processing module 204 of the media device 108 , and then transmitted to the crowdsource server 114 over network 120 .
- the data collected in step 704 is transmitted to the crowdsource server 114 over network 120 (as part of step 704 ), and a fingerprint module 130 in the crowdsource server 114 generates the fingerprint of display device 106 .
- the crowdsource server 114 uses the fingerprint to determine if the display device 106 is represented in a device remote control database 132 .
- FIG. 7C illustrates an example device remote control database 132 , according to some embodiments.
- Each unique fingerprint has a row 746 in the device remote control database 132 .
- a fingerprint's row 746 may store (1) the fingerprint in a “Fingerprint” column 740 , (2) whether or not the DUT corresponding to the fingerprint supports CEC (or other standards or command sets) in a “CEC Supported” column 742 , and/or (3) information specifying IR signals of remote control commands supported by the DUT in a “Remote Control Signals” column 744 .
- fingerprints B, B′ and B′′ in rows 746 B, 746 c and 746 D.
- Different media devices 108 in different media systems 104 may use the same display device 106 (that is, the same television, for example). But, these different media devices 108 may have collected different data on the display device in step 704 , resulting in the generation of different fingerprints in step 706 .
- the different data may be the result of variabilities in the televisions (some may be older, newer, refurbished, damaged, etc.), and/or the gathering of different information by the media devices 108 in step 704 .
- the same television model may have multiple entries in the device remote control database 132 .
- the crowdsource server 114 performs step 708 by matching the fingerprint generated in step 706 with the fingerprints in the Fingerprint column 740 of the rows 746 of the device remote control database 132 . If no match exists, then in step 710 , the crowdsource server 114 determines that the DUT is not currently represented in the device remote control database 132 . Accordingly, in step 712 , the crowdsource server 114 adds a new row 746 to the device remote control database 132 . The crowdsource server 114 inserts the fingerprint from step 706 in the Fingerprint column 740 of the new row 746 .
- the crowdsource server 114 inserts information indicating whether or not the DUT supports CEC (if provided in step 704 ) in the CEC Supported column 742 of the new row 746 .
- the crowdsource server 114 inserts information specifying IR signals corresponding to remote control commands supported by the DUT (if provided in step 704 ) into the Remote Control Signals column 744 of the new row 746 .
- the crowdsource server 114 determines that the DUT is already represented in the device remote control database 132 . Accordingly, in step 714 , the crowdsource server 114 retrieves from the matching row 746 information indicating whether or not the DUT supports CEC from the CEC Supported column 742 of the matching row 746 (if such information is present in the matching row 746 ), and information specifying IR signals corresponding to remote control commands supported by the DUT from the Remote Control Signals column 744 of the matching row 746 (if such information is present in the matching row 746 ).
- the crowdsource server 114 provides such retrieved information to the media device 108 from step 704 .
- the Media device 108 may use such information to control the display device 106 .
- the media device 108 may thereafter control the display device 106 using CEC commands transmitted over the HDMI link 105 , where such CEC commands correspond to commands issued by the system remote control 110 , and where such commands issued by the system remote 110 are received and understood by the media device 108 .
- the user 112 uses the system remote control 110 to transmit a “volume up” command to the display device 106 .
- the media device 108 may receive and recognize the “volume up” command. In response, the media device 108 may transmit a CEC “volume up” command to the display device 106 over the HDMI link 105 , to thereby cause the display device 106 to increase volume.
- the media device 108 may use its IR transceiver 216 to transmit this information to the IR transceiver 306 in the system remote control 110 .
- the system remote control 110 may store the received information in the storage module 304 . In this manner, the media device 108 can program the system remote control 110 to directly communicate with and control the display device 106 using the stored IR signals (which are supported by the display device 106 ).
- the crowdsource server 114 updates the CEC Supported column 742 of the matching row 746 with information indicating whether or not the DUT supports CEC (if provided in step 704 ).
- the crowdsource server 114 also updates the Remote Control Signals column 744 of the matching row 746 with information specifying IR signals corresponding to remote control commands supported by the DUT (if provided in step 704 ). In some embodiments, such updates add to rather than replace the information already existing in the matching row.
- whether or not the DUT supports CEC may be set by a majority vote of the data received in step 704 or by some other heuristic applied to the data received in step 704 from media devices 108 (for example, if 5 media devices 108 indicate the DUT supports CEC, and 3 indicate the DUT does not support CEC, then the crowdsource server 114 indicates in the CEC Supported column 742 of the matching row 746 that the DUT supports CEC).
- the IR signals in the Remote Control Signals column 742 may be populated from data received in step 704 from multiple media devices 108 , with conflicts resolved by majority vote (for example, if 2 media devices 108 indicate the volume up IR code is ABC, and 3 indicate the IR code is XYZ, then XYZ is saved in the Remote Control Signals column 742 of the matching row 746 ) or by some other heuristic. In this manner, information in the device remote control database 132 is continually updated and improved by crowdsourcing information from multiple media devices 108 in multiple media systems 104 .
- FIG. 8 illustrates a method 802 for determining infrared signals transmitted by a remote control and associated with commands of interest, according to some embodiments.
- Method 802 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 8 , as will be understood by a person of ordinary skill in the art. Method 802 shall be described with reference to FIGS. 1-3 . However, method 802 is not limited to that example embodiment.
- user 112 points the IR transmitter 142 of the display device remote control 140 to the camera 308 of the system remote control 110 (or any other computing device having the functionality of the system remote control 110 , such as a smart phone, tablet computer, laptop computer, digital assistant, etc.).
- the user 112 is instructed to issue a command using the display device remote control 140 .
- the instruction may be transmitted by the media device 108 and displayed on a screen of the system remote control 110 or on the display device 106 .
- the user 112 may issue the command by pressing a button on the display device remote control 140 . Assume, for purposes of illustration, the user 112 presses the Volume Up button on the display device remote control 140 .
- the camera 308 of the system remote control 110 captures the IR signal transmitted by the IR transmitter 142 of the display device remote control 140 .
- the camera 308 of the system remote control 110 may have IR filters or other well-known functional and/or structural features to facilitate the capture of IR signals. In other embodiments, the camera 308 may lack some or all of those features. For example, the shutter speed of the camera 308 may be too slow to fully capture an IR signal in a single transmission. Accordingly, in step 810 , the IR processing module 310 determines if the IR signal was fully captured in step 808 . If the IR signal was not fully captured, then in step 812 , the user 112 is requested to issue the same command using the display device remote control 140 . The IR signal is again captured by the camera 308 in step 808 , and then combined with previous captures of the IR signal. Steps 808 - 812 are repeated until the IR processing module 310 in step 810 determines that the IR signal corresponding to the command of step 806 has been fully captured.
- control moves to 814 .
- the media device 108 or the system remote control 110 determines if there are more commands of the display device remote control 140 that need to be captured. If there are more commands to capture, then control returns to step 804 .
- the system remote control 110 transmits the IR signals corresponding to commands of the display device remote control 140 (which were collected in steps 804 - 814 ) to the media device 108 . Such transmission may occur as each IR signal is collected, or as batches of the IR signals.
- the media device 108 may then transmit these IR signals to the crowdsource server 114 in step 704 of FIG. 7A , as described above.
- the system remote control 110 may also save these IR signals in the storage module 304 .
- the system remote control 110 may use the IR signals stored in the storage module 304 to directly communicate with and control the display device 106 .
- the system remote control 110 which is not natively supported by the display device 106 —is configured/programmed to directly communication with and control the display device 106 .
- FIG. 8 requires the involvement of the user 112 to detect the IR codes of the display device remote control 140 .
- FIG. 9 illustrates a method 902 for determining IR signals transmitted by the display device remote control 140 , without the need of user involvement.
- Method 902 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 9 , as will be understood by a person of ordinary skill in the art. Method 902 shall be described with reference to FIGS. 1-3 . However, method 902 is not limited to that example embodiment.
- a high-gain, wideband IR sensor/detector 109 monitors and captures IR signals transmitted by the display device remote control 140 .
- the wideband IR detector 109 includes IR filters or other well-known functional and/or structural features to facilitate the capture of IR signals.
- the wideband IR detector 109 is positioned proximate to the display device 106 and/or the media device 108 .
- the wideband IR detector 109 transmits the captured IR signal to the media device 108 .
- the media device 108 identifies the action(s) taken by the display device 106 after transmission of the IR signals by the display device remote control 140 in step 904 .
- the media device 108 may perform 906 by monitoring the CEC traffic on the HDMI link 105 , or listening on microphone 212 for audible actions taken by the display device (such as turning off or changing the volume level, for example).
- the media device 108 may associate the IR signals captured in step 904 with the action(s) identified in 906 . In other words, the media device 108 may determine that the IR signals are associated with a command of the display device remote control 140 to cause the display device 106 to perform the identified action.
- Steps 904 - 908 may be periodically or continuously performed to identify the IR signals associated with the command set supported by the display device remote control 140 and the display device 106 .
- the media device 108 may transmit information specifying these IR signals associated with the command set of the display device remote control 140 and the display device 106 to the system remote control 110 .
- the system remote control 110 may store such information in its storage module 304 .
- the system remote control 110 which is not natively supported by the display device 106 —is configured/programmed to directly communication with and control the display device 106 .
- the media device 108 may also transmit these IR signals to the crowdsource server 114 as part of step 704 of FIG. 7A , as described above.
- system remote control 110 may use these stored IR signals to directly communicate with and control the display device 106 .
- Computer system 1000 can be any computer or computing device capable of performing the functions described herein.
- one or more computer systems 1000 can be used to implement any embodiments of FIGS. 1-9 , and/or any combination or sub-combination thereof.
- Computer system 1000 includes one or more processors (also called central processing units, or CPUs), such as a processor 1004 .
- processors also called central processing units, or CPUs
- Processor 1004 is connected to a communication infrastructure or bus 1006 .
- One or more processors 1004 can each be a graphics processing unit (GPU).
- a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications.
- the GPU can have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.
- Computer system 1000 also includes user input/output device(s) 1003 , such as monitors, keyboards, pointing devices, etc., that communicate with communication infrastructure 1006 through user input/output interface(s) 1002 .
- user input/output device(s) 1003 such as monitors, keyboards, pointing devices, etc.
- communication infrastructure 1006 such as keyboards, pointing devices, etc.
- Computer system 1000 also includes a main or primary memory 1008 , such as random access memory (RAM).
- Main memory 1008 can include one or more levels of cache.
- Main memory 1008 has stored therein control logic (i.e., computer software) and/or data.
- Computer system 1000 can also include one or more secondary storage devices or memory 1010 .
- Secondary memory 1010 can include, for example, a hard disk drive 1012 and/or a removable storage device or drive 1014 .
- Removable storage drive 1014 can be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.
- Removable storage drive 1014 can interact with a removable storage unit 1018 .
- Removable storage unit 1018 includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data.
- Removable storage unit 1018 can be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device.
- Removable storage drive 1014 reads from and/or writes to removable storage unit 1018 in a well-known manner.
- secondary memory 1010 can include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system 1000 .
- Such means, instrumentalities or other approaches can include, for example, a removable storage unit 1022 and an interface 1020 .
- the removable storage unit 1022 and the interface 1020 can include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.
- Computer system 1000 can further include a communication or network interface 1024 .
- Communication interface 1024 enables computer system 1000 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number 1028 ).
- communication interface 1024 can allow computer system 1000 to communicate with remote devices 1028 over communications path 1026 , which can be wired and/or wireless, and which can include any combination of LANs, WANs, the Internet, etc. Control logic and/or data can be transmitted to and from computer system 1000 via communication path 1026 .
- a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device.
- control logic software stored thereon
- control logic when executed by one or more data processing devices (such as computer system 1000 ), causes such data processing devices to operate as described herein.
- references herein to “one embodiment,” “an embodiment,” “an example embodiment,” or similar phrases indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other.
- Coupled can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Theoretical Computer Science (AREA)
- Electromagnetism (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Selective Calling Equipment (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
Description
- This application is a continuation of U.S. Nonprovisional patent application Ser. No. 16/653,118 (Attorney Docket No. 3634.0980002), filed Oct. 15, 2019, which is a continuation of U.S. Pat. No. 10,447,537 (Attorney Docket No. 3634.0980000), filed Jul. 11, 2017, issued on Oct. 15, 2019, both titled “Automatic Determination Of Display Device Functionality,” the contents of both are hereby incorporated herein by reference in its entirety.
- This disclosure is generally directed to automatically determining functionality supported by electronic devices.
- Often, a user's entertainment system includes components that have different functionality and capabilities. Consider Consumer Electronics Control (CEC), which is a feature of HDMI (High-Definition Multimedia Interface) that allows users to command and control devices connected through HDMI using a single remote control. A user's entertainment system may have some components that support CEC, and others that do not support CEC. In such systems, it is difficult if not impossible for the user to use a single remote control to control all the components in his entertainment system. This is especially true if it is not apparent to the user which components are CEC-enabled, and which are not. Also, even components that are CEC-enabled may support only a portion of the CEC command set. Thus, even for CEC-enabled components, the user may be able to use the remote control to perform only some functions. Users may be frustrated by such inconsistencies and limits, thereby negatively impacting enjoyment of their entertainment systems.
- Provided herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for automatically determining the functionality and capabilities of electronic components. For illustrative purposes, embodiments are described with respect to an example of controlling display devices in a media streaming environment. However, this disclosure is not limited to these example embodiments. Instead, the functions and structures described herein are applicable to automatically determining the functionality and capabilities of any electronic components in any environment or application.
- Some embodiments operate by transmitting a command to the display device in question (sometimes called the device under test—DUT—herein) and monitoring the display device. Then, it is determined whether a subsequent action by the display device was one of a set of proper responses to the command. If the action was proper, then it is determined that the display device supports the command set associated with the command. The command set may be the Consumer Electronics Control (CEC) command set, although this disclosure is not limited to that example.
- In some embodiments, the command transmitted to the display device may be an audible control command (that is, a command that impacts the audio of the display device), such as Volume Up or Down, Mute, Power Off, etc. In this example, a microphone or similar means may be used to detect volume changes made by the display device in response to the audible control command.
- In some embodiments, the command may be a CEC command transmitted over a HDMI link. In this example, traffic on the HDMI link may be monitored for messages transmitted by the display device.
- This disclosure also includes embodiments for programming a system remote control for a display device, wherein the display device does not natively support the system remote control. Such embodiments operate by capturing infrared (IR) signals transmitted by a remote control that is supported by the device, and then monitoring for and determining one or more actions performed by the display device. The captured IR signals are associated with the determined actions, and the system remote control is programmed with the captured IR signals. Thereafter, the system remote control can be used by users to command the display device to perform the determined actions using the captured IR signals.
- This Summary is provided merely for purposes of illustrating some example embodiments to provide an understanding of the subject matter described herein. Accordingly, the above-described features are merely examples and should not be construed to narrow the scope or spirit of the subject matter in this disclosure. Other features, aspects, and advantages of this disclosure will become apparent from the following Detailed Description, Figures, and Claims.
- The accompanying drawings are incorporated herein and form a part of the specification.
-
FIG. 1 illustrates a block diagram of a multimedia environment, according to some embodiments. -
FIG. 2 illustrates a block diagram of a media device, according to some embodiments. -
FIG. 3 illustrates a block diagram of a system remote control, according to some embodiments. -
FIG. 4 illustrates a method for automatically determining the functionality and capabilities of a device under test (DUT) by monitoring how the DUT responds to commands, according to some embodiments. -
FIG. 5 illustrates a method for automatically determining the functionality and capabilities of a DUT by detecting and processing audible outputs of the DUT generated in response to commands, according to some embodiments. -
FIG. 6 illustrates a method for automatically determining the functionality and capabilities of a DUT by monitoring traffic on a bus to determine how the DUT responds to commands, according to some embodiments. -
FIG. 7A illustrates a method for automatically determining the functionality and capabilities of a DUT by fingerprinting and crowdsourcing the DUT, according to some embodiments. -
FIG. 7B illustrates an example fingerprint data structure, according to some embodiments. -
FIG. 7C illustrates an example device remote control database, according to some embodiments. -
FIGS. 8 and 9 illustrate methods for determining infrared signals transmitted by a remote control and associated with commands of interest, according to some embodiments. -
FIG. 10 illustrates an example computer system useful for implementing various embodiments. - In the drawings, like reference numbers generally indicate identical or similar elements. Additionally, generally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.
-
FIG. 1 illustrates a block diagram of amultimedia environment 102, according to some embodiments. In a non-limiting example,multimedia environment 102 is directed to streaming media. However, this disclosure is not limited to that example embodiment. - The
multimedia environment 102 may include one ormore media systems 104, one ormore content servers 122 and/or one or more crowdsource servers 114 communicatively coupled via anetwork 120. In various embodiments, thenetwork 120 can include, without limitation, wired and/or wireless intranet, extranet, Internet, cellular, Bluetooth and/or any other short range, long range, local, regional, ad hoc, and/or global communications network, as well as any combination thereof. -
Media system 104 may include adisplay device 106,media device 108, systemremote control 110 and/or displaydevice remote control 140.Display device 106 may be a monitor, television, computer, smart phone, tablet, and/or projector, to name just a few examples.Media device 108 may be a streaming media device, DVD device, audio/video playback device, cable box, set top box, and/or digital video recording device, to name just a few examples. In some embodiments, themedia device 108 can be a part of, integrated with, operatively coupled to, and/or connected todisplay device 106. Themedia device 108 may be configured to communicate withnetwork 120. - Content servers 122 (also called content sources 122) may each include databases to store
content 124 andmetadata 126.Content 124 may include any combination of music, videos, movies, TV programs, multimedia, images, still pictures, text, graphics, gaming applications, advertisements, software, and/or any other content or data objects in electronic form. In some embodiments,metadata 126 comprises data aboutcontent 124. For example,metadata 126 may include associated or ancillary information indicating or related to writer, director, producer, composer, artist, actor, summary, chapters, production, history, year, trailers, alternate versions, related content, applications, and/or any other information pertaining or relating to thecontent 124.Metadata 126 may also or alternatively include links to any such information pertaining or relating to thecontent 124.Metadata 126 may also or alternatively include one or more indexes ofcontent 124, such as but not limited to a trick mode index. -
FIG. 2 illustrates an example block diagram of themedia device 108, according to some embodiments.Media device 108 may include astreaming module 202,processing module 204,user interface module 206,database 208,microphone 212,camera 214 and/orIR transceiver 216. -
FIG. 3 illustrates an example block diagram of the systemremote control 110, according to some embodiments. Systemremote control 110 may include auser interface 302,storage module 304,microphone 305,IR transceiver 306,camera 308 and/orIR processing module 310. - In some embodiments,
user 112 may use systemremote control 110 to interact with theuser interface module 206 ofmedia device 108 to select content, such as a movie, TV show, music, book, application, game, etc. Thestreaming module 202 ofmedia device 108 may request the selected content from content server(s) 122 over thenetwork 120. Content server(s) 122 may transmit the requested content to thestreaming module 202.Media device 108 may transmit the received content to displaydevice 106 for presentation touser 112. In streaming embodiments, thestreaming module 202 may transmit the content to displaydevice 106 in real time or near real time as it receives such content from content server(s) 122. In non-streaming embodiments,media device 108 may buffer or store the content received from content server(s) 122 indatabase 208 for later playback ondisplay device 106. - As noted above,
media system 104 may include a display deviceremote control 140 in addition to systemremote control 110. Display deviceremote control 140 may be specific to (i.e., native to) thedisplay device 106, to control display device 106 (and may not be capable of controlling other devices such as media device 108). For example, where thedisplay device 106 is a television, the display deviceremote control 140 may be the remote control that was included when theuser 112 purchased the television. The display deviceremote control 140 may transmit commands from a command set supported by thedisplay device 106, to control thedisplay device 106. In some embodiments, an infrared (IR) transmitter 142 in the display deviceremote control 140 may transmit such commands as IR signals to anIR receiver 107 in thedisplay device 107. It is noted that other devices inmultimedia environment 102 may have IR transmitters, receivers and/or transceivers, such asmedia device 108, systemremote control 110, etc. - Unlike display device
remote control 140 that is able to controlonly display device 106, the systemremote control 110 can be configured to control multiple devices inmedia system 104, such as but not limited tomedia device 108 anddisplay device 106. Systemremote control 110 is able to achieve this functionality even though some devices, such as display device 106 (as well as other devices in media system 104) may support different (or at least partially different) command sets. At times herein, systemremote control 110 is said to be non-native to thedisplay device 106. - In some embodiments,
user 112 may be required to manually program systemremote control 110 to work with different devices in media system 104 (such asdisplay device 106, for example). Assumedisplay device 106 is a television, for example. In some embodiments,user 112 may be required to identify the command set supported by the television (by looking in the user manual or searching on-line, for example), and then input into the systemremote control 110 the code associated with that command set. However, this can be a difficult and frustrating process for many users, especially users who are not technically inclined. - Accordingly, in other embodiments,
media device 108 and/or crowdsource server 114 may automatically determine the functionality and capabilities of thedisplay device 106 and/or other devices inmedia system 104. This may include determining the respective command sets of the devices inmedia system 104. Then,media device 108 may automatically configure systemremote control 110 to operate with andcontrol display device 106 and/or other devices inmedia system 104. Accordingly, embodiments solve the technical problem of enabling a single remote control to operate with and control multiple non-native devices in a media environment, where the command sets supported by the devices may differ, and/or may be unknown tousers 112. Such features and advantages are achieved by causing media device and/or crowdsource server 114 to operate according to instructions representative of the operations described herein (such as shown in the flowcharts ofFIGS. 4-9 ). - Determining the Functionality and Capabilities of a Device Under Test (DUT) by Monitoring how the DUT Responds to Commands
-
FIG. 4 illustrates amethod 402 for automatically determining the functionality and capabilities of a device under test (DUT) by monitoring how the DUT responds to commands, according to some embodiments.Method 402 may be useful when it is not known what command set(s), if any, are supported by the DUT. -
Method 402 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 4 , as will be understood by a person of ordinary skill in the art. -
Method 402 shall be described with reference toFIGS. 1 and 2 . However,method 402 is not limited to that example embodiment. - In 404, a command from a command set is transmitted to a DUT. For example,
media device 108 may transmit a command to thedisplay device 106 over alink 105. In some embodiments, link 105 may be an HDMI (High-Definition Multimedia Interface) link and the command may be from the Consumer Electronics Control (CEC) control set. As will be appreciated by persons skilled in the art, CEC is a feature of HDMI that allows users to command and control devices connected through HDMI using a single remote control. - In 406, signals transmitted by the DUT are monitored. In 408, it is determined whether the DUT responded as expected to the command. For example, in 406,
media device 108 may monitor link 105 for signals transmitted by thedisplay device 106. In 408,media device 108 may determine if signals received overlink 105 fromdisplay device 106 are appropriate given the CEC command transmitted in 404 and the syntax of the associated command set (that is, the CEC command set). - If, in 408, the DUT responded as expected to the command, then in 410 it is determined that the DUT supports the command set associated with the command. For example, if the
display device 106 responded as expected to the CEC command transmitted instep 404, then themedia device 108 may conclude that thedisplay device 106 supports CEC. - Also in 410, if it is determined the DUT supports the command set associated with the command, then thereafter the DUT is controlled using commands from that command set. For example, in some embodiments, the
media device 108 may program or otherwise configure the systemremote control 110 to interact with thedisplay device 106 using the command set. In an embodiment, themedia device 108 may transmit the command set to the systemremote control 110 using itsIR transceiver 216. The systemremote control 110 may receive this command set using itsIR transceiver 306, and store the received command set in thestorage module 304. The received command set may be in the form of information identifying one of a number of command sets previously stored in thestorage module 304 of the systemremote control 110, or may be information specifying IR signals corresponding to commands in the command set. In this manner, the systemremote control 110 is enabled to directly command and control thedisplay device 106 by using itsIR transceiver 306 to transmit IR signals to thedisplay device 106. - Alternatively or additionally in 410, if it is determined the DUT supports the command set associated with the command, then thereafter the
media device 108 commands and controls the DUT using commands from that command set. For example, assume it is determined in 410 that the DUT supports CEC. Then in 410, themedia device 108 may thereafter control thedisplay device 106 using CEC commands transmitted over the HDMI link 105, where such CEC commands correspond to commands issued by the systemremote control 110, and where such commands issued by the system remote 110 are received and understood by themedia device 108. For example, assume theuser 112 uses the systemremote control 110 to transmit a “volume up” command to thedisplay device 106. Also assume the particular “volume up” command transmitted by the systemremote control 110 is from a command set supported by themedia device 108 but not supported by thedisplay device 106. In this embodiment, themedia device 108 may receive and recognize the “volume up” command. In response, themedia device 108 may transmit a CEC “volume up” command to thedisplay device 106 over the HDMI link 105, to thereby cause thedisplay device 106 to increase volume. - If, in 410, it is determined the DUT did not respond as expected to the command issued in 404, then 412 is performed. In 412, it is determined whether there are more tests to perform to help identify a command set supported by the DUT. If there are more tests, then those tests are performed in
step 414. For example, if there are other commands from the same or different command sets to interrogate the DUT with, then step 414 is implemented by returning control to step 404. Alternatively or additionally, if there are different tests or methods to perform—such as those described herein—then step 414 is implemented by performing those tests/methods. - If, in 412, it is determined there are no more tests to perform, then 416 is performed. In 416, it is determined the DUT does not support a command set currently recognized by
media system 104. In this case, it is not possible to program or use systemremote control 110 to control thedisplay device 106. Instead, currently, theuser 112 will be required to use the display deviceremote control 140 to control thedisplay device 106. However, using a crowdsource embodiment (described below), the command set supported by thedisplay device 106 may be recognized in the future. Accordingly, themedia device 108 may performflowchart 402 in the future to attempt to identify the command set supported by thedisplay device 106 and enable the systemremote control 110 to control thedisplay device 106. - Automatically Determining the Functionality and Capabilities of a DUT by Detecting and Processing Audible Outputs of the DUT Generated in Response to Commands
-
FIG. 5 illustrates an example implementation of themethod 402 ofFIG. 4 . In particular,FIG. 5 illustrates amethod 502 for automatically determining the functionality and capabilities of a DUT by detecting and processing audible outputs of the DUT generated in response to commands, according to some embodiments. It is noted that, as used herein, the term audible includes sounds capable of being heard by humans, and sounds not capable of being heard by humans.Method 502 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 5 , as will be understood by a person of ordinary skill in the art.Method 502 shall be described with reference toFIGS. 1-3 . However,method 502 is not limited to that example embodiment. - In 504,
media device 108 may transmit a CEC “volume up” command to thedisplay device 106 overlink 105. In an embodiment, link 105 is an HDMI link. It is noted that CEC and the CEC “volume up” command are used herein for illustrative purposes only, and this disclosure is not limited to that example embodiment.Method 502 is applicable to any other command set and any other command within such other command set, as long as there is a finite set of valid, audible responses to the command made by the DUT (that is, the display device 106). Step 504 generally corresponds to step 404 inFIG. 4 . - In 506,
media device 108 may monitor the audio outputs ofdisplay device 106 using itsmicrophone 212, ormicrophone 305 in the systemremote control 110. Step 506 generally corresponds to step 406 inFIG. 4 . - In 508, responsive to the monitoring in
step 506,media device 108 may determine if thedisplay device 106 responded in a proper or valid way to the command issued instep 504, by analyzing the audio outputs of thedisplay device 106. For example,display device 106 may determine if thedisplay device 106 increased its volume after the CEC “volume up” command was issued instep 504. Step 508 generally corresponds to step 408 inFIG. 4 . - If it is determined that the
display device 106 responded in a proper or valid way to the command issued instep 504, then instep 510 themedia device 108 may conclude that thedisplay device 106 supports CEC. Thereafter, the systemremote control 110 and/or themedia device 108 may command and control thedisplay device 106 using CEC commands, as discussed above with reference to step 410 ofFIG. 4 . - The remainder of
flowchart 502—that is, steps 512, 514 and 516—respectively correspond tosteps FIG. 4 . - Automatically Determining the Functionality and Capabilities of a DUT by Monitoring Traffic on a Bus to Determine how the DUT Responds to Commands
-
FIG. 6 illustrates an example implementation of themethod 402 ofFIG. 4 . In particular,FIG. 6 illustrates amethod 602 for automatically determining the functionality and capabilities of a DUT by monitoring traffic on a bus to determine how the DUT responds to commands, according to some embodiments.Method 602 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 6 , as will be understood by a person of ordinary skill in the art.Method 602 shall be described with reference toFIGS. 1 and 2 . However,method 602 is not limited to that example embodiment. - In 604,
media device 108 may transmit a CEC “sleep” command to thedisplay device 106 overlink 105. Then, themedia device 108 may transmit a CEC “give device power status” command to thedisplay device 106 overlink 105. In an embodiment, link 105 is an HDMI link. It is noted that CEC and the CEC “sleep” and “give device power status” commands are used herein for illustrative purposes only, and this disclosure is not limited to that example embodiment.Method 602 is applicable to any other command set and any other command(s) within such other command set, as long as there is a finite set of valid responses to the command(s) that are transmitted by the DUT (that is, the display device 106) over thelink 105. Step 604 generally corresponds to step 404 inFIG. 4 . - In 606,
media device 108 may monitor thelink 105 for responses by thedisplay device 106 to the sequence of CEC “sleep” and “give device power status” commands. Step 606 generally corresponds to step 406 inFIG. 4 . - In 608, responsive to the monitoring in
step 606,media device 108 may determine if thedisplay device 106 responded in a proper or valid way to the command(s) issued instep 604, by analyzing the traffic over thelink 105. For example,media device 108 may determine that thedisplay device 106 responded properly if it transmitted a CEC “report power status” message over theHDMI link 105. As will be appreciated by persons skilled in the art, in some embodiments, the “report power status” message may be a proper response to the “give device power status” when the interrogated device has been put to sleep pursuant to the “sleep” command. Step 608 generally corresponds to step 408 inFIG. 4 . - If it is determined that the
display device 106 responded in a proper or valid way to the command issued instep 604, then instep 610 themedia device 108 may conclude that thedisplay device 106 supports CEC. Thereafter, the systemremote control 110 and/or themedia device 108 may command and control thedisplay device 106 using CEC commands, as discussed above with reference to step 410 ofFIG. 4 . - The remainder of
flowchart 602—that is, steps 612, 614 and 616—respectively correspond tosteps FIG. 4 . - Automatically Determining the Functionality and Capabilities of a DUT by Fingerprinting and Crowdsourcing the DUT
-
FIG. 7A illustrates amethod 702 for automatically determining the functionality and capabilities of a DUT by fingerprinting and crowdsourcing the DUT.Method 702 may be part offlowchart 402 ofFIG. 4 —asstep 414, for example—or may be stand alone and distinct fromflowchart 402. -
Method 702 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 7A , as will be understood by a person of ordinary skill in the art.Method 702 shall be described with reference toFIGS. 1 and 2 . However,method 702 is not limited to that example embodiment. - In 704, data may be collected on the DUT for purpose of generating an identification (sometimes called a fingerprint herein) of the DUT. The data collected in
step 704 represents attributes, features and/or characteristics of the DUT. - For example, the
media device 108 may collect such data on thedisplay device 106. For example, themedia device 108 may query or otherwise interact with thedisplay device 106 overlink 105 using well known standards or interfaces to collect data on thedisplay device 106. Such well known standards or interfaces may include CEC and extended display identification data (EDID), for example. CEC was discussed above. As will be appreciated by persons skilled in the art, EDID is a standard published by the Video Electronics Standards Association (VESA) that defines an interface/data structure that can be used by a digital display (such as display device 106) to describe its capabilities to a content source (such as media device 108). Data collected instep 704 may include responses to CEC and/or EDID commands/messages and/or timing data of the latency of responding to such commands/messages. - Additionally or alternatively, data collected in
step 704 may include timing relating to the HDCP negotiation process between themedia device 108 and thedisplay device 106. As will be appreciated by persons skilled in the art, HDCP (high-bandwidth digital content protection) is a form of digital copy protection to prevent copying of digital audio and video content as it travels across connections. - Additionally or alternatively, data collected in
step 704 may include the model number of thedisplay device 106. Specifically, theuser 112 may be prompted to enter the model number of thedisplay device 106 using the user interface 302 (i.e., a keypad or touch screen) of the systemremote control 110. Or, theuser 112 may be prompted to take a picture of the model number of thedisplay device 106 using thecamera 308 of the system remote control 110 (or any other handheld computing device, such as a smart phone). Themedia device 108 and/or crowdsource server 114 may then identify the model number of thedisplay device 106 by applying character recognition techniques to the picture. - Also in
step 704, themedia device 108 may transmit this collected data to the crowdsource server 114 vianetwork 120. Themedia device 108 may also transmit to the crowdsource server 114 information indicating whether or not thedisplay device 106 supports CEC (as determined insteps FIGS. 8 and 9 ). - In 706, a fingerprint of the DUT (that is, the display device 106) may be generated using the data collected in 704. As noted above, the data collected in
step 704 represents attributes, features and/or characteristics of thedisplay device 106. In 706, the collected data may be ordered and/or combined to generate a fingerprint of thedisplay device 106.FIG. 7B illustrates anexample data structure 720 of a fingerprint, according to some embodiments.Fingerprint data structure 720 may include amodel number field 722, aCEC data field 724, anEDID data field 726, aHDCP data field 728, as well as other fields corresponding to data collected instep 704. In some embodiments, the fingerprint ofdisplay device 106 is generated by processingmodule 204 of themedia device 108, and then transmitted to the crowdsource server 114 overnetwork 120. In other embodiments, the data collected instep 704 is transmitted to the crowdsource server 114 over network 120 (as part of step 704), and afingerprint module 130 in the crowdsource server 114 generates the fingerprint ofdisplay device 106. - In 708, the crowdsource server 114 uses the fingerprint to determine if the
display device 106 is represented in a deviceremote control database 132.FIG. 7C illustrates an example deviceremote control database 132, according to some embodiments. Each unique fingerprint has a row 746 in the deviceremote control database 132. A fingerprint's row 746 may store (1) the fingerprint in a “Fingerprint”column 740, (2) whether or not the DUT corresponding to the fingerprint supports CEC (or other standards or command sets) in a “CEC Supported”column 742, and/or (3) information specifying IR signals of remote control commands supported by the DUT in a “Remote Control Signals”column 744. - It is noted that some fingerprints may be similar. This is indicated in the example of
FIG. 7C by fingerprints B, B′ and B″ inrows Different media devices 108 indifferent media systems 104 may use the same display device 106 (that is, the same television, for example). But, thesedifferent media devices 108 may have collected different data on the display device instep 704, resulting in the generation of different fingerprints instep 706. The different data may be the result of variabilities in the televisions (some may be older, newer, refurbished, damaged, etc.), and/or the gathering of different information by themedia devices 108 instep 704. As a result, the same television model may have multiple entries in the deviceremote control database 132. - The crowdsource server 114 performs
step 708 by matching the fingerprint generated instep 706 with the fingerprints in theFingerprint column 740 of the rows 746 of the deviceremote control database 132. If no match exists, then instep 710, the crowdsource server 114 determines that the DUT is not currently represented in the deviceremote control database 132. Accordingly, instep 712, the crowdsource server 114 adds a new row 746 to the deviceremote control database 132. The crowdsource server 114 inserts the fingerprint fromstep 706 in theFingerprint column 740 of the new row 746. The crowdsource server 114 inserts information indicating whether or not the DUT supports CEC (if provided in step 704) in the CEC Supportedcolumn 742 of the new row 746. The crowdsource server 114 inserts information specifying IR signals corresponding to remote control commands supported by the DUT (if provided in step 704) into the Remote Control Signalscolumn 744 of the new row 746. - Returning to step 710, if a fingerprint match exists, then in
step 710, the crowdsource server 114 determines that the DUT is already represented in the deviceremote control database 132. Accordingly, instep 714, the crowdsource server 114 retrieves from the matching row 746 information indicating whether or not the DUT supports CEC from the CEC Supportedcolumn 742 of the matching row 746 (if such information is present in the matching row 746), and information specifying IR signals corresponding to remote control commands supported by the DUT from the Remote Control Signalscolumn 744 of the matching row 746 (if such information is present in the matching row 746). - The crowdsource server 114 provides such retrieved information to the
media device 108 fromstep 704. TheMedia device 108 may use such information to control thedisplay device 106. For example, assume such information provided by the crowdsource server 114 indicates that thedisplay device 106 supports CEC. Then, themedia device 108 may thereafter control thedisplay device 106 using CEC commands transmitted over the HDMI link 105, where such CEC commands correspond to commands issued by the systemremote control 110, and where such commands issued by the system remote 110 are received and understood by themedia device 108. For example, assume theuser 112 uses the systemremote control 110 to transmit a “volume up” command to thedisplay device 106. Also assume the particular “volume up” command transmitted by the systemremote control 110 is from a command set supported by themedia device 108 but not supported by thedisplay device 106. In this embodiment, themedia device 108 may receive and recognize the “volume up” command. In response, themedia device 108 may transmit a CEC “volume up” command to thedisplay device 106 over the HDMI link 105, to thereby cause thedisplay device 106 to increase volume. - Additionally or alternatively, assume such information provided by the crowdsource server 114 specifies IR signals corresponding to remote control commands supported by the
display device 106. In this embodiment, themedia device 108 may use itsIR transceiver 216 to transmit this information to theIR transceiver 306 in the systemremote control 110. The systemremote control 110 may store the received information in thestorage module 304. In this manner, themedia device 108 can program the systemremote control 110 to directly communicate with and control thedisplay device 106 using the stored IR signals (which are supported by the display device 106). - In 716, the crowdsource server 114 updates the CEC Supported
column 742 of the matching row 746 with information indicating whether or not the DUT supports CEC (if provided in step 704). The crowdsource server 114 also updates the Remote Control Signalscolumn 744 of the matching row 746 with information specifying IR signals corresponding to remote control commands supported by the DUT (if provided in step 704). In some embodiments, such updates add to rather than replace the information already existing in the matching row. For example, whether or not the DUT supports CEC (as indicated by the CEC Supported column 742) may be set by a majority vote of the data received instep 704 or by some other heuristic applied to the data received instep 704 from media devices 108 (for example, if 5media devices 108 indicate the DUT supports CEC, and 3 indicate the DUT does not support CEC, then the crowdsource server 114 indicates in the CEC Supportedcolumn 742 of the matching row 746 that the DUT supports CEC). Similarly, the IR signals in the Remote Control Signalscolumn 742 may be populated from data received instep 704 frommultiple media devices 108, with conflicts resolved by majority vote (for example, if 2media devices 108 indicate the volume up IR code is ABC, and 3 indicate the IR code is XYZ, then XYZ is saved in the Remote Control Signalscolumn 742 of the matching row 746) or by some other heuristic. In this manner, information in the deviceremote control database 132 is continually updated and improved by crowdsourcing information frommultiple media devices 108 inmultiple media systems 104. - Determining Infrared Signals Transmitted by a Remote Control and Associated with Commands of Interest
-
FIG. 8 illustrates amethod 802 for determining infrared signals transmitted by a remote control and associated with commands of interest, according to some embodiments.Method 802 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 8 , as will be understood by a person of ordinary skill in the art.Method 802 shall be described with reference toFIGS. 1-3 . However,method 802 is not limited to that example embodiment. - In 804,
user 112 points the IR transmitter 142 of the display deviceremote control 140 to thecamera 308 of the system remote control 110 (or any other computing device having the functionality of the systemremote control 110, such as a smart phone, tablet computer, laptop computer, digital assistant, etc.). - In 804, the
user 112 is instructed to issue a command using the display deviceremote control 140. The instruction may be transmitted by themedia device 108 and displayed on a screen of the systemremote control 110 or on thedisplay device 106. Theuser 112 may issue the command by pressing a button on the display deviceremote control 140. Assume, for purposes of illustration, theuser 112 presses the Volume Up button on the display deviceremote control 140. - In 806, the
camera 308 of the systemremote control 110 captures the IR signal transmitted by the IR transmitter 142 of the display deviceremote control 140. - In some embodiments, the
camera 308 of the systemremote control 110 may have IR filters or other well-known functional and/or structural features to facilitate the capture of IR signals. In other embodiments, thecamera 308 may lack some or all of those features. For example, the shutter speed of thecamera 308 may be too slow to fully capture an IR signal in a single transmission. Accordingly, instep 810, theIR processing module 310 determines if the IR signal was fully captured instep 808. If the IR signal was not fully captured, then instep 812, theuser 112 is requested to issue the same command using the display deviceremote control 140. The IR signal is again captured by thecamera 308 instep 808, and then combined with previous captures of the IR signal. Steps 808-812 are repeated until theIR processing module 310 instep 810 determines that the IR signal corresponding to the command ofstep 806 has been fully captured. - Once the
IR processing module 310 determines in 810 that the IR signal has been fully captured, then control moves to 814. In 814, themedia device 108 or the systemremote control 110 determines if there are more commands of the display deviceremote control 140 that need to be captured. If there are more commands to capture, then control returns to step 804. - Otherwise, in 816, the system
remote control 110 transmits the IR signals corresponding to commands of the display device remote control 140 (which were collected in steps 804-814) to themedia device 108. Such transmission may occur as each IR signal is collected, or as batches of the IR signals. Themedia device 108 may then transmit these IR signals to the crowdsource server 114 instep 704 ofFIG. 7A , as described above. The systemremote control 110 may also save these IR signals in thestorage module 304. - In 818, the system
remote control 110 may use the IR signals stored in thestorage module 304 to directly communicate with and control thedisplay device 106. As a result offlowchart 802, the systemremote control 110—which is not natively supported by thedisplay device 106—is configured/programmed to directly communication with and control thedisplay device 106. -
FIG. 8 requires the involvement of theuser 112 to detect the IR codes of the display deviceremote control 140. Additionally or alternatively,FIG. 9 illustrates amethod 902 for determining IR signals transmitted by the display deviceremote control 140, without the need of user involvement.Method 902 can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 9 , as will be understood by a person of ordinary skill in the art.Method 902 shall be described with reference toFIGS. 1-3 . However,method 902 is not limited to that example embodiment. - In 904, a high-gain, wideband IR sensor/
detector 109 monitors and captures IR signals transmitted by the display deviceremote control 140. In some embodiments, thewideband IR detector 109 includes IR filters or other well-known functional and/or structural features to facilitate the capture of IR signals. Thewideband IR detector 109 is positioned proximate to thedisplay device 106 and/or themedia device 108. Thewideband IR detector 109 transmits the captured IR signal to themedia device 108. - In 906, the
media device 108 identifies the action(s) taken by thedisplay device 106 after transmission of the IR signals by the display deviceremote control 140 instep 904. In some embodiments, themedia device 108 may perform 906 by monitoring the CEC traffic on the HDMI link 105, or listening onmicrophone 212 for audible actions taken by the display device (such as turning off or changing the volume level, for example). - In 908, the
media device 108 may associate the IR signals captured instep 904 with the action(s) identified in 906. In other words, themedia device 108 may determine that the IR signals are associated with a command of the display deviceremote control 140 to cause thedisplay device 106 to perform the identified action. - Steps 904-908 may be periodically or continuously performed to identify the IR signals associated with the command set supported by the display device
remote control 140 and thedisplay device 106. - In 910, the
media device 108 may transmit information specifying these IR signals associated with the command set of the display deviceremote control 140 and thedisplay device 106 to the systemremote control 110. The systemremote control 110 may store such information in itsstorage module 304. As a result offlowchart 902, the systemremote control 110—which is not natively supported by thedisplay device 106—is configured/programmed to directly communication with and control thedisplay device 106. - The
media device 108 may also transmit these IR signals to the crowdsource server 114 as part ofstep 704 ofFIG. 7A , as described above. - In 912, the system
remote control 110 may use these stored IR signals to directly communicate with and control thedisplay device 106. - Example Computer System
- Various embodiments and/or components therein can be implemented, for example, using one or more computer systems, such as
computer system 1000 shown inFIG. 10 .Computer system 1000 can be any computer or computing device capable of performing the functions described herein. For example, one ormore computer systems 1000 can be used to implement any embodiments ofFIGS. 1-9 , and/or any combination or sub-combination thereof. -
Computer system 1000 includes one or more processors (also called central processing units, or CPUs), such as aprocessor 1004.Processor 1004 is connected to a communication infrastructure orbus 1006. - One or
more processors 1004 can each be a graphics processing unit (GPU). In some embodiments, a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications. The GPU can have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc. -
Computer system 1000 also includes user input/output device(s) 1003, such as monitors, keyboards, pointing devices, etc., that communicate withcommunication infrastructure 1006 through user input/output interface(s) 1002. -
Computer system 1000 also includes a main orprimary memory 1008, such as random access memory (RAM).Main memory 1008 can include one or more levels of cache.Main memory 1008 has stored therein control logic (i.e., computer software) and/or data. -
Computer system 1000 can also include one or more secondary storage devices ormemory 1010.Secondary memory 1010 can include, for example, ahard disk drive 1012 and/or a removable storage device or drive 1014.Removable storage drive 1014 can be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive. -
Removable storage drive 1014 can interact with aremovable storage unit 1018.Removable storage unit 1018 includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data.Removable storage unit 1018 can be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device.Removable storage drive 1014 reads from and/or writes toremovable storage unit 1018 in a well-known manner. - According to an exemplary embodiment,
secondary memory 1010 can include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed bycomputer system 1000. Such means, instrumentalities or other approaches can include, for example, aremovable storage unit 1022 and aninterface 1020. Examples of theremovable storage unit 1022 and theinterface 1020 can include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface. -
Computer system 1000 can further include a communication ornetwork interface 1024.Communication interface 1024 enablescomputer system 1000 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number 1028). For example,communication interface 1024 can allowcomputer system 1000 to communicate with remote devices 1028 overcommunications path 1026, which can be wired and/or wireless, and which can include any combination of LANs, WANs, the Internet, etc. Control logic and/or data can be transmitted to and fromcomputer system 1000 viacommunication path 1026. - In some embodiments, a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to,
computer system 1000,main memory 1008,secondary memory 1010, andremovable storage units - Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of this disclosure using data processing devices, computer systems and/or computer architectures other than that shown in
FIG. 10 . In particular, embodiments can operate with software, hardware, and/or operating system implementations other than those described herein. - It is to be appreciated that the Detailed Description section, and not the Summary and Abstract sections, is intended to be used to interpret the claims. The Summary and Abstract sections can set forth one or more but not all exemplary embodiments as contemplated by the inventors, and thus, are not intended to limit this disclosure or the appended claims in any way.
- While this disclosure describes exemplary embodiments for exemplary fields and applications, it should be understood that the disclosure is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of this disclosure. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and/or entities illustrated in the figures and/or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.
- Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments can perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.
- References herein to “one embodiment,” “an embodiment,” “an example embodiment,” or similar phrases, indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, some embodiments can be described using the terms “connected” and/or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
- The breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
Claims (21)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/494,567 US20220103428A1 (en) | 2017-07-11 | 2021-10-05 | Automatic determination of display device functionality |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/646,275 US10447537B2 (en) | 2017-07-11 | 2017-07-11 | Automatic determination of display device functionality |
US16/653,118 US11140034B2 (en) | 2017-07-11 | 2019-10-15 | Automatic determination of display device functionality |
US17/494,567 US20220103428A1 (en) | 2017-07-11 | 2021-10-05 | Automatic determination of display device functionality |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/653,118 Continuation US11140034B2 (en) | 2017-07-11 | 2019-10-15 | Automatic determination of display device functionality |
Publications (1)
Publication Number | Publication Date |
---|---|
US20220103428A1 true US20220103428A1 (en) | 2022-03-31 |
Family
ID=64999316
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/646,275 Active US10447537B2 (en) | 2017-07-11 | 2017-07-11 | Automatic determination of display device functionality |
US16/653,118 Active US11140034B2 (en) | 2017-07-11 | 2019-10-15 | Automatic determination of display device functionality |
US17/494,567 Pending US20220103428A1 (en) | 2017-07-11 | 2021-10-05 | Automatic determination of display device functionality |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/646,275 Active US10447537B2 (en) | 2017-07-11 | 2017-07-11 | Automatic determination of display device functionality |
US16/653,118 Active US11140034B2 (en) | 2017-07-11 | 2019-10-15 | Automatic determination of display device functionality |
Country Status (3)
Country | Link |
---|---|
US (3) | US10447537B2 (en) |
EP (1) | EP3652896A4 (en) |
WO (1) | WO2019014188A1 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10447537B2 (en) | 2017-07-11 | 2019-10-15 | Roku, Inc. | Automatic determination of display device functionality |
US11272252B2 (en) * | 2018-07-19 | 2022-03-08 | Roku, Inc. | Modifying playback of replacement content based on control messages |
US11102542B2 (en) * | 2019-06-06 | 2021-08-24 | Rovi Guides, Inc. | Systems and methods for controlling access from a first content platform to content items available on a second content platform |
US20220095005A1 (en) * | 2020-09-22 | 2022-03-24 | Universal Electronics Inc. | System and Method for Facilitating an Enabling of a Device Functionality |
US11169771B1 (en) * | 2020-11-13 | 2021-11-09 | Roku, Inc. | Transmitting messages to a display device based on detected audio output |
WO2022250810A1 (en) * | 2021-05-24 | 2022-12-01 | Arris Enterprises Llc | Audio tone assisted rcu programming |
CN114222172A (en) * | 2021-12-14 | 2022-03-22 | 海宁奕斯伟集成电路设计有限公司 | Method and device for acquiring HDMI-CEC function of electronic equipment |
Citations (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5481252A (en) * | 1990-10-26 | 1996-01-02 | Samsung Electronics Co., Ltd. | Mode conversion method for multifunctional programmable remote controller |
US20030163542A1 (en) * | 2002-02-28 | 2003-08-28 | Koninklijke Philips Electronics N.V. | Remote control signals updated and stored via network |
US6914551B2 (en) * | 2002-04-12 | 2005-07-05 | Apple Computer, Inc. | Apparatus and method to facilitate universal remote control |
US7116229B1 (en) * | 2004-03-31 | 2006-10-03 | Zilog, Inc. | Programming a remote control device using RFID technology |
US20100039282A1 (en) * | 2008-08-12 | 2010-02-18 | Hostage Christine M | Universal Remote Control Programming |
US20110109490A1 (en) * | 2009-11-12 | 2011-05-12 | At&T Intellectual Property I, L.P. | Programming a universal remote control via direct interaction with an original remote control |
US8031270B1 (en) * | 2006-01-31 | 2011-10-04 | Cypress Semiconductor Corporation | Remote control system |
US20120088549A1 (en) * | 2010-10-06 | 2012-04-12 | Lg Electronics Inc. | Mobile terminal, display device and controlling method thereof |
US20120124245A1 (en) * | 2010-11-17 | 2012-05-17 | Flextronics Id, Llc | Universal remote control with automated setup |
US20120133841A1 (en) * | 2010-11-30 | 2012-05-31 | Verizon Patent And Licensing, Inc. | Universal remote control systems, methods, and apparatuses |
US20120188113A1 (en) * | 2011-01-26 | 2012-07-26 | Samsung Electronics Co., Ltd. | Remote control apparatus and electronic device remotely controlled by the same |
US20130136455A1 (en) * | 2010-06-04 | 2013-05-30 | Koninklijke Philips Electronics N.V. | Programming a universal remote control |
US8508401B1 (en) * | 2010-08-31 | 2013-08-13 | Logitech Europe S.A. | Delay fixing for command codes in a remote control system |
US20140153927A1 (en) * | 2012-12-05 | 2014-06-05 | Echostar Technologies L.L.C. | Detection of remote control for configuration of universal remote |
US20160328336A1 (en) * | 2015-05-05 | 2016-11-10 | Foshan Bolang De Technology Co., Ltd. | Full-control remote controller based on waveform downloading and address selecting and an implementing method thereof |
US20170024999A1 (en) * | 1998-07-23 | 2017-01-26 | Universal Electronics Inc. | System and method for automatically setting up a universal remote control |
US20170032665A1 (en) * | 2015-07-30 | 2017-02-02 | Roku, Inc. | Universal Remote Control With Automatic State Synchronization |
US20190266888A1 (en) * | 2016-08-30 | 2019-08-29 | Shenzhen Unitivelink Electronics Co., Ltd. | Infrared remote control learning method and device |
US10447537B2 (en) * | 2017-07-11 | 2019-10-15 | Roku, Inc. | Automatic determination of display device functionality |
US20230098728A1 (en) * | 2021-09-30 | 2023-03-30 | Orange | Method of generating a personalized television channel, corresponding device, system and computer program |
Family Cites Families (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5227780A (en) * | 1989-03-16 | 1993-07-13 | Houston Satellite Systems, Inc. | Apparatus with a portable UHF radio transmitter remote for controlling one or more of infrared controlled appliances |
US5903226A (en) * | 1993-03-15 | 1999-05-11 | Prince Corporation | Trainable RF system for remotely controlling household appliances |
US5386251A (en) * | 1993-06-03 | 1995-01-31 | Zilog, Inc. | Television receiver with learning remote control system capable of being controlled by a remote control device manufactured by different companies |
US6124804A (en) * | 1994-11-10 | 2000-09-26 | Matsushita Electric Industrial Co., Ltd. | Remote controller, remote control interface, and remote control system including a remote controller and a remote control interface |
TW391536U (en) * | 1997-12-04 | 2000-05-21 | Wang Mau Sen | Remote controller capable of simultaneously copying internal code and carrier frequency |
US7006802B2 (en) * | 1998-11-09 | 2006-02-28 | Tsui Philip Y W | Universal transmitter |
US6249673B1 (en) * | 1998-11-09 | 2001-06-19 | Philip Y. W. Tsui | Universal transmitter |
US8271287B1 (en) * | 2000-01-14 | 2012-09-18 | Alcatel Lucent | Voice command remote control system |
US7562128B1 (en) * | 2000-03-06 | 2009-07-14 | Philips Koninklijke Philips Electronics N.V. | STB connects remote to web site for customized code downloads |
US7062175B2 (en) * | 2001-04-19 | 2006-06-13 | X10 Wireless Technology, Inc. | Repeating radio frequency transmission system for extending the effective operational range of an infrared remote control system |
JP3745651B2 (en) * | 2001-06-18 | 2006-02-15 | Necディスプレイソリューションズ株式会社 | Display device remote adjustment method and display device |
US6944704B2 (en) * | 2001-10-04 | 2005-09-13 | Sony Corporation | Method and apparatus for utilizing extended AV/C command frames including status inquiry, notify inquiry and control inquiry command types |
CA2526125C (en) * | 2003-05-16 | 2015-08-18 | M-Qube, Inc. | System and method for determining and delivering appropriate multimedia content to data communication devices |
US8190723B2 (en) * | 2003-12-14 | 2012-05-29 | Cisco Technology, Inc. | Method and system for automatically determining commands for a network element |
TWI271659B (en) * | 2004-05-05 | 2007-01-21 | Prolific Technology Inc | Memory card equipped with a multi-interface function and method for choosing a compatible transmission mode |
US7748040B2 (en) * | 2004-07-12 | 2010-06-29 | Architecture Technology Corporation | Attack correlation using marked information |
FR2884943B1 (en) * | 2005-04-25 | 2007-07-27 | Canon Europa Nv Naamlooze Venn | METHOD FOR CONTROLLING CONTROL IN A COMMUNICATION NETWORK, CONTROL DEVICE, COMPUTER PROGRAM PRODUCT, AND CORRESPONDING STORAGE MEDIUM |
US7907222B2 (en) * | 2005-09-08 | 2011-03-15 | Universal Electronics Inc. | System and method for simplified setup of a universal remote control |
US8041025B2 (en) * | 2006-08-07 | 2011-10-18 | International Business Machines Corporation | Systems and arrangements for controlling modes of audio devices based on user selectable parameters |
JP4334590B2 (en) * | 2007-12-12 | 2009-09-30 | 株式会社東芝 | Information reproducing apparatus and information reproducing method |
US9852615B2 (en) * | 2011-03-25 | 2017-12-26 | Universal Electronics Inc. | System and method for facilitating appliance control via a smart device |
US9088663B2 (en) * | 2008-04-18 | 2015-07-21 | Universal Electronics Inc. | System for appliance control via a network |
US20090307591A1 (en) * | 2008-06-09 | 2009-12-10 | Pham Duyet D | Virtual remote controller |
US20100053468A1 (en) * | 2008-08-30 | 2010-03-04 | Mike Harvill | Device ir setup using ir detector |
US8180891B1 (en) * | 2008-11-26 | 2012-05-15 | Free Stream Media Corp. | Discovery, access control, and communication with networked services from within a security sandbox |
US8502925B2 (en) * | 2009-12-29 | 2013-08-06 | VIZIO Inc. | Television programming of a remote control |
US20110296453A1 (en) * | 2010-05-31 | 2011-12-01 | Thirunarayanan Srinivasan | Pay-Per-Website Visit ad system |
US9490998B1 (en) * | 2010-11-08 | 2016-11-08 | Google Inc. | Network-based remote control |
US9019435B2 (en) * | 2011-09-22 | 2015-04-28 | Universal Electronics Inc. | System and method for configuring controlling device functionality |
US9113203B2 (en) * | 2012-06-28 | 2015-08-18 | Google Inc. | Generating a sequence of audio fingerprints at a set top box |
US9715821B2 (en) * | 2012-07-23 | 2017-07-25 | Comcast Cable Communications, Llc | System and method for managing device controls |
US9160743B2 (en) * | 2013-02-12 | 2015-10-13 | Qualcomm Incorporated | Biometrics based electronic device authentication and authorization |
US9078083B2 (en) * | 2013-02-20 | 2015-07-07 | Verizon Patent And Licensing Inc. | Application server awareness of device capabilities in a wireless network |
US20160366456A1 (en) * | 2013-08-30 | 2016-12-15 | U-Me Holdings LLC | Device that is automatically programmable according to location |
CN106233742B (en) * | 2014-08-26 | 2019-07-09 | 松下知识产权经营株式会社 | Display control unit, display control method and recording medium |
US9613586B2 (en) * | 2014-12-23 | 2017-04-04 | Roku, Inc. | Providing a representation for a device connected to a display device |
KR20160084776A (en) * | 2015-01-06 | 2016-07-14 | 삼성전자주식회사 | Method for transmitting configuration information and electronic apparatus |
US9911321B2 (en) * | 2015-03-18 | 2018-03-06 | Logitech Europe S.A. | Simplified adaptable controller |
US11025979B2 (en) * | 2015-08-31 | 2021-06-01 | Roku, Inc. | Crowd sourced indexing and/or searching of content |
US9848214B2 (en) * | 2015-10-01 | 2017-12-19 | Sorenson Media, Inc. | Sequentially overlaying media content |
WO2017173377A1 (en) * | 2016-04-01 | 2017-10-05 | Caavo Inc | Method and apparatus for implementing hdmi cec |
CN109690478B (en) * | 2016-09-14 | 2022-06-10 | 瑞士优北罗股份有限公司 | Software update system, over-the-air firmware update system, and method of updating client device |
-
2017
- 2017-07-11 US US15/646,275 patent/US10447537B2/en active Active
-
2018
- 2018-07-10 EP EP18832316.6A patent/EP3652896A4/en active Pending
- 2018-07-10 WO PCT/US2018/041401 patent/WO2019014188A1/en unknown
-
2019
- 2019-10-15 US US16/653,118 patent/US11140034B2/en active Active
-
2021
- 2021-10-05 US US17/494,567 patent/US20220103428A1/en active Pending
Patent Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5481252A (en) * | 1990-10-26 | 1996-01-02 | Samsung Electronics Co., Ltd. | Mode conversion method for multifunctional programmable remote controller |
US20170024999A1 (en) * | 1998-07-23 | 2017-01-26 | Universal Electronics Inc. | System and method for automatically setting up a universal remote control |
US20030163542A1 (en) * | 2002-02-28 | 2003-08-28 | Koninklijke Philips Electronics N.V. | Remote control signals updated and stored via network |
US6914551B2 (en) * | 2002-04-12 | 2005-07-05 | Apple Computer, Inc. | Apparatus and method to facilitate universal remote control |
US7116229B1 (en) * | 2004-03-31 | 2006-10-03 | Zilog, Inc. | Programming a remote control device using RFID technology |
US8031270B1 (en) * | 2006-01-31 | 2011-10-04 | Cypress Semiconductor Corporation | Remote control system |
US20100039282A1 (en) * | 2008-08-12 | 2010-02-18 | Hostage Christine M | Universal Remote Control Programming |
US20110109490A1 (en) * | 2009-11-12 | 2011-05-12 | At&T Intellectual Property I, L.P. | Programming a universal remote control via direct interaction with an original remote control |
US20130136455A1 (en) * | 2010-06-04 | 2013-05-30 | Koninklijke Philips Electronics N.V. | Programming a universal remote control |
US8508401B1 (en) * | 2010-08-31 | 2013-08-13 | Logitech Europe S.A. | Delay fixing for command codes in a remote control system |
US20120088549A1 (en) * | 2010-10-06 | 2012-04-12 | Lg Electronics Inc. | Mobile terminal, display device and controlling method thereof |
US20120124245A1 (en) * | 2010-11-17 | 2012-05-17 | Flextronics Id, Llc | Universal remote control with automated setup |
US20120133841A1 (en) * | 2010-11-30 | 2012-05-31 | Verizon Patent And Licensing, Inc. | Universal remote control systems, methods, and apparatuses |
US20120188113A1 (en) * | 2011-01-26 | 2012-07-26 | Samsung Electronics Co., Ltd. | Remote control apparatus and electronic device remotely controlled by the same |
US20140153927A1 (en) * | 2012-12-05 | 2014-06-05 | Echostar Technologies L.L.C. | Detection of remote control for configuration of universal remote |
US20160328336A1 (en) * | 2015-05-05 | 2016-11-10 | Foshan Bolang De Technology Co., Ltd. | Full-control remote controller based on waveform downloading and address selecting and an implementing method thereof |
US20170032665A1 (en) * | 2015-07-30 | 2017-02-02 | Roku, Inc. | Universal Remote Control With Automatic State Synchronization |
US20190266888A1 (en) * | 2016-08-30 | 2019-08-29 | Shenzhen Unitivelink Electronics Co., Ltd. | Infrared remote control learning method and device |
US10447537B2 (en) * | 2017-07-11 | 2019-10-15 | Roku, Inc. | Automatic determination of display device functionality |
US11140034B2 (en) * | 2017-07-11 | 2021-10-05 | Roku, Inc. | Automatic determination of display device functionality |
US20230098728A1 (en) * | 2021-09-30 | 2023-03-30 | Orange | Method of generating a personalized television channel, corresponding device, system and computer program |
Also Published As
Publication number | Publication date |
---|---|
EP3652896A4 (en) | 2021-03-31 |
US10447537B2 (en) | 2019-10-15 |
WO2019014188A1 (en) | 2019-01-17 |
EP3652896A1 (en) | 2020-05-20 |
US20190020543A1 (en) | 2019-01-17 |
US20200044922A1 (en) | 2020-02-06 |
US11140034B2 (en) | 2021-10-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220103428A1 (en) | Automatic determination of display device functionality | |
US10971111B2 (en) | Providing a representation for a device connected to a display device | |
US11294457B2 (en) | Display apparatus and controlling method thereof | |
US11700410B2 (en) | Crowd sourced indexing and/or searching of content | |
US20230333719A1 (en) | Customizing a user interface based on user capabilities | |
CN112075085B (en) | Electronic device and control method thereof | |
US11012754B2 (en) | Display apparatus for searching and control method thereof | |
US20140324623A1 (en) | Display apparatus for providing recommendation information and method thereof | |
US10503776B2 (en) | Image display apparatus and information providing method thereof | |
US9693112B2 (en) | Remote control system for a smart television | |
US20190132645A1 (en) | Electronic apparatus and controlling method thereof | |
CN115460452A (en) | Display device and channel playing method | |
KR20210051048A (en) | Electronic apparatus and control method thereof | |
EP3748982B1 (en) | Electronic device and content recognition information acquisition therefor | |
KR20190065601A (en) | Electronic apparatus and controlling method thereof | |
US20220309095A1 (en) | Display device | |
US20200089963A1 (en) | Electronic apparatus, control method thereof and eletronic system | |
KR20230056452A (en) | A display apparatus and a method of operating the display apparatus | |
KR20240078140A (en) | Computing device and operating method for the same | |
KR20200069936A (en) | Apparatus for providing information contained in media and method for the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: ROKU, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ASNIS, ILYA;DALY, DANIEL MARTIN;GARNER, GREGORY MACK;AND OTHERS;SIGNING DATES FROM 20170623 TO 20170707;REEL/FRAME:059831/0555 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING RESPONSE FOR INFORMALITY, FEE DEFICIENCY OR CRF ACTION |