US10474241B2 - Method and device for transmitting/receiving data using HDMI - Google Patents

Method and device for transmitting/receiving data using HDMI Download PDF

Info

Publication number
US10474241B2
US10474241B2 US15/318,336 US201515318336A US10474241B2 US 10474241 B2 US10474241 B2 US 10474241B2 US 201515318336 A US201515318336 A US 201515318336A US 10474241 B2 US10474241 B2 US 10474241B2
Authority
US
United States
Prior art keywords
information
sink device
source device
hdmi
hec
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/318,336
Other languages
English (en)
Other versions
US20170115740A1 (en
Inventor
Dokyun Kim
Hyunsik YANG
Hyeonjae LEE
Jangwoong Park
Jinkwon LIM
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Priority to US15/318,336 priority Critical patent/US10474241B2/en
Publication of US20170115740A1 publication Critical patent/US20170115740A1/en
Assigned to LG ELECTRONICS INC. reassignment LG ELECTRONICS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KIM, DOKYUN, LEE, HYEONJAE, LIM, Jinkwon, PARK, JANGWOONG, YANG, Hyunsik
Application granted granted Critical
Publication of US10474241B2 publication Critical patent/US10474241B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G5/006—Details of the interface to the display terminal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42207—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43632—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wired protocol, e.g. IEEE 1394
    • H04N21/43635—HDMI
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4367—Establishing a secure communication between the client and a peripheral device or smart card
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N5/4403—
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0261—Improving the quality of display appearance in the context of movement of objects on the screen or movement of the observer relative to the screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/04—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
    • G09G2370/045—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller using multiple communication channels, e.g. parallel and serial
    • G09G2370/047—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller using multiple communication channels, e.g. parallel and serial using display data channel standard [DDC] communication
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/06—Consumer Electronics Control, i.e. control of another device by a display or vice versa
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/12—Use of DVI or HDMI protocol in interfaces along the display data pipeline

Definitions

  • the present invention relates to a method and an apparatus for transmitting and receiving data by using HDMI (High Definition Multimedia Interface) and more specifically, a method and an apparatus for transmitting and receiving motion information obtained through a capture device between a sink device and a source device by using HDMI.
  • HDMI High Definition Multimedia Interface
  • HDMI is an interface/standard that develops a Digital Visual Interface (DVI) which is an interface standard of personal computers and displays for a use of AV electronic products. Since the HDMI transmits video/audio from a player to a display device without compressing it, there is very little latency between a source device and a sink device. And the HDMI has high format compatibility since the HDMI does not require separate decoder chip or software. In addition, the HDMI is available to make wirings between AV devices be simplified, which were complicated because video signals, audio signals and control signals are transmitted on only one cable, and the HDMI provides a High-bandwidth Digital Content Protection (HDCP) technique, thereby providing the copyright protection function.
  • HDMI High-bandwidth Digital Content Protection
  • An object of the present invention is to provide a method for transmitting and receiving data between a sink device and a source device through HDMI.
  • an object of the present invention is to provide a method for transmitting and receiving image data or video data between a sink device equipped with a capture device (for example, camera) and a source device.
  • a capture device for example, camera
  • an object of the present invention is to provide a method for delivering motion data and information to a source device through a sink device equipped with a capture device.
  • an object of the present invention is to provide a method for delivering motion data and information from a source device equipped with a capture device to a sink device in case the source device supports the capture device for the sink device not equipped with a capture device.
  • the present invention provides a method and an apparatus for transmitting and receiving data by using HDMI (High Definition Media Interface).
  • HDMI High Definition Media Interface
  • a method for transmitting and receiving data comprises requesting reading EDID (Extended Display Identification Data) from a sink device in case the sink device is connected; receiving from the sink device EDID including capability information of the sink device, wherein the capability information indicates whether the sink device is capable of processing gesture information; requesting gesture information indicating a predetermined gesture which may be recognized or extracted from the sink device on the basis of the EDID; and receiving the gesture information from the sink device.
  • EDID Extended Display Identification Data
  • the gesture information according to the present invention is received through raw data including the gesture information or received as metadata extracted from the raw data, wherein the raw data corresponds to video data or image data obtained from the sink device, and the metadata represents the gesture information.
  • the gesture information according to the present invention is received through at least one of an HDMI CEC (Consumer Electronics Control) channel or HEC (HDMI Ethernet Channel).
  • HDMI CEC Consumer Electronics Control
  • HEC HDMI Ethernet Channel
  • the gesture information in case the gesture information is received as the metadata, the gesture information is received through the HDMI CEC channel or HEC, and in case the gesture information is received through the raw data, the gesture information is received through the HEC.
  • the method according to the present invention comprises receiving an inquiry message requesting status information indicating availability of the HEC from the sink device; and transmitting a first reporting message including the status information to the sink device, wherein the status information indicates either activation or inactivation of the HEC.
  • the method according to the present invention further comprises receiving a set message which activates the HEC from the sink device in case the status information indicates inactivation of the HEC; and activating the HEC and transmitting a second reporting message which includes status information indicating activation status of the HEC to the sink device.
  • a method receiving a write request of capability information of the source device included in an SCDCS (Status and Control Data Channel Structure) from the source device, wherein the capability information indicates whether the sink device is capable of processing the gesture information, comprises reading the written capability information; requesting gesture information indicating a predetermined gesture that may be recognized or extracted from the source device on the basis of the reading result; and receiving the gesture information from the source device.
  • SCDCS Status and Control Data Channel Structure
  • the gesture information according to the present invention is received through raw data including the gesture information or received as metadata extracted from the raw data, wherein the raw data corresponds to video data or image data obtained from the sink device, and the metadata represents the gesture information.
  • the gesture information according to the present invention is received through at least one of an HDMI CEC (Consumer Electronics Control) channel or HEC (HDMI Ethernet Channel).
  • HDMI CEC Consumer Electronics Control
  • HEC HDMI Ethernet Channel
  • the gesture information in case the gesture information is received as the metadata, the gesture information is received through the HDMI CEC channel or HEC, and in case the gesture information is received through the raw data, the gesture information is received through the HEC.
  • the method according to the present invention comprises receiving an inquiry message requesting status information indicating availability of the HEC from the sink device; and transmitting a first reporting message including the status information to the sink device, wherein the status information indicates either activation or inactivation of the HEC.
  • the method according to the present invention further comprises receiving a set message which activates the HEC from the sink device in case the status information indicates inactivation of the HEC; and activating the HEC and transmitting a second reporting message which includes status information indicating activation status of the HEC to the sink device.
  • a source device comprises an HDMI transmitter transmitting and receiving data through HDMI; and a controller controlling the HDMI transmitter, wherein the source device requests a sink device to read EDID (Extended Display Identification Data) in case the sink device is connected; receives from the sink device EDID including capability information of the sink device; requests gesture information indicating a predetermined gesture which may be recognized or extracted from the sink device on the basis of the EDID; and receives the gesture information from the sink device, wherein the capability information indicates whether the sink device is capable of processing the gesture information.
  • EDID Extended Display Identification Data
  • the gesture information according to the present invention is received through raw data including the gesture information or received as metadata extracted from the raw data, wherein the raw data corresponds to video data or image data obtained from the sink device, and the metadata represents the gesture information.
  • the gesture information according to the present invention is received through at least one of an HDMI CEC (Consumer Electronics Control) channel or HEC (HDMI Ethernet Channel).
  • HDMI CEC Consumer Electronics Control
  • HEC HDMI Ethernet Channel
  • the gesture information in case the gesture information is received as the metadata, the gesture information is received through the HDMI CEC channel or HEC, and in case the gesture information is received through the raw data, the gesture information is received through the HEC.
  • a sink device comprises an HDMI transmitter transmitting and receiving data through HDMI; and a controller controlling the HDMI transmitter, wherein the sink device receives a write request of capability information of the source device included in an SCDCS (Status and Control Data Channel Structure) from the source device; reads the written capability information; requests gesture information indicating a predetermined gesture that may be recognized or extracted from the source device on the basis of the reading result; and receives the gesture information from the source device, wherein the capability information indicates whether the sink device is capable of processing the gesture information.
  • SCDCS Status and Control Data Channel Structure
  • the gesture information according to the present invention is received through raw data including the gesture information or received as metadata extracted from the raw data, wherein the raw data corresponds to video data or image data obtained from the sink device, and the metadata represents the gesture information.
  • the gesture information according to the present invention is received through at least one of an HDMI CEC (Consumer Electronics Control) channel or HEC (HDMI Ethernet Channel).
  • HDMI CEC Consumer Electronics Control
  • HEC HDMI Ethernet Channel
  • the gesture information in case the gesture information is received as the metadata, the gesture information is received through the HDMI CEC channel or HEC, and in case the gesture information is received through the raw data, the gesture information is received through the HEC.
  • HDMI is capable of transmitting and receiving data between a sink device and a source device.
  • bilateral communication is made possible, by which a source device and a sink device may transmit and receive motion information to and from each other.
  • motion data and/or motion information may be transmitted to a source device through a sink device equipped with a capture device.
  • a sink device not equipped with a capture device may transmit motion data and/or motion information through a source device equipped with a capture device.
  • FIG. 1 illustrates an HDMI system and data transmission and reception channels included in the HDMI system according to one embodiment of the present invention.
  • FIG. 2 illustrates a source device and a sink device in a HDMI system according to an embodiment of the present invention.
  • FIG. 3 illustrates an EDID structure according to an embodiment of the present invention.
  • FIGS. 4 to 5 illustrate an embodiment of an EDID extension block.
  • FIG. 6 illustrates an HF (HDMI Forum)-VSDB (Vendor-Specific Data Block) according to an embodiment of the present invention.
  • FIG. 7 illustrates an SCDC (Status and Control Data Channel) structure according to an embodiment of the present invention.
  • FIG. 8 illustrates an HDMI system using a CEC channel according to an embodiment of the present invention.
  • FIGS. 9 and 10 illustrate a structure of a CEC message according to an embodiment of the present invention.
  • FIG. 11 illustrates a method for transmitting and receiving data through HDMI according to an embodiment of the present invention.
  • FIG. 12 illustrates a method for transmitting and receiving data through HEAC (HDMI Ethernet Audio Channel) of HDMI according to an embodiment of the present invention.
  • HEAC HDMI Ethernet Audio Channel
  • FIG. 13 illustrates state transition of a HEC channel and a CEC (Capability Discovery and Control) message according to an embodiment of the present invention.
  • FIG. 14 illustrates a MAC frame format according to an embodiment of the present invention.
  • FIG. 15 illustrates one example of gesture information according to an embodiment of the present invention.
  • FIG. 16 illustrates a method for transmitting and receiving A/V data according to an embodiment of the present invention.
  • FIG. 17 illustrates a method for transmitting gesture information through CEC (Consumer Electronics Control) in case a sink device according to an embodiment of the present invention supports a camera.
  • CEC Consumer Electronics Control
  • FIG. 18 illustrates a method for transmitting gesture information through CEC (Consumer Electronics Control) in case a source device according to an embodiment of the present invention supports a camera.
  • CEC Consumer Electronics Control
  • FIG. 19 illustrates a method for transmitting gesture information through HEC (HDMI Ethernet Channel) in case a sink device according to an embodiment of the present invention supports a camera.
  • HEC HDMI Ethernet Channel
  • FIG. 20 illustrates a method for transmitting gesture information through HEC (HDMI Ethernet Channel) in case a source device according to an embodiment of the present invention supports a camera.
  • HEC HDMI Ethernet Channel
  • FIG. 21 illustrates a method for checking whether a source device according to the present invention is capable of processing gesture information of a sink device through EDID.
  • FIG. 22 illustrates HF-VSDB according to another embodiment of the present invention.
  • FIG. 23 illustrates a method for checking whether a sink device according to the present invention is capable of processing gesture information of a source device through SCDCS.
  • FIG. 24 illustrates a method for transmitting and receiving gesture information of a source device through CEC (Consumer Electronics Control) according to a yet another embodiment of the present invention.
  • CEC Consumer Electronics Control
  • FIG. 25 illustrates a method for transmitting and receiving gesture information of a sink device through CEC (Consumer Electronics Control) according to a still another embodiment of the present invention.
  • CEC Consumer Electronics Control
  • FIG. 26 illustrates a method for transmitting and receiving gesture information of a sink device through HEC (HDMI Ethernet Channel) according to a further embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • FIG. 27 illustrates a method for transmitting and receiving gesture information of a source device through HEC (HDMI Ethernet Channel) according to an additional embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • FIG. 28 illustrates a method for transmitting and receiving gesture information according to a yet even another embodiment of the present invention.
  • FIGS. 29 and 30 illustrate a structure of a CEC (Consumer Electronics Control) command according to an embodiment of the present invention.
  • CEC Consumer Electronics Control
  • FIGS. 31 and 32 illustrate a data structure for the case in which gesture information is transmitted through HEC (HDMI Ethernet Channel) according to an embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • FIG. 1 illustrates an HDMI system and data transmission and reception channels included in the HDMI system according to one embodiment of the present invention.
  • HDMI system Devices that transmit and receive video/audio/control data using HDMI can be called collectively an HDMI system, and the HDMI system can comprise a source device 1010 , a sink device 1020 , and an HDMI cable.
  • a device that transmits video/audio data through HDMI corresponds to the source device 1010
  • a device that receives video/audio data through HDMI corresponds to the sink device 1020
  • an HDMI cable supporting data transmission and reception connects the two devices.
  • the HDMI cables and the connectors may perform pairing of four channels that provides a Transition Minimized Differential Signaling (TMDS) data channel and a TMDS clock channel.
  • TMDS Transition Minimized Differential Signaling
  • the TMDS data channels may be used for forwarding video data, audio data and auxiliary data.
  • the HDMI system provides a VESA (Video Electronics Standards Association) DDC (Display Data Channel).
  • the DDC is used for configuration of one source device and one sink device and exchange of status information between them.
  • a CEC protocol can provide a high-level control function among various audio-visual products in a user environment and may be used optionally.
  • an optional HEAC HDMI Ethernet and Audio Return Channel
  • Video data, audio data and supplementary data may be transmitted and received through three TMDS data channels.
  • a TMDS clock runs a video pixel rate, and is transmitted through a TMDS clock channel.
  • the TMDS clock may be used as a reference frequency for data recovery in three TMDS data channels in an HDMI receiver.
  • the data of 8 bits per TMDS data channel may be transformed into a sequence of 10 bits of which transition is minimized, which is DC balanced, and transmitted in serial manner with a rate of 10 bits per TMDS clock period.
  • the HDMI uses a packet structure.
  • data may be transmitted in word of 10 bits which is generated by using a BCH error correction code and an error reduction coding.
  • the source device can figure out configuration information and available functions of the sink device by reading out E-EDID (Enhanced Extended Display Identification Data) of the sink device in the DDC (Display Data Channel).
  • E-EDID Enhanced Extended Display Identification Data
  • DDC Display Data Channel
  • the E-EDID may be called EDID information.
  • a utility line can be used for an optional extension function such as HEAC.
  • FIG. 2 illustrates a source device and a sink device in a HDMI system according to an embodiment of the present invention.
  • the device that transmits video/audio data through HDMI corresponds to the source device 2100
  • the device that receives video/audio data through HDMI corresponds to the sink device 2200 .
  • the source device 2100 can include at least one of a display unit 2110 , user input interface unit 2120 , video encoding unit (video encoder) 2130 , control unit 2140 , HDMI transmitter 2150 , memory unit 2160 , storage unit 2170 , multimedia unit 2180 , or power supply unit 2190 .
  • the sink device 2200 can include at least one of an EDID EEPROM 2210 , video decoding unit 2220 , display unit 2230 , user input interface unit 2240 , HDMI receiver 2250 , control unit 2260 , power supply unit 2270 , memory unit 2280 , or multimedia unit 2290 . In what follows, descriptions about units performing the same operation will not be repeated.
  • the source device 2100 represents a physical device transmitting or streaming contents stored in the storage unit to the sink device 2200 .
  • the source device 2100 can send a request message to the sink device; or receive and process a request message from the sink device.
  • the source device 2100 can provide an UI through which a response message that the sink device 2200 transmits with respect to a transmitted request message is processed and delivered to the user, and in case the source device 2100 includes the display unit 2110 , the UI can be displayed.
  • the sink device 2200 can receive contents from the source device 2100 , transmit a request message to the source device 2100 , or transmit a response message by processing a message received from the source device 2100 .
  • the sink device 2200 can also provide an UI through which a response message received from the source device 2100 is processed and delivered to the user, and in case the sink device 2200 includes the display unit 2230 , the UI can be displayed.
  • the source device 2100 and the sink device 2200 can include a user input interface unit 2120 , 2240 that receives the user's action or input, and as an example, the user input interface 2120 , 2240 can correspond to a remote controller, voice reception/recognition device, or touch input sensing/receiving device.
  • the memory unit 2160 , 2280 represents a volatile physical device in which various types of data are stored temporarily.
  • the storage unit 2170 represents a nonvolatile physical device in which various types of data can be stored.
  • the EDID EEPROM 2210 represents an EEPROM that stores EDID information.
  • the aforementioned memory unit, storage unit, and EDID EEPROM all perform the function of storing data, which may be called collectively a memory unit.
  • the display unit 2110 , 2230 displays data received through HDMI, data stored in a content storage, or UI on a screen according to the control of the control unit.
  • the multimedia unit 2180 , 2290 plays various types of multimedia contents.
  • the multimedia unit 2180 , 2290 may be implemented independently of the control unit 2140 , 2260 or implemented as one physical component together with the control unit.
  • the power supply unit 2190 , 2270 supplies power required for operating a source device, sink device, and sub-units belonging to the source and sink devices.
  • the HDMI transmitter 2150 is the unit that is installed in the source device 2100 and transmits and receives data through HDMI.
  • the HDMI transmitter 2150 transmits and receives data including a command between devices and a request, action, or response message as well as audio/video data.
  • the video encoding unit 2130 compresses video data to be transmitted through the HDMI transmitter 2150 .
  • the HDMI receiver 2250 is the unit that is installed in the sink device 2200 and transmits and receives data through HDMI.
  • the HDMI receiver 2250 transmits and receives data including a command between device
  • the video decoding unit 2130 performs decompression of compressed video data received through the HDMI receiver 2250 .
  • channels, data structure, and functions provided by the HDMI will be described in more detail.
  • the HDMI system provides a DDC (Display Data Channel), which is a protocol standard defined by the VESA (Video Electronics Standard Association) specifying transmission of digital information between the monitor and the computer graphic adaptor.
  • DDC Display Data Channel
  • VESA Video Electronics Standard Association
  • HDMI devices can transmit information about display modes supported by the monitor to the graphic adaptor through the DDC, and the graphic adaptor can transmit images to the monitor according to the information.
  • the VGA standard defined four pins (Pin 11, 12, 4, and 15) of an analog VGA connector for the purpose of recognizing monitor types; among the four pins, only the pin 11, 12, and 4 have been actually used, and 7 types of monitor types could be recognized.
  • the following provide DDC specifications for the respective versions.
  • Pin 9 is used to supply 5V DC power (up to 50 mA) from the computer to the monitor to read the EDID stored in the EEPROM even if the monitor is powered off.
  • FIG. 3 illustrates an EDID structure according to an embodiment of the present invention.
  • EDID is a data structure containing various types of information about the display device defined in the VESA and can be transmitted to the source device through the DDC channel or read by the source device.
  • the data structure of version 1.3 has been used in the IT display device, CE display device, and video interface (HDMI).
  • FIG. 3 shows an overview of the information represented by the respective addresses in the EDID data structure.
  • FIGS. 4 to 5 illustrate an embodiment of an EDID extension block.
  • FIG. 4 illustrates an EDID extension block
  • FIG. 5( a ) a video data block
  • FIG. 5( b ) an audio data block
  • FIG. 5( c ) a speaker allocation data block.
  • Timing information specified in the EDID is intended for IT display devices and can use the EDID 1.3 extension block defined in the CEA-861 to represent the timing information of CE display devices.
  • the version 3 CEA extension block is defined in the CEA-861B standard and specifies four optional data blocks (video, audio, speaker allocation, and vendor-specific data block).
  • the short video descriptor represents the video identification code defined in the CEA-861.
  • the short audio descriptor represents the audio format code defined in the CEA-861.
  • the speaker allocation data block descriptor represents the data block payload defined in the CEA-861.
  • FIG. 6 illustrates an HF (HDMI Forum)-VSDB (Vendor-Specific Data Block) according to an embodiment of the present invention.
  • the HF-VSDB shown in FIG. 6 is the data block that defines vendor-specific data, where HDMI can define HDMI-specific data by using the data block.
  • the HF-VSDB can be included in the E-EDID of the sink device, and in that case, it can be positioned at CEA extension version 3 within the E-EDID of the sink device.
  • the present invention is capable of signaling decompression capability information of the sink device through an HF-VSDB of the EDID, which will be described later.
  • FIG. 7 illustrates an SCDC (Status and Control Data Channel) structure according to an embodiment of the present invention.
  • the SCDC (Status and Control Data Channel) corresponds to a point-to-point communication protocol based on which the source device and the sink device exchange data with each other.
  • the SCDC communication can use the aforementioned DDC channel (I2C line).
  • the SCDC is a one-to-one communication protocol based on the I2C serial communication that enables HDMI source devices and sink devices to exchange data among them.
  • the SCDC includes a mechanism in which the sink device, an I2C slave, requests status check read from the source device, an I2C master, and the source device receiving the request reads the corresponding status from the sink device.
  • the SCDCS can be stored in the memory of the sink device and include data having the structure as shown in FIG. 7 .
  • R/W indicates whether the source device can only read or read/write the SCDCS data stored in the sink device.
  • FIG. 8 illustrates an HDMI system using a CEC channel according to an embodiment of the present invention.
  • HDMI CEC stands for HDMI Consumer Electronics Control, and as described earlier, it refers to the protocol designed for a network in which a plurality of multimedia products are connected to each other via an HDMI cable to allow a user to control various products simultaneously through a single operation. In what follows, it will be called HDMI CEC.
  • an HDMI CEC system largely comprises an HDMI sink device 1020 and an HDMI source device 1010 ; and the sink device 1020 may include an HDMI connector, HDMI receiver, CPU (Central Processing Unit), and CEC converter while the source device 1010 may include an HDMI connector, HDMI transmitter, CPU, and CEC converter.
  • the sink device 1020 may include an HDMI connector, HDMI receiver, CPU (Central Processing Unit), and CEC converter
  • the source device 1010 may include an HDMI connector, HDMI transmitter, CPU, and CEC converter.
  • the sink device 1020 may be called a receiver or an initiator while the source device 1010 may be called a transmitter or a follower.
  • an initiator may denote a device which initiates, delivers, and controls a command while a follower may denote a device which responds to a command, operates according to a request, and informs of the result after the operation.
  • the sink device 1020 may include a TV and a repeater while the source device 1010 may include a DVD player, set-top box (STB), personal computer, notebook, and recorder.
  • STB set-top box
  • the HDMI connector may include five types of connectors such as type A, B, C, D, and E.
  • type A may refer to an ordinary 19 pin HDMI connector
  • type B may refer to a 29 pin connector designed for transmitting UHD video
  • type C may refer to a 19 pin mini-connector
  • type D may refer to a 19 pin micro-connector
  • type E may refer to a connector for vehicles.
  • an HDMI transmitter and an HDMI receiver may include three TMDS (Transition Minimized Differential Signaling) data transmission channels which transmit multimedia information through an HDMI connector and control channels such as a clock channel, DDC (Display Data Channel) (not shown in the figure), CEC (Consumer Electronics Control) channel, utility channel (not shown in the figure), HPD (Hot Plug Detect) channel (not shown in the figure).
  • TMDS Transition Minimized Differential Signaling
  • DDC Display Data Channel
  • CEC Consumer Electronics Control
  • utility channel not shown in the figure
  • HPD Hot Plug Detect
  • the TMDS channel comprises three data transmission channels and transmits video and audio data.
  • the CEC channel is a control channel delivering a CEC protocol which is a set of control commands by which devices connected through HDMI may be controlled by a TV remote controller.
  • the CEC channel enables one touch play, by which a user may play various device connected through HDMI by using a TV remote controller without resorting to separate control of the devices, thereby improving user convenience.
  • the CPU of the sink device 1020 may include a graphic user interface (GUI) layer, application layer, and CEC protocol layer while the CPU of the source device 1010 may include an application layer and CEC protocol layer.
  • GUI graphic user interface
  • the GUI layer processes data received through HDMI receiver and outputs the processed data through a graphic user interface
  • the application layer processes data according to a method defined by the user
  • the CEC protocol layer processes received CEC data and delivers required information to a upper layer.
  • the CEC protocol layer may perform physical address allocation and logical address allocation to which the present invention is applied.
  • the CEC converter of the sink device 1020 and the source device 1010 may generate a signal for data transmission or manages re-transmission for reliable transmission, for example, frame transmission, line error handling, and frame validation. Also, the CEC converter performs the role of an interface which processes an HDMI CEC signal through data communication with the CPU in frame units.
  • the signal processing unit to which the present invention is applied may perform the role of processing a command, request, action, or response message or data between devices.
  • FIGS. 9 and 10 illustrate a structure of a CEC message according to an embodiment of the present invention.
  • the CEC enables the user to control a CEC-enabled device connected through HDMI by using a remote controller.
  • a remote controller When connected to each other through HDMI, physical addresses are assigned to the respective devices, and logical addresses are additionally assigned to CEC-enabled devices.
  • Each individual device may exchange messages through a CEC line at the occurrence of an event, and each message may be used to perform the following functions.
  • a CEC message may be transmitted through a CEC frame, and a CEC frame may comprise a start bit, a header block, a first data block, and a second data block as shown in FIG. 9 .
  • the size of each block is 10 bits, and the size of the largest message excluding the start bit is 16*10 bits.
  • the header block may include address of a source device and destination address
  • the first data block may comprise an opcode block
  • the second data block may comprise an operand block.
  • an opcode may denote the name used for identifying a message.
  • Each data block and header block may comprise an information bits field, EOM (End Of Message) field, and ACK field.
  • EOM End Of Message
  • the information bits field may include data, opcode, or address.
  • the EOM (End Of Message) field indicates the last block of a message; for example, if the EOM field is 0, it indicates existence of one or more data blocks while, if the EOM field is 1, it indicates that a message is completed. Even if a message contains additional data after the EOM field is transmitted, a device has to ignore the additional data.
  • the ACK field is sent by being set to ‘1’ by the initiator, and the follower indicates reception of a message by setting the ACK field to ‘0’.
  • FIG. 11 illustrates a method for transmitting and receiving data through HDMI according to an embodiment of the present invention.
  • the source device 1010 changes the power line from low level to high level and applies a current S 11010 . Subsequent to the aforementioned operation, the source device 1010 may operate the EEPROM in which EDID information of the sink device 1020 is stored and related circuit.
  • the sink device 1020 switches the HPD (Hot Plug Detect) line from low to high level to inform the source device 1010 of the fact that a cable has been connected properly and EDID information may be accessible as EDID-related circuit has been activated S 11020 .
  • HPD Het Plug Detect
  • the source device 1010 may transmit an EDID information read request to the sink device 1020 through DDC S 11030 .
  • the sink device 1020 may transmit the EDID information stored in the EEPROM through the DDC S 11040 .
  • the EDID information may be transmitted being included in the aforementioned VSDB.
  • the source device 1010 and the sink device 1020 perform an encryption protocol according to a HDCP (High-bandwidth Digital Content Protection) key exchange procedure through the DDC S 11050 .
  • HDCP High-bandwidth Digital Content Protection
  • the sink device 1020 transmits ⁇ Give Deck Status>[“on”] message to the source device 1010 , which is a CEC message for turning on the deck of the sink device 1020 , S 11060 .
  • the source device 1010 may inform the sink device 1020 of the deck status of the source device 1010 , S 11070 .
  • the sink device 1020 may make the source device 1010 play A/V data by transmitting ⁇ Play> through the CEC S 11080 .
  • the source device may inform that the deck of the source device 1010 is being played S 11090 .
  • the source device 1010 may transmit video data, audio data, and infoframe to the sink device 1020 through the TMDS S 11100 .
  • the sink device 1020 may make the source device 1010 skip the current chapter by transmitting a ⁇ Dec Control>[“Skip Forward”] message through the CEC S 11110 .
  • the source device 1010 which has skipped the current chapter may report the deck status by transmitting a ⁇ Deck Status>[“Skip Forward”] message S 11120 .
  • the source device may transmit video data, audio data and/or infoframe to the sink device 1010 through the TDMS S 11130 .
  • the present invention proposes a method for transmitting and receiving gesture information obtained from a capture device through a method for transmitting and receiving data by using HDMI.
  • FIG. 12 illustrates a method for transmitting and receiving data through HEAC (HDMI Ethernet Audio Channel) of HDMI according to an embodiment of the present invention.
  • HEAC HDMI Ethernet Audio Channel
  • HEAC stands for HDMI Ethernet Audio Channel.
  • Ethernet complies with the IEEE 802.3 specification
  • audio data comply with the IEC60958-1 specification.
  • Pin 14 is assigned to utility/HEAC+ (optional, HDMI 1.4+ with an audio return channel and Ethernet), pin 17 is DDC/CEC/HEC ground, and pin 19 is HEAC ⁇ (optional, HDMI 1.4+ with an audio return channel and Ethernet).
  • the HEC manages Ethernet connectivity of HDMI, supporting full duplex mode and 100 Mbps network speed by using a 100 base-tx type cable.
  • the HEC network consists of a sink device, repeater, and source device connected to each other though HEAC cables, and a repeater or a source device equipped with at least two HEC connectors has internal layer 2 switch (L2SW) for forwarding the MAC frame.
  • L2SW layer 2 switch
  • ARC stands for Audio Return Channel and may apply a digital sound signal transmitted to a source device through a sink device, for example, TV through a repeater inversely to the repeater, for example, an A/V amp via the HDMI. Therefore, a TV transmitting an audio signal becomes a source device, and which audio format the amp supports may be checked by using the CEC.
  • FIG. 13 illustrates state transition of a HEC channel and a CEC (Capability Discovery and Control) message according to an embodiment of the present invention.
  • HEC CDC Capability Discovery and Control
  • the HEC may have the following states:
  • the HEC CDC is transmitted through the CEC and may discover an HDEC in the Potential HEC state.
  • the HEC CDC may activate or deactivate an HEC and deliver channel status.
  • the HEC state of a device becomes the Potential HEC (PHEC) state.
  • a device to which HDMI is connected may transmit ⁇ CDC_HEC_Discover> message to discover HEC capability of a different device connected by HDMI S 13010 .
  • the different device may transmit ⁇ CDC_HEC_ReportState> message in response to the ⁇ CDC_HEC_Discover> message.
  • the device may check whether a different device connected by HDMI supports HEC function or not.
  • the device may transmit ⁇ CEC_HEC_SetState> message to the different device to activate the HEC.
  • the different device which receives the ⁇ CEC_HEC_SetState> message may activate all of HECs between the device and the different device and transmit ⁇ CEC_HEC_ReportState> message to the device to report activation state of HEC S 13020 .
  • the device may transmit ⁇ CDC_HEC_SetState> message to deactivate the HEC in the activation state S 13050 .
  • the different device which receives the ⁇ CDC_HEC_SetState> message may deactivate all of the HECs between the device and the different device.
  • those devices not supporting the HEC function may return to the PHEC state S 13040 .
  • FIG. 13( b ) illustrates the CDC message frame structure, and the CDC message may comprise a start bit field, CEC header block field, CEC opcode block field, CEC initiator physical address field, CDC opcode field and/or CDC parameter field.
  • CEC header block field Since the start bit field, CEC header block field, CEC opcode block field, and CEC initiator physical address field are the same as illustrated in FIGS. 9 and 10 , descriptions thereof will be omitted.
  • the CDC opcode field indicates to which message the CDC message corresponds, and the CDC parameter field may include parameters of each CDC message.
  • the present invention proposes a method for transmitting and receiving data obtained by a capture device through the HEC described above.
  • FIG. 14 illustrates a MAC frame format according to an embodiment of the present invention.
  • FIG. 14 illustrates the structure of the MAC frame forwarded through the HEC described above, where data may be delivered by using the MAC data frame (IEEE 802.3).
  • the present invention proposes a method for transmitting data obtained from a capture device by a sink device or a source device to the source device or sink device through HEC by using the MAC frame.
  • FIG. 15 illustrates one example of gesture information according to an embodiment of the present invention.
  • Devices equipped with a capture device may recognize or extract a predetermined gesture from video or images captured through the capture device.
  • the gestures recognized or extracted as described above are called gesture information.
  • the aforementioned gesture information basically includes (a) face information, hand information, body information of a human, or a combination of the aforementioned information.
  • the information about a human face may be obtained by using a technology such as face detection, face recognition, face part tracking, or gaze tracking.
  • the hand or body information may include hand gesture representing motion, position, or shape of a hand; finger gesture representing motion, position, or shape of a finger; and body gesture representing motion, position, or shape of a body.
  • the source device or the sink device In order for the sink device or the source device to recognize or extract specific information such as the gesture information from the video or the image, the source device or the sink device needs to be informed beforehand about which information the gesture information describes. In other words, the source device or the sink device has to be informed of specific technology or information beforehand to recognize or extract the gesture information.
  • the gesture information may be transmitted as raw data such as the video or the image; or as metadata representing the gesture information in the raw data.
  • the metadata may represent x-axis information, y-axis information, or coordinates such as width and height.
  • the metadata consist of simple, small amount of data, being convenient for transmission and reception.
  • the raw data are YUV video or image expressing color by using a brightness signal (Y) and chrominance signals (U, V) captured by a capture device; and it is sufficient to have only the brightness signal (Y) to extract metadata from the raw data.
  • Y brightness signal
  • U, V chrominance signals
  • the present invention proposes a method for transmitting the aforementioned gesture information to a sink device or a source device through HDMI.
  • FIG. 16 illustrates a method for transmitting and receiving A/V data according to an embodiment of the present invention.
  • a sink device and a source device may transmit and receive A/V data through HDMI.
  • the source device 1010 and the sink device 1020 are connected to each other through an HDMI cable S 16010 .
  • the source device 1010 switches a 5V power line from low level to high level and applies a current S 16020 .
  • the source device 1010 may operate EEPROM which stores EDID information of the sink device 1020 and related circuit.
  • the sink device 1020 switches the HPD (Hot Plug Detect) line from low to high level S 16030 to inform the source device 1010 of the fact that a cable has been connected properly and EDID information may be accessible as EDID-related circuit has been activated.
  • HPD Het Plug Detect
  • the source device 1010 may transmit an EDID information read request to the sink device 1020 through DDC S 16040 .
  • the sink device 1020 may transmit the EDID information stored in the EEPROM through the DDC S 16050 .
  • the EDID information may be transmitted being included in the aforementioned VSDB.
  • the source device 1010 may determine operation parameters (timing, format, and so on) of A/V data to be transmitted to the sink device 1020 by parsing the received EDID information S 16060 and transmit to the source device determined operation parameters related to uncompressed A/V data to be transmitted.
  • the method illustrated in FIG. 16 is intended for transmitting and receiving A/V data stored in a source device, but it may not be used for transmitting and receiving video or image data obtained from a capture device (for example, camera or camcorder) installed in the source device or sink device, or may not be used for transmitting and receiving gesture information included in the video or image data.
  • a capture device for example, camera or camcorder
  • the present invention proposes a method for transmitting and receiving not only video or image data obtained from a capture device but also gesture information obtained from the video or image data through HDMI.
  • FIG. 17 illustrates a method for transmitting gesture information through CEC (Consumer Electronics Control) in case a sink device according to an embodiment of the present invention supports a camera.
  • CEC Consumer Electronics Control
  • data captured from the installed camera (video or image data) or gesture information included in the data may be transmitted to a source device through HDMI.
  • the sink device may transmit an EDID information read request through DDC in case the sink device is connected to the source device by an HDMI cable S 17010 .
  • the sink device may receive the EDID information stored in the EEPROM from the source device through the DDC S 17020 .
  • the EDID information may be transmitted being included in the aforementioned VSDB.
  • the EDID may include capability information of the source device related to the gesture information described with reference to FIG. 15 .
  • the capability information may indicate whether a source device is capable of processing gesture information and include information about whether the source device supports a service related to the gesture information or the source device is capable of processing raw data which may be video or image data containing the gesture information.
  • the sink device may determine whether the source device is capable of processing gesture information by parsing the EDID information received from the source device.
  • the sink device does not transmit the gesture information to the source device but may transmit only the data S 17070 .
  • the sink device may receive a request for transmitting gesture information through CEC from the source device S 17030 .
  • the sink device which has received the gesture information may obtain gesture information from a camera installed in the sink device S 17040 and transmit the gesture information obtained through CEC to the source device S 17050 .
  • the source device may process the received gesture information S 17060 .
  • FIG. 18 illustrates a method for transmitting gesture information through CEC (Consumer Electronics Control) in case a source device according to an embodiment of the present invention supports a camera.
  • CEC Consumer Electronics Control
  • data (video or image data) captured by the camera or gesture information included in the data may be transmitted to a sink device through HDMI.
  • the source device may transmit an EDID information read request to the sink device and receive the EDID stored in the EEPROM from the sink device through DDC in response to the EDID information read request S 18020 .
  • the EDID information may be transmitted being included in the aforementioned VSDB.
  • the EDID may include capability information of the sink device related to the gesture information described with reference to FIG. 15 .
  • the capability information may indicate whether a sink device is capable of processing gesture information and include information about whether the sink device supports a service related to the gesture information or the source device is capable of processing raw data which may be video or image data containing the gesture information.
  • the source device may determine whether the sink device is capable of processing gesture information by parsing the EDID information received from the sink device.
  • the source device may request the sink device to write capability information related to the gesture information of the source device into the SCDCS of the sink device.
  • the source device and the sink device may determine whether the sink device and the source device are capable of processing gesture information through reading the EDID and SCDCS.
  • the source device does not transmit the gesture information to the sink device but may transmit only the data S 18080 .
  • the source device may receive a request for transmitting gesture information through CEC from the sink device S 18040 .
  • the source device which has received the gesture information may obtain gesture information from a camera installed in the sink device S 18050 and transmit the obtained gesture information to the source device through CEC S 18060 .
  • the sink device may process the received gesture information S 18070 .
  • FIG. 19 illustrates a method for transmitting gesture information through HEC (HDMI Ethernet Channel) in case a sink device according to an embodiment of the present invention supports a camera.
  • HEC HDMI Ethernet Channel
  • FIG. 19 includes operations additional to the steps of FIG. 17 ; descriptions for the same steps appearing in FIGS. 17 and 19 will not be repeated; however, the descriptions may be applied to FIG. 19 in the same manner as in FIG. 17 .
  • the sink device may determine whether the source device is capable of processing gesture information by parsing the EDID information received from the source device.
  • the sink device does not transmit the gesture information to the source device but may transmit only the data S 19090 .
  • the sink device checks whether HEC has been activated or deactivated for the source device S 19030 ; in case the HEC has been deactivated, the sink device activates the HEC as described in FIG. 13 , S 19040 .
  • the sink device may receive a request for transmitting gesture information from the source device through HEC S 19050 .
  • the sink device which has received the request for gesture information may obtain gesture information from a camera installed in the sink device S 19060 and transmit the obtained gesture information to the source device through HEC S 19070 .
  • the source device may process the received gesture information S 19080 .
  • FIG. 20 illustrates a method for transmitting gesture information through HEC (HDMI Ethernet Channel) in case a source device according to an embodiment of the present invention supports a camera.
  • HEC HDMI Ethernet Channel
  • FIG. 20 includes operations additional to the steps of FIG. 18 ; descriptions for the same steps appearing in FIGS. 18 and 20 will not be repeated; however, the descriptions may be applied to FIG. 20 in the same manner as in FIG. 18 .
  • the source device in case the source device is incapable of processing the gesture information, the source device does not transmit the gesture information to the sink device but may transmit only the data S 20100 .
  • the source device checks whether HEC has been activated or deactivated for the sink device S 20040 ; in case the HEC has been deactivated, the source device activates the HEC as described in FIG. 13 , S 20050 .
  • the source device may receive a request for transmitting gesture information from the sink device through HEC S 20060 .
  • the source device which has received the request for gesture information may obtain gesture information from a camera installed in the source device S 20070 and transmit the obtained gesture information to the sink device through HEC S 20090 .
  • the source device may process the received gesture information S 20090 .
  • FIG. 21 illustrates a method for checking whether a source device according to the present invention is capable of processing gesture information of a sink device through EDID.
  • whether the sink device is capable of processing gesture information may be checked by reading EDID.
  • the source device 1010 may determine whether the sink device 1020 is capable of processing gesture information by reading the EDID of the sink device 1020 , S 21010 .
  • the source device 1010 may transmit an EDID information read request to the sink device 1020 , and the sink device 1020 may transmit EDID including capability information related to whether the sink device is capable of processing the gesture information to the source device 1010 in response to the request.
  • the EDID information according to an embodiment of the present invention may be transmitted as the VSDB described above, which will be described with reference to FIG. 22 .
  • the source device 1010 which has received the EDID may interpret the EDID to check whether the sink device 1020 is capable of processing gesture information, for example, whether the sink device 1020 is capable of processing gesture information and/or whether the sink device 1020 is capable of processing the data described above S 21020 .
  • the source device may request gesture information from the sink device S 21030 and receive the gesture information from the sink device 1020 .
  • FIG. 22 illustrates HF-VSDB according to another embodiment of the present invention.
  • FIG. 22 describes an embodiment which further includes capability information related to gesture information processing in addition to the HF-VSDB described in FIG. 6 .
  • the HF-VSDB of FIG. 22 is one embodiment of EDID information described above; and may include capability information and fields indicating capability of processing gesture information and raw data including the gesture information.
  • FIG. 22 defines a new HF-VSDB, in which the version number in the Version field may be set to 2 to distinguish the new HF-VSDB from the previous version.
  • the version number in the Version field may be set to 2 to distinguish the new HF-VSDB from the previous version.
  • at least one bit from among bit 5 to 4 of byte 6 block and bit 7 to 3 of byte 7 block of the HF-VSDB may be used.
  • the corresponding bit if the corresponding bit is set to 1, it may indicate that a sink device may receive gesture information while, if the corresponding bit is set to 0, it may indicate that the sink device may not receive gesture information.
  • the decompression capability information may include at least one of the following fields described with reference to FIG. 15 .
  • the HF-VSDB may be used in an embodiment of the present invention for determining a sink device's capability of processing gesture information through EDID.
  • FIG. 23 illustrates a method for checking whether a sink device according to the present invention is capable of processing gesture information of a source device through SCDCS.
  • the source device may write the source device's capability of processing gesture information through SCDC, and the sink device may check whether the source device is capable of processing gesture information by reading the SCDC.
  • S 23010 step is the same as S 22010 step of FIG. 22 , description thereof will be omitted.
  • the source device 1010 may request the sink device 1020 to write “1” into the RR_Enable field of the SCDCS of the sink device in case a read request is supported, but “0” otherwise S 23020 .
  • the source device 1010 may request writing to the field of SCDCS of the sink device related to processing of gesture information of the source device 1010 , S 23030 .
  • the sink device 1020 may determine whether the source device is capable of processing gesture information by reading the information written to the SCDCS, and if it is found that the source device is capable of processing gesture information, the sink device may request gesture information from the source device 1010 , S 23050 .
  • the sink device may receive only metadata, but if it is set to “1”, the sink device may receive metadata and/or raw data.
  • FIG. 24 illustrates a method for transmitting and receiving gesture information of a source device through CEC (Consumer Electronics Control) according to a yet another embodiment of the present invention.
  • CEC Consumer Electronics Control
  • the sink device may transmit gesture information to a source device through CEC.
  • the source device 1010 may determine whether the sink device 1020 is capable of processing gesture information.
  • the source device 1010 may request gesture information from the sink device 1020 through a CEC message requesting gesture information such as ⁇ Request Gesture capability> S 24010 .
  • the sink device 1020 which has received the request may transmit gesture information by including it in the Operand field of a CEC message such as ⁇ Response Gesture Data capability>[capability] in response to the request S 24020 .
  • FIG. 25 illustrates a method for transmitting and receiving gesture information of a sink device through CEC (Consumer Electronics Control) according to a still another embodiment of the present invention.
  • CEC Consumer Electronics Control
  • the source device may transmit gesture information to a sink device through CEC.
  • the sink device 1020 may determine whether the source device 1010 is capable of processing gesture information.
  • the sink device 1020 may request gesture information from the source device 1010 through a CEC message requesting gesture information such as ⁇ Request Gesture capability> S 25010 .
  • the source device 1010 which has received the request may transmit gesture information by including it in the Operand field of a CEC message such as ⁇ Response Gesture Data capability>[capability] in response to the request S 25020 .
  • FIG. 26 illustrates a method for transmitting and receiving gesture information of a sink device through HEC (HDMI Ethernet Channel) according to a further embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • a source device may request the sink device to transmit obtained gesture information and receive gesture information from the sink device through HEC (HEMI Ethernet Channel).
  • HEC HEMI Ethernet Channel
  • a source device 1010 may determine whether a sink device 1020 is capable of processing gesture information through the process of FIGS. 21 to 23 . Afterwards, the sink device 1020 may transmit a CEC message such as ⁇ CDC_HEC_InquireState> inquiring the state of the HEC as described in FIG. 13 through a CEC message S 26010 .
  • a CEC message such as ⁇ CDC_HEC_InquireState> inquiring the state of the HEC as described in FIG. 13 through a CEC message S 26010 .
  • the sink device 1020 has to perform a physical address allocation mechanism irrespective of whether the CEC function is supported for the HDMI host device while a physical address may not be allocated for other devices if the CEC function is not supported.
  • a physical address may consist of 4 digits.
  • the sink device 1020 generates a physical address 0.0.0.0 and reads an address from the EDID within the sink device 1020 .
  • the sink device 1020 has to generate a physical address of a source device 1010 connected to the sink device 1020 , and part of the EDID VSDB of the source device may be used for physical addresses.
  • the next digit value may be increased by 1.
  • the physical address of the source device is assumed to be 1.0.0.0.
  • the sink device 1020 may receive a CEC message such as ⁇ CDC_HEC_ReportState> including information related to the state of the HEC through the CEC in response to the CEC message inquiring the HEC state S 26020 .
  • a CEC message such as ⁇ CDC_HEC_ReportState> including information related to the state of the HEC through the CEC in response to the CEC message inquiring the HEC state S 26020 .
  • the ⁇ CEC_HEC_ReportState> message may include operands as shown in Table 1 below.
  • Ext Con 1 Indicates that an External inactive” Network Connection is supported but inactive
  • Ext Con Active 2 Indicates that an External Network Connection is supported and active
  • HEC Set State “Deactivate 0 1 byte Sets the required HEC HEC” Functionality state of the HDMI
  • Initiator's 0 ⁇ N ⁇ 4 bits The value of the nibble of the (Sink's) input 15 Initiator's physical address that is Port number different in the directly follower's physical address.
  • the sink device 1020 may transmit a CEC message such as ⁇ CDC_HEC_SetState> to the source device 1010 through the CEC S 26030 and activate all of HECs between the sink device 1020 and the source device 1010 .
  • the source device 1010 may transmit a CEC message such as ⁇ CDC_HEC_ReportState> including state information of the HEC to the sink device to inform the sink device 1020 that the HEC has been activated S 26040 .
  • the S 26030 and the S 26040 steps may not be performed.
  • the source device 1010 may request gesture information described with reference to FIG. 15 from the sink device 1020 , S 26050 .
  • the sink device 1020 which has received the request for gesture information may transmit type of gesture information that the sink device 1020 supports and transmit the gesture information obtained from the camera to the source device 1010 , S 26080 .
  • the S 26050 to S 26080 steps may be performed through a CEC channel or HEC, and the gesture information may be transmitted in the form of raw data, which is video or image data obtained by the camera, or metadata.
  • FIG. 27 illustrates a method for transmitting and receiving gesture information of a source device through HEC (HDMI Ethernet Channel) according to an additional embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • a sink device may request the source device to transmit gesture information and receive gesture information from the source device through HEC (HDMI Ethernet Channel).
  • HEC HDMI Ethernet Channel
  • the sink device 1020 may determine whether the source device 1010 is capable of processing gesture information. Afterwards, the source device 1010 may transmit a CEC message such as ⁇ CDC_HEC_InquireState> inquiring the state of the HEC described in FIG. 13 to the sink device 1020 , S 27010 .
  • a CEC message such as ⁇ CDC_HEC_InquireState> inquiring the state of the HEC described in FIG. 13 to the sink device 1020 , S 27010 .
  • the sink device 1020 has to perform a physical address allocation mechanism irrespective of whether the CEC function is supported for the HDMI host device while a physical address may not be allocated for other devices if the CEC function is not supported.
  • a physical address may consist of 4 digits.
  • the sink device 1020 generates a physical address 0.0.0.0 and reads an address from the EDID within the sink device 1020 .
  • the sink device 1020 has to generate a physical address of a source device 1010 connected to the sink device 1020 , and part of the EDID VSDB of the source device may be used for physical addresses.
  • the next digit value may be increased by 1.
  • the physical address of the source device is assumed to be 1.0.0.0.
  • the source device 1010 may receive a CEC message such as ⁇ CDC_HEC_ReportState> including information related to the state of the HEC through the CEC in response to the CEC message inquiring the HEC state S 27020 .
  • a CEC message such as ⁇ CDC_HEC_ReportState> including information related to the state of the HEC through the CEC in response to the CEC message inquiring the HEC state S 27020 .
  • the ⁇ CEC_HEC_ReportState> message may include operands as shown in Table 1 above.
  • the source device 1010 may transmit a CEC message such as ⁇ CDC_HEC_SetState> to the sink device 1020 through the CEC S 27030 and activate all of HECs between the source device 1010 and the sink device 1020 .
  • the sink device 1020 may transmit a CEC message such as ⁇ CDC_HEC_ReportState> including state information of the HEC to the source device to inform the source device 1010 that the HEC has been activated S 27040 .
  • the S 27030 and the S 27040 steps may not be performed.
  • the sink device 1020 may request gesture information described with reference to FIG. 15 from the source device 1020 , S 27050 .
  • the source device 1010 which has received the request for gesture information may transmit type of gesture information that the source device 1010 supports and transmit the gesture information obtained from the camera to the sink device 1020 , S 27080 .
  • the S 27050 to S 27080 steps may be performed through a CEC channel or HEC, and the gesture information may be transmitted in the form of raw data, which is video or image data obtained by the camera, or metadata.
  • FIG. 28 illustrates a method for transmitting and receiving gesture information according to a yet even another embodiment of the present invention.
  • a sink device equipped with a camera may transmit gesture information in the form of raw data or metadata according to whether a source device is capable of processing video or image data including gesture information.
  • the source device may transmit an EDID information read request to the sink device and receive the EDID stored in the EEPROM from the sink device through DDC in response to the EDID information read request S 28020 .
  • the EDID information may be transmitted being included in the aforementioned VSDB.
  • the EDID may include capability information of the source device related to the gesture information described with reference to FIG. 15 .
  • the capability information may indicate whether or not the gesture information can be processed and include information about whether the source device supports a service related to the gesture information or the source device is capable of processing raw data which may be video or image data containing the gesture information.
  • the sink device may determine whether the source device is capable of processing gesture information by parsing the EDID information received from the source device.
  • the source device may request the sink device to write capability information related to the gesture information of the source device into the SCDCS of the sink device.
  • the sink device may determine whether the sink device and the source device are capable of processing gesture information by reading out the SCDCS. In other words, the sink device may check whether the source device is capable of processing gesture information and raw data by reading out the Gesture_Handling field and the Gesture_Enable field of the SCDCS shown in FIG. 23( a ) .
  • the sink device may perform the process designated by the user or transmit raw data including gesture information to the source device S 28050 , and the source device may obtain gesture information by parsing the received raw data s 28060 .
  • the sink device may transmit metadata described with reference to FIG. 15 to the source device through the CEC S 28040 .
  • the sink device may transmit metadata to the source device through the HEC or CEC.
  • metadata may be transmitted mostly through the CEC.
  • the sink device in case the sink device is incapable of processing the gesture information, but only the source device is capable of processing raw data, the sink device may transmit raw data to the source device through the HEC, and even if the sink device is capable of processing gesture information, in case size of the raw data exceeds HEC Data Capability, the sink device may transmit meta data through the CEC or raw data through the HEC.
  • the sink device may transmit raw data through the HEC and metadata through the CEC.
  • the role of the sink device as shown in FIG. 28 may be performed by the source device and the role of the source device by the sink device.
  • FIGS. 29 and 30 illustrate a structure of a CEC (Consumer Electronics Control) command according to an embodiment of the present invention.
  • CEC Consumer Electronics Control
  • FIG. 29 illustrates CEC commands for requesting and transmitting gesture information according to the present invention
  • a sink device or a source device may request metadata through the ⁇ Request Gesture Capability> message and receive metadata through the ⁇ Report Gesture Capability> message.
  • FIG. 30 illustrates operands included in the CEC messages of FIG. 29 ; through the metadata, gesture information related to face, hand, and body may be transmitted.
  • the metadata may include coordinate information such as x-axis, y-axis, width, and height.
  • FIGS. 31 and 32 illustrate a data structure for the case in which gesture information is transmitted through HEC (HDMI Ethernet Channel) according to an embodiment of the present invention.
  • HEC HDMI Ethernet Channel
  • raw data including gesture information may be transmitted through a MAC frame of the HEC.
  • gesture information of the raw data may be transmitted being included in a data field of the MAC frame shown in FIG. 31 .
  • the gesture information included in the data field comprises a header and payload, and as shown in FIG. 32( c ) , the header may include a Format field, Resolution field, Gesture_op field, Gesture_Rev field and/or Payload Info field.
  • the gesture information may be transmitted being included in the payload field.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Hardware Design (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
US15/318,336 2014-06-12 2015-06-12 Method and device for transmitting/receiving data using HDMI Active 2035-09-04 US10474241B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/318,336 US10474241B2 (en) 2014-06-12 2015-06-12 Method and device for transmitting/receiving data using HDMI

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201462011046P 2014-06-12 2014-06-12
US15/318,336 US10474241B2 (en) 2014-06-12 2015-06-12 Method and device for transmitting/receiving data using HDMI
PCT/KR2015/005955 WO2015190877A1 (fr) 2014-06-12 2015-06-12 Procédé et dispositif de transmission/réception de données par hdmi

Publications (2)

Publication Number Publication Date
US20170115740A1 US20170115740A1 (en) 2017-04-27
US10474241B2 true US10474241B2 (en) 2019-11-12

Family

ID=54833878

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/318,336 Active 2035-09-04 US10474241B2 (en) 2014-06-12 2015-06-12 Method and device for transmitting/receiving data using HDMI

Country Status (3)

Country Link
US (1) US10474241B2 (fr)
KR (1) KR20170016845A (fr)
WO (1) WO2015190877A1 (fr)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10319336B2 (en) * 2016-02-16 2019-06-11 Samsung Electronics Co., Ltd. Electronic device and control method thereof
KR20180068470A (ko) * 2016-12-14 2018-06-22 삼성전자주식회사 다중 디스플레이 시스템을 구성하는 디스플레이 장치 및 그 제어 방법
KR102546765B1 (ko) * 2018-09-28 2023-06-22 한화비전 주식회사 비디오 모니터링 시스템
KR102222871B1 (ko) 2019-02-22 2021-03-04 삼성전자주식회사 디스플레이 장치 및 이의 영상 표시 방법
JP7003079B2 (ja) * 2019-03-14 2022-01-20 株式会社東芝 電子機器
JP7412090B2 (ja) * 2019-05-08 2024-01-12 株式会社ディーアンドエムホールディングス オーディオシステム
JP2021150791A (ja) * 2020-03-18 2021-09-27 ソニーグループ株式会社 撮像装置および撮像装置の制御方法
KR20220068008A (ko) * 2020-11-18 2022-05-25 엘지전자 주식회사 디스플레이 장치 및 그의 edid 정보 변경 방법
WO2022245182A1 (fr) * 2021-05-20 2022-11-24 엘지전자 주식회사 Procédé de transmission/réception de données vidéo et dispositif associé
KR20240029322A (ko) * 2022-08-26 2024-03-05 삼성전자주식회사 싱크 장치, 소스 장치 및 그 제어 방법

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080043421A (ko) 2006-11-14 2008-05-19 엘지전자 주식회사 디바이스 서치 및 인터페이스 방법
KR20090066582A (ko) 2007-12-20 2009-06-24 엘지전자 주식회사 에이치디엠아이 오디오 및 비디오 신호 분기방법
KR100935375B1 (ko) 2008-10-20 2010-01-06 (주) 마스타 화질 검사 시스템
US20100066919A1 (en) * 2006-11-07 2010-03-18 Sony Corporation Electronic equipment,control information transmission method, and control information reception method
US20110157310A1 (en) * 2008-09-02 2011-06-30 Hiroshi Mitani Three-dimensional video transmission system, video display device and video output device
US20140130091A1 (en) 2011-12-28 2014-05-08 Jiang Liu User interface interaction system and method for handheld device and tv set
US20160246057A1 (en) * 2014-01-20 2016-08-25 Sony Corporation Image display device and image display method, image output device and image output method, and image display system
US20170055005A1 (en) * 2014-05-01 2017-02-23 Sony Corporation Communication device or communication method, and computer program

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100066919A1 (en) * 2006-11-07 2010-03-18 Sony Corporation Electronic equipment,control information transmission method, and control information reception method
KR20080043421A (ko) 2006-11-14 2008-05-19 엘지전자 주식회사 디바이스 서치 및 인터페이스 방법
KR20090066582A (ko) 2007-12-20 2009-06-24 엘지전자 주식회사 에이치디엠아이 오디오 및 비디오 신호 분기방법
US20110157310A1 (en) * 2008-09-02 2011-06-30 Hiroshi Mitani Three-dimensional video transmission system, video display device and video output device
KR100935375B1 (ko) 2008-10-20 2010-01-06 (주) 마스타 화질 검사 시스템
US20140130091A1 (en) 2011-12-28 2014-05-08 Jiang Liu User interface interaction system and method for handheld device and tv set
US20160246057A1 (en) * 2014-01-20 2016-08-25 Sony Corporation Image display device and image display method, image output device and image output method, and image display system
US20170055005A1 (en) * 2014-05-01 2017-02-23 Sony Corporation Communication device or communication method, and computer program

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PCT International Application No. PCT/KR2015/005955, International Search Report dated Sep. 25, 2015, 2 pages.

Also Published As

Publication number Publication date
US20170115740A1 (en) 2017-04-27
KR20170016845A (ko) 2017-02-14
WO2015190877A1 (fr) 2015-12-17

Similar Documents

Publication Publication Date Title
US10474241B2 (en) Method and device for transmitting/receiving data using HDMI
US10085058B2 (en) Device and method for transmitting and receiving data using HDMI
US9819995B2 (en) Device and method for data transmission and reception using HDMI
KR102397289B1 (ko) Hdmi를 사용하여 데이터를 송수신하기 위한 방법 및 장치
US9942605B2 (en) Devices and methods for transmitting and receiving data of a sink device to change video display resolution based on error counter value detected at the sink device using HDMI
US9779687B2 (en) Device and method for transmitting and receiving data using HDMI
US10038871B2 (en) Method and device for transmitting and receiving power using HDMI
US10162769B2 (en) Method and apparatus for transmitting and receiving data using HDMI
US9554179B2 (en) Method and apparatus for transmitting and receiving data using HDMI
US10754537B2 (en) Method and apparatus for processing human interface device (HID)-based data using high-speed interface
US11818499B2 (en) Reception device, method for controlling reception device, and transmission/reception system
US10009650B2 (en) Method and apparatus for processing object-based audio data using high-speed interface
US20230199246A1 (en) Video data transmission and reception method using high-speed interface, and apparatus therefor

Legal Events

Date Code Title Description
AS Assignment

Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIM, DOKYUN;YANG, HYUNSIK;LEE, HYEONJAE;AND OTHERS;REEL/FRAME:044167/0600

Effective date: 20161226

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4