US20160134909A1 - System and method for distributing auxiliary data embedded in video data - Google Patents
System and method for distributing auxiliary data embedded in video data Download PDFInfo
- Publication number
- US20160134909A1 US20160134909A1 US15/001,514 US201615001514A US2016134909A1 US 20160134909 A1 US20160134909 A1 US 20160134909A1 US 201615001514 A US201615001514 A US 201615001514A US 2016134909 A1 US2016134909 A1 US 2016134909A1
- Authority
- US
- United States
- Prior art keywords
- data
- auxiliary data
- video data
- embedded
- auxiliary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/08—Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23614—Multiplexing of additional data and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/16—Analogue secrecy systems; Analogue subscription systems
- H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
- H04N21/2353—Processing of additional data, e.g. scrambling of additional data or processing content descriptors specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2665—Gathering content from different sources, e.g. Internet and satellite
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4348—Demultiplexing of additional data and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
- H04N21/8133—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
- H04N21/8173—End-user applications, e.g. Web browser, game
Definitions
- This invention relates generally to techniques for implementing television systems, and relates more particularly to a system and method for distributing auxiliary data embedded in video data.
- enhanced system capability to perform various advanced operations can offer additional benefits to the end user, but may also place increased demands on the control and management of various system components.
- an enhanced electronic system that effectively supports synchronized television widget functionality may benefit from methods providing flexible carriage of the data stream supporting this functionality.
- a system and method are disclosed for distributing auxiliary data embedded in video data.
- a content source or other appropriate entity initially produces content data that typically includes video data and audio data.
- the content source or other appropriate entity then creates auxiliary data to support advanced interactive features such as “synchronized widgets” or Internet-enhanced interactivity on a television device.
- the content source or other appropriate entity inserts or embeds the auxiliary data into the video data.
- the content source or other appropriate entity then encodes the audio data and the video data (including the embedded auxiliary data) to create a compressed distribution multiplex.
- the content source or other appropriate entity distributes the distribution multiplex in any effective manner to a television in an electronic network.
- a decoder device of a set-top box or other appropriate entity receives and decodes the distribution multiplex distributed by the content source to reproduce uncompressed audio data and uncompressed video data (including the embedded auxiliary data).
- a detection module of the television scans the video data to locate the embedded auxiliary data by utilizing any effective techniques.
- an extraction module of the television extracts the located auxiliary data from the video data.
- an auxiliary data module of the television processes the extracted auxiliary data to successfully display and support one or more interactive applications residing in the television.
- FIG. 1A is a block diagram of an electronic system, in accordance with one embodiment of the present invention.
- FIG. 1B is a diagram of a display from the television of FIG. 1A , in accordance with one embodiment of the present invention
- FIG. 2 is a block diagram for one embodiment of the content source of FIG. 1A , in accordance with the present invention
- FIG. 3 is a block diagram for one embodiment of the source memory of FIG. 2 , in accordance with the present invention.
- FIG. 4 is a block diagram for one embodiment of the auxiliary data of FIG. 3 , in accordance with the present invention.
- FIG. 5 is a block diagram for one embodiment of the television from FIG. 1A , in accordance with the present invention.
- FIG. 6 is a block diagram for one embodiment of the TV memory from FIG. 5 , in accordance with the present invention.
- FIGS. 7A and 7B are diagrams of auxiliary data embedded in video data, in accordance with two different embodiments of the present invention.
- FIGS. 8A-8C are a flowchart of method steps for distributing auxiliary data, in accordance with one embodiment of the present invention.
- the present invention relates to an improvement in television systems.
- the following description is presented to enable one of ordinary skill in the art to make and use the invention, and is provided in the context of a patent application and its requirements.
- Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the generic principles herein may be applied to other embodiments.
- the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.
- the present invention is described herein as a system and method for distributing auxiliary data embedded in video data, and includes a content source that embeds the auxiliary data into the video data.
- the content source then encodes the video data together with the auxiliary data to create a distribution multiplex including compressed video data.
- a decoder receives and decompresses the distribution multiplex to reproduce the video data with the auxiliary data embedded.
- a television or other device then detects and extracts the auxiliary data from the video data.
- the television or other device processes the auxiliary data to support a variety of possible interactive applications including displaying a synchronized widget on a display of the television.
- electronic system 110 may include, but is not limited to, a content source 114 , a set-top box 118 , an interface 126 , a television 122 , an optional network 134 , and an optional server 130 .
- electronic system 110 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with the FIG. 1A embodiment.
- any number of televisions 122 may be similarly deployed in electronic system 110 .
- network 134 and server 130 may not be included in all embodiments of the present invention.
- content source 114 may be implemented as one or more electronic devices or other entities that prepare and distribute content data, including video data and audio data, for reproduction by television 122 .
- content source 114 may be implemented as any appropriate entity.
- content source 114 may include a television broadcasting facility, a satellite television distribution facility, or an Internet server entity. Additional details regarding the implementation and utilization of content source 114 are further discussed below in conjunction with FIGS. 2-4 .
- content source 114 creates an encoded distribution multiplex containing the content data in a compressed format, and then distributes the distribution multiplex through a distribution network via path 116 to a decoder device.
- the decoder device is implemented in a set-top box 118 .
- the decoder device may be implemented as any appropriate entity, either external to, or integral with, television 122 .
- additional devices or entities may be interposed between content source 114 and set-top box 118 . Examples of such entities may include, but are not limited to, a broadcast network affiliate and a service provider (such as a satellite or cable head-end).
- set-top box 118 decodes the encoded distribution multiplex to generate uncompressed A/V data (video data and audio data) that is provided to television 122 via an appropriate interface 126 .
- interface 126 may be implemented in any effective manner.
- interface 126 may be implemented according to a High Definition Multimedia Interface (HDMI) standard that provides a high-speed parallel interface to deliver uncompressed video data and audio data, and control/timing signals to television 122 .
- Television 122 may then responsively receive and reproduce the video data and audio data for utilization by a system user. Additional details regarding the implementation and utilization of television 122 are further discussed below in conjunction with FIGS. 5-6 .
- HDMI High Definition Multimedia Interface
- electronic system 110 supports the creation and display of various types of “widgets” on television 122 in addition to the main content data. Widgets may include discrete areas that are displayed on television 122 to provide any desired type of information. Additional details regarding widgets are further provided below in conjunction with FIG. 1B .
- electronic system 110 advantageously supports synchronized widgets that provide information that is related to (synchronized) to the main content data that is currently being displayed on television 122 . In order to successfully support synchronized widgets, electronic system 110 also provides certain types of auxiliary data to television 122 .
- television 122 may obtain auxiliary data from any appropriate source including, but not limited to, content source 114 or server 130 .
- television 122 may communicate with server 130 via any effective network 134 including, but not limited to, the Internet. Additional details regarding the creation, distribution, and utilization of auxiliary data are further discussed below in conjunction with FIGS. 4, 7, and 8 .
- the present invention generally involves embedding auxiliary data in a video signal so that the auxiliary data may be quickly and easily recovered by receiving devices like television 122 .
- content source 114 inserts auxiliary data within a distributed video signal so that the auxiliary data travels through the distribution chain, comes into a consumer's home via a compressed interface (from a cable, satellite, or IPTV service provider), is de-compressed in set-top box 118 , and then travels to television 122 in an uncompressed format, where television 122 retrieves and utilizes the embedded auxiliary data to support synchronized widgets.
- the foregoing techniques are intended to circumvent service providers or other entities from blocking the consumer's access to auxiliary data that is required to provide enhanced functionality to television 122 .
- Certain cable, satellite, and IPTV entities typically provide system users with set-top boxes that are interfaced to digital televisions via HDMI uncompressed video interfaces or other appropriate means. If a content owner wishes to include auxiliary data (such as a URL, applet, etc.) with the content data, and if that auxiliary data travels with the content data as a separate digital stream (or as metadata within the compressed bit stream), the auxiliary data will be blocked at the set-top box 118 .
- auxiliary data such as a URL, applet, etc.
- a set-top box 114 does not pass ancillary data streams in the distribution multiplex, because the set-top box decodes only audio data and video data, and then passes only the uncompressed video data and audio data across to the television. Ancillary data streams are therefore unavailable to the television. If service providers (those offering the set-top boxes) perceive that providing access to any ancillary data is competitive to their business model, they may not be inclined to help the consumer electronics industry by providing such access.
- the present invention advantageously embeds auxiliary data within the video signal (encoded within the video image, not as a separate ancillary data stream).
- the present invention therefore successfully overcomes the architectural roadblock discussed above.
- the implementation and utilization of the FIG. 1A electronic system 110 is further discussed below in conjunction with FIGS. 1B-8B .
- FIG. 1B a diagram of a display 138 from the television 122 of FIG. 1A is shown, in accordance with one embodiment of the present invention.
- the FIG. 1B embodiment is presented for purposes of illustration, and in alternate embodiments, display 138 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with the FIG. 1B embodiment.
- display 138 includes a main screen region that typically displays video data provided by a content source 114 ( FIG. 1A ).
- display 138 also includes a widget 144 that resides in a discrete area displayed on display 138 to provide any desired type of additional information.
- widget 144 may be implemented in any desired shape or size, and may be displayed in any appropriate location.
- any desired number of different widgets are equally contemplated, including the possibility of multiple widgets on the display at any given time.
- display 138 supports synchronized widgets that function to provide information that is related (synchronized) to the video data that is currently being displayed on display 138 .
- widget 144 may be utilized to display financial information of specific relevance to the viewer (e.g., his/her investment portfolio) during a television program regarding economic news or investment topics.
- widget 144 may be utilized during a televised automobile race to display relevant information or statistics regarding specific race car drivers, racecars, or automobile racing in general. Additional details regarding the implementation and utilization of synchronized widgets 144 is further discussed below in conjunction with FIGS. 2-8B .
- content source 114 may include, but is not limited to, a central processing unit (CPU) 212 , a source memory 220 , and input/output interfaces (I/O interfaces) 224 .
- CPU central processing unit
- source memory 220 volatile and non-volatile memory
- I/O interfaces input/output interfaces
- content source 114 may be implemented using components and configurations in addition to, or instead of, those components and configurations discussed in conjunction with the FIG. 2 embodiment.
- content source 114 may alternately be implemented as any other desired type of electronic device or entity.
- CPU 212 may be implemented to include any appropriate and compatible microprocessor device(s) that preferably execute software instructions to thereby control and manage the operation of content source 114 .
- source memory 220 may be implemented to include any combination of desired storage devices, including, but not limited to, read-only memory (ROM), random-access memory (RAM), and various types of non-volatile memory, such as floppy disks or hard disks. The contents and functionality of source memory 220 are further discussed below in conjunction with FIGS. 3 and 4 .
- I/O interfaces 224 may include one or more input and/or output interfaces to receive and/or transmit any required types of information for content source 114 .
- content source 114 may utilize I/O interfaces 224 to communicate with other entities in electronic system 110 ( FIG. 1A ).
- a system user may utilize I/O interfaces 224 to communicate with content source 114 by utilizing any appropriate and effective techniques. Additional details regarding content source 114 are further discussed below in conjunction with FIGS. 3-4 .
- source memory 220 includes, but is not limited to, one or more source applications 312 , video data 316 , audio data 318 , an encoder 320 , auxiliary (aux) data 322 , an auxiliary (aux) data manager 324 , and miscellaneous information 328 .
- source memory 220 may include components in addition to, or instead of, those components discussed in conjunction with the FIG. 3 embodiment.
- source application(s) 312 may include program instructions that are preferably executed by CPU 212 ( FIG. 2 ) to perform various functions and operations for content source 114 .
- the particular nature and functionality of source application(s) 312 preferably varies depending upon factors such as the specific type and particular functionality of the corresponding content source 114 .
- video data 316 may include any appropriate information or data for display on, or for processing within, television 122 ( FIG. 1A ).
- audio data 318 may include any appropriate information or data for reproduction by television 122 ( FIG. 1A ).
- encoder 320 may include any appropriate means for converting video data 316 and audio data 318 into a compressed distribution multiplex for distribution to television 122 .
- aux data manager 324 coordinates and manages various functions for creating aux data 322 , and embedding aux data 322 as an integral part of video data 316 , in accordance with the present invention.
- Miscellaneous 328 may include any additional information for utilization by content source 114 .
- the present invention is disclosed and discussed as being implemented primarily as software. However, in alternate embodiments, some or all of the functions of the present invention may be performed by appropriate electronic hardware circuits that are configured for performing various functions that are equivalent to those functions of the software modules discussed herein. Additional details regarding the functionality of aux data manager 324 and aux data 322 are further discussed below in conjunction with FIGS. 4, 7, and 8 .
- aux data 322 may include, but is not limited to, trigger data 412 , widget content 416 , synchronization (sync) data 418 , content identification (ID) data 420 , pointer data 422 , and miscellaneous information 424 .
- aux data 322 may be implemented using various components and functionalities in addition to, or instead of, those components and functionalities discussed in conjunction with the FIG. 4 embodiment.
- trigger data 412 may include any type of information that defines various characteristics of a widget 144 ( FIG. 1B ).
- trigger data 412 may include, but is not limited to, data that defines a widget 144 with respect to widget visual appearance and behavior, information presented by a widget (such as readout values), widget graphical states (such as colors, levels, or settings), and optimal widget location, shape, size, and display times.
- widget content 416 may include any content data for display in a widget 144 .
- widget content 416 may alternately be obtained from sources or entities other than aux data 322 .
- synchronization (sync) data 418 may include any appropriate means for allowing television 122 to detect aux data 322 while it is embedded in video data 316 .
- sync data 418 may include a pre-defined identification pattern that indicates the specific location of aux data 322 within video data 316 .
- content ID data 420 may include any appropriate information for identifying the specific content of a given corresponding program.
- content ID data 420 may include an International Standard Audio-Visual Number (ISAN) number as an identifier.
- ISAN International Standard Audio-Visual Number
- pointer data 422 may include any type of required information that television 122 utilizes to locate and obtain additional information (such as widget content or trigger data) for using in producing synchronized widgets 144 .
- pointer data 422 may include, but is not limited to, a URL that identifies an Internet location where more information pertaining to the currently-displayed video data 316 may be found.
- the URL could represent a website on server 130 ( FIG. 1A ) or elsewhere providing more information about a product being advertised, a URL of a home page of an episode or series, a website where a viewer could sign up for a service or vote on a program, etc.
- miscellaneous 424 may include any additional information for utilization by television 122 .
- miscellaneous 424 may include one or more scripts or executable programs. Additional details regarding the creation, distribution, and utilization of aux data 322 are further discussed below in conjunction with FIGS. 7 and 8 .
- TV 122 may include, but is not limited to, a central processing unit (CPU) 512 , a display 138 , a TV memory 520 , and input/output interfaces (I/O interfaces) 524 .
- CPU central processing unit
- TV 122 may be implemented using components and configurations in addition to, or instead of, those components and configurations discussed in conjunction with the FIG. 5 embodiment.
- TV 122 may alternately be implemented as any other desired type of electronic device or entity.
- CPU 512 may be implemented to include any appropriate and compatible microprocessor device(s) that preferably execute software instructions to thereby control and manage the operation of TV 122 .
- the FIG. 5 display 138 may include any effective type of display technology including a cathode-ray-tube monitor or a liquid-crystal display device with an appropriate screen for displaying various information to a device user.
- TV memory 520 may be implemented to include any combination of desired storage devices, including, but not limited to, read-only memory (ROM), random-access memory (RAM), and various types of non-volatile memory, such as floppy disks or hard disks. The contents and functionality of TV memory 520 are further discussed below in conjunction with FIG. 6 .
- I/O interfaces 524 may include one or more input and/or output interfaces to receive and/or transmit any required types of information for TV 122 .
- TV 122 may utilize I/O interfaces 524 to communicate with other entities in electronic system 110 ( FIG. 1A ).
- a system user may utilize I/O interfaces 524 to communicate with TV 122 by utilizing any appropriate and effective techniques. Additional details regarding TV 122 are further discussed below in conjunction with FIGS. 6-8B .
- TV memory 520 includes, but is not limited to, one or more TV applications 612 , video data 316 , audio data 318 , a detection module 620 , and extraction module 622 , an auxiliary (aux) data module 624 , auxiliary (aux) data 322 , and miscellaneous information 628 .
- TV memory 520 may include components in addition to, or instead of, those components discussed in conjunction with the FIG. 6 embodiment.
- TV application(s) 312 may include program instructions that are preferably executed by CPU 512 ( FIG. 5 ) to perform various functions and operations for TV 122 .
- the particular nature and functionality of TV application(s) 612 preferably varies depending upon factors such as the specific type and particular functionality of the corresponding TV 122 .
- video data 316 may include any appropriate information or data for display on television 122 ( FIG. 1A ).
- audio data 318 may include any appropriate information or data for reproduction by television 122 ( FIG. 1A ).
- detection module 620 may be utilized by TV 122 to detect and locate aux data 322 that has been embedded in video data 316 , as discussed above.
- extraction module 620 may be utilized by TV 122 to remove the detected aux data 322 from the video data 316 .
- aux data module 624 coordinates and manages various functions for processing the extracted aux data 322 to effectively support synchronized widgets 144 ( FIG. 1B ) or other TV applications, in accordance with the present invention.
- Miscellaneous 628 may include any additional information for utilization by TV 122 .
- the present invention is disclosed and discussed as being implemented primarily as software. However, in alternate embodiments, some or all of the functions of the present invention may be performed by appropriate electronic hardware circuits that are configured for performing various functions that are equivalent to those functions of the software modules discussed herein. Additional details regarding the functionality of aux data module 324 and aux data 322 are further discussed below in conjunction with FIGS. 7 and 8 .
- FIGS. 7A and 7B diagrams of auxiliary (aux) data 322 embedded in video data 316 are shown, in accordance with two different embodiments of the present invention.
- FIGS. 7A and 7B present a frontal view of a display 138 from TV 122 ( FIG. 1A ).
- the embodiments of FIGS. 7A and 7B are presented for purposes of illustration, and in alternate embodiments, aux data 322 may be embedded using techniques and configurations in addition to, or instead of, certain of those techniques and configurations discussed in conjunction with the embodiments of FIGS. 7A and 7B .
- display 138 includes a main screen region that typically displays video data 316 provided by a content source 114 ( FIG. 1A ).
- the displayed video data 316 on display 138 also includes embedded aux data 322 that is preferably located in an unobtrusive area of display 138 .
- aux data 322 may be implemented in any desired shape or size, and may be displayed in any appropriate location(s) on display 138 .
- the aux data 322 in FIG. 7A is depicted as a small cross-hatched rectangle. However, any effective configuration or appearance is equally contemplated for implementing aux data 322 .
- aux data 322 may be encoded to represent any required information (see FIG. 4 ) in any effective manner.
- aux data 322 may be formatted by utilizing conventional or enhanced bar code technologies.
- aux data 322 could be effectively formatted as a video two-dimensional bar code that is embedded in a corner or at the edge of the displayed video data 316 .
- the bar code or other formatting of aux data 322 could be displayed as a part of a small graphical logo icon known as a “bug.”
- aux data 322 may encoded or displayed by utilizing any other effective techniques.
- aux data 322 could be implemented by modulating various attributes of luminance or chrominance information.
- aux data 322 may be implemented by utilizing multiple display lines.
- aux data 322 could represent a substantial amount of information, and could be quite small and dense, as aux data 322 would be read by the TV 122 processing video data 316 in video memory.
- the type of video barcode used for aux data 322 is embedded in a digital video signal, which is processed directly by the TV 122 (as pixel luminance or chrominance samples).
- quantization errors in the video compression could possibly obliterate a video barcode (so a bar code occurring within a fast-moving, hard-to-compress video sequence might not survive). However, if the bar code is left on-screen for some amount of time (a few seconds), that concern is mitigated. The resulting barcode image may not need to be shown with high contrast (black lines on white background), since TV 122 will be able to extract the information via a filtering mechanism. The bar code could thus be encoded with various shades of gray (as long as there is enough contrast for reliable extraction).
- aux data 322 could be displayed in conjunction with a graphical logo icon (“bug”), as a caption or border, or it could be placed at one more of the extreme edges of the image (because these are usually cropped before display, and are less obtrusive in any case).
- the bits of aux data 322 could be spread out spatially over the area of the video frame if the pattern of their location was known to the TV 122 beforehand. Even a small amount of aux data 322 , such as the content ID data 420 or the pointer data 422 of FIG. 4 , can be of great help in enhancing the user experience, as this information can be expanded via an interaction with a web server 130 (see FIG. 1A ) to obtain additional required information including, but not limited to, aux data 322 or content data.
- display 138 includes a main screen region that typically displays video data 316 provided by a content source 114 ( FIG. 1A ).
- the displayed video data 316 on display 138 also includes embedded aux data 322 that is preferably located in an unobtrusive area of display 138 .
- aux data 322 may be implemented in any desired shape or size, and may be displayed in any appropriate location(s) on display 138 .
- the aux data 322 in FIG. 7B is depicted as a thin cross-hatched line. However, any effective configuration or appearance is equally contemplated for implementing aux data 322 .
- aux data 322 may be encoded to represent the required information (see FIG. 4 ) in any effective manner.
- aux data 322 may be formatted as one or more horizontal lines of digital video information positioned in or near the region of the video signal's vertical blanking interval (VBI). Because digital television is often encoded with 1280 to 1920 horizontal pixels per scan line, the FIG. 7B VBI configuration for aux data 322 may provide a substantial amount of digital information to TV 122 .
- VBI vertical blanking interval
- the present invention thus supports a method of camouflaging aux data 322 as video data 316 so that a portion of active video (potentially visible to the viewer) is used to convey the aux data 322 .
- the present invention includes standardizing an encoding format for video aux data 322 to survive video compression and decompression.
- the present invention further supports embedding aux data 322 in the video image so that the aux data 322 can be recovered (detected, extracted, and processed by TV 122 ) in a standardized way, without excessive CPU overhead.
- the implementation and utilization of aux data 322 are further discussed below in conjunction with FIGS. 8A-8C .
- FIGS. 8A-8C a flowchart of method steps for distributing auxiliary data 322 embedded in video data 316 is shown, in accordance with one embodiment of the present invention.
- the FIG. 8 example is presented for purposes of illustration, and in alternate embodiments, the present invention may utilize steps and sequences other than certain of those steps and sequences discussed in conjunction with the FIG. 8 embodiment.
- a content source 114 or other appropriate entity initially produces A/V content data that typically includes video data 316 and audio data 318 .
- the content source 114 or other appropriate entity then creates auxiliary (aux) data 322 to support various advanced interactive features on a television device 122 , such as displaying one or more synchronized widgets 144 .
- the content source 114 or other appropriate entity inserts the aux data 322 into the video data 316 .
- step 818 the content source 114 or other appropriate entity compresses the audio data 318 and the video data 316 (including the embedded aux data 322 ) to create a compressed distribution multiplex.
- the FIG. 8A process then advances to step 822 of FIG. 8B through connecting letter “A.”
- a set-top box 118 or other appropriate entity receives and demultiplexes the distribution multiplex distributed by content source 114 to produce compressed audio data and video data.
- a decoder device of set-top box 118 or other appropriate entity then uncompresses the compressed audio data and video data to produce uncompressed audio data 318 and uncompressed video data 316 (including the embedded aux data 322 ).
- the set-top box 118 or other appropriate entity formats the audio data 318 and the video data 316 for delivery to a display 138 of the television 122 .
- the FIG. 8B process then advances to step 828 of FIG. 8C through connecting letter “B.”
- step 828 of FIG. 8C television 122 or other appropriate entity receives the uncompressed audio data 318 and uncompressed video data 316 (including the embedded aux data 322 ).
- a detection module 620 of the television 122 scans the video data 316 to detect the embedded aux data 322 by utilizing any effective techniques.
- an extraction module 622 of television 122 extracts the located aux data 322 from the video data 316 .
- an aux data module 624 processes the extracted aux data 322 to successfully support appropriate advanced interactive features, such as displaying one or more synchronized widgets 144 on a display 138 of television 122 .
- the FIG. 8C process may then terminate.
- aux data 322 may similarly be created and inserted into the video data 316 by any other appropriate entity at any point along the distribution path.
- aux data 322 may be inserted without completely decompressing video data 316 .
- individual macro-blocks of compressed video data 316 (without any aux data 322 ) could be replaced by corresponding compressed macro-blocks that contain the aux data 322 already embedded.
- the present invention thus provides an improved system and method for distributing auxiliary data embedded in video data.
Abstract
A system and method for distributing auxiliary data embedded in video data includes a content source that embeds the auxiliary data into the video data. The content source then encodes the video data together with the auxiliary data to create a distribution multiplex including compressed video data. A decoder receives and decompresses the distribution multiplex to reproduce the video data with the auxiliary data embedded. A television or other device then detects and extracts the auxiliary data from the video data. The television or other device processes the auxiliary data to support a variety of possible interactive applications including displaying a synchronized widget on a display of the television.
Description
- This application claims priority in, U.S. Provisional Patent Application No. 61/251,099 entitled “Video-Embedded Metadata” that was filed on Oct. 13, 2009. The foregoing related Application is commonly assigned, and is hereby incorporated by reference.
- 1. Field of the Invention
- This invention relates generally to techniques for implementing television systems, and relates more particularly to a system and method for distributing auxiliary data embedded in video data.
- 2. Description of the Background Art
- Implementing effective methods for distribution of metadata within digital television systems is a significant consideration for designers and manufacturers of contemporary electronic entertainment systems. However, effectively implementing such systems may create substantial challenges for system designers. For example, enhanced demands for increased system functionality and performance may require more capabilities and require additional hardware and software resources. Impediments to the effective delivery of metadata in advanced systems may result in a corresponding detrimental economic impact due to operational inefficiencies, lost revenue opportunities, and reduced functionality.
- Furthermore, enhanced system capability to perform various advanced operations can offer additional benefits to the end user, but may also place increased demands on the control and management of various system components. For example, an enhanced electronic system that effectively supports synchronized television widget functionality may benefit from methods providing flexible carriage of the data stream supporting this functionality.
- Due to growing demands on system resources and substantially increasing data magnitudes, it is apparent that developing new techniques for implementing and utilizing data distribution through digital television systems is a matter of concern for related electronic technologies. Therefore, for all the foregoing reasons, developing effective systems for implementing and utilizing data distribution through digital television systems remains a significant consideration for designers, manufacturers, and users of contemporary electronic entertainment systems.
- In accordance with the present invention, a system and method are disclosed for distributing auxiliary data embedded in video data. In accordance with one embodiment of the present invention, a content source or other appropriate entity initially produces content data that typically includes video data and audio data. The content source or other appropriate entity then creates auxiliary data to support advanced interactive features such as “synchronized widgets” or Internet-enhanced interactivity on a television device. The content source or other appropriate entity inserts or embeds the auxiliary data into the video data.
- The content source or other appropriate entity then encodes the audio data and the video data (including the embedded auxiliary data) to create a compressed distribution multiplex. The content source or other appropriate entity distributes the distribution multiplex in any effective manner to a television in an electronic network. In certain embodiments, a decoder device of a set-top box or other appropriate entity receives and decodes the distribution multiplex distributed by the content source to reproduce uncompressed audio data and uncompressed video data (including the embedded auxiliary data).
- A detection module of the television scans the video data to locate the embedded auxiliary data by utilizing any effective techniques. Next, an extraction module of the television extracts the located auxiliary data from the video data. Finally, an auxiliary data module of the television processes the extracted auxiliary data to successfully display and support one or more interactive applications residing in the television. For all of the foregoing reasons, the present invention thus provides an improved system and method for distributing auxiliary data embedded in video data.
-
FIG. 1A is a block diagram of an electronic system, in accordance with one embodiment of the present invention; -
FIG. 1B is a diagram of a display from the television ofFIG. 1A , in accordance with one embodiment of the present invention; -
FIG. 2 is a block diagram for one embodiment of the content source ofFIG. 1A , in accordance with the present invention; -
FIG. 3 is a block diagram for one embodiment of the source memory ofFIG. 2 , in accordance with the present invention; -
FIG. 4 is a block diagram for one embodiment of the auxiliary data ofFIG. 3 , in accordance with the present invention; -
FIG. 5 is a block diagram for one embodiment of the television fromFIG. 1A , in accordance with the present invention; -
FIG. 6 is a block diagram for one embodiment of the TV memory fromFIG. 5 , in accordance with the present invention; -
FIGS. 7A and 7B are diagrams of auxiliary data embedded in video data, in accordance with two different embodiments of the present invention; and -
FIGS. 8A-8C are a flowchart of method steps for distributing auxiliary data, in accordance with one embodiment of the present invention. - The present invention relates to an improvement in television systems. The following description is presented to enable one of ordinary skill in the art to make and use the invention, and is provided in the context of a patent application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the generic principles herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.
- The present invention is described herein as a system and method for distributing auxiliary data embedded in video data, and includes a content source that embeds the auxiliary data into the video data. The content source then encodes the video data together with the auxiliary data to create a distribution multiplex including compressed video data. A decoder receives and decompresses the distribution multiplex to reproduce the video data with the auxiliary data embedded. A television or other device then detects and extracts the auxiliary data from the video data. The television or other device processes the auxiliary data to support a variety of possible interactive applications including displaying a synchronized widget on a display of the television.
- Referring now to
FIG. 1A , a block diagram of anelectronic system 110 is shown, in accordance with one embodiment of the present invention. In theFIG. 1A embodiment,electronic system 110 may include, but is not limited to, acontent source 114, a set-top box 118, aninterface 126, atelevision 122, anoptional network 134, and anoptional server 130. In alternate embodiments,electronic system 110 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with theFIG. 1A embodiment. For example, any number oftelevisions 122 may be similarly deployed inelectronic system 110. In addition,network 134 andserver 130 may not be included in all embodiments of the present invention. - In the
FIG. 1A embodiment,content source 114 may be implemented as one or more electronic devices or other entities that prepare and distribute content data, including video data and audio data, for reproduction bytelevision 122. In theFIG. 1A embodiment,content source 114 may be implemented as any appropriate entity. For example,content source 114 may include a television broadcasting facility, a satellite television distribution facility, or an Internet server entity. Additional details regarding the implementation and utilization ofcontent source 114 are further discussed below in conjunction withFIGS. 2-4 . - In the
FIG. 1A embodiment,content source 114 creates an encoded distribution multiplex containing the content data in a compressed format, and then distributes the distribution multiplex through a distribution network viapath 116 to a decoder device. In theFIG. 1A embodiment, the decoder device is implemented in a set-top box 118. However, in other embodiments, the decoder device may be implemented as any appropriate entity, either external to, or integral with,television 122. In certain embodiments, additional devices or entities may be interposed betweencontent source 114 and set-top box 118. Examples of such entities may include, but are not limited to, a broadcast network affiliate and a service provider (such as a satellite or cable head-end). - In the
FIG. 1A embodiment, set-top box 118 decodes the encoded distribution multiplex to generate uncompressed A/V data (video data and audio data) that is provided totelevision 122 via anappropriate interface 126. In theFIG. 1A embodiment,interface 126 may be implemented in any effective manner. For example,interface 126 may be implemented according to a High Definition Multimedia Interface (HDMI) standard that provides a high-speed parallel interface to deliver uncompressed video data and audio data, and control/timing signals totelevision 122.Television 122 may then responsively receive and reproduce the video data and audio data for utilization by a system user. Additional details regarding the implementation and utilization oftelevision 122 are further discussed below in conjunction withFIGS. 5-6 . - In the
FIG. 1A embodiment,electronic system 110 supports the creation and display of various types of “widgets” ontelevision 122 in addition to the main content data. Widgets may include discrete areas that are displayed ontelevision 122 to provide any desired type of information. Additional details regarding widgets are further provided below in conjunction withFIG. 1B . In theFIG. 1A embodiment,electronic system 110 advantageously supports synchronized widgets that provide information that is related to (synchronized) to the main content data that is currently being displayed ontelevision 122. In order to successfully support synchronized widgets,electronic system 110 also provides certain types of auxiliary data totelevision 122. - In various embodiments,
television 122 may obtain auxiliary data from any appropriate source including, but not limited to,content source 114 orserver 130. In theFIG. 1A embodiment,television 122 may communicate withserver 130 via anyeffective network 134 including, but not limited to, the Internet. Additional details regarding the creation, distribution, and utilization of auxiliary data are further discussed below in conjunction withFIGS. 4, 7, and 8 . - The present invention generally involves embedding auxiliary data in a video signal so that the auxiliary data may be quickly and easily recovered by receiving devices like
television 122. In certain embodiments,content source 114 inserts auxiliary data within a distributed video signal so that the auxiliary data travels through the distribution chain, comes into a consumer's home via a compressed interface (from a cable, satellite, or IPTV service provider), is de-compressed in set-top box 118, and then travels totelevision 122 in an uncompressed format, wheretelevision 122 retrieves and utilizes the embedded auxiliary data to support synchronized widgets. The foregoing techniques are intended to circumvent service providers or other entities from blocking the consumer's access to auxiliary data that is required to provide enhanced functionality totelevision 122. - Certain cable, satellite, and IPTV entities typically provide system users with set-top boxes that are interfaced to digital televisions via HDMI uncompressed video interfaces or other appropriate means. If a content owner wishes to include auxiliary data (such as a URL, applet, etc.) with the content data, and if that auxiliary data travels with the content data as a separate digital stream (or as metadata within the compressed bit stream), the auxiliary data will be blocked at the set-
top box 118. - Conventionally, a set-
top box 114 does not pass ancillary data streams in the distribution multiplex, because the set-top box decodes only audio data and video data, and then passes only the uncompressed video data and audio data across to the television. Ancillary data streams are therefore unavailable to the television. If service providers (those offering the set-top boxes) perceive that providing access to any ancillary data is competitive to their business model, they may not be inclined to help the consumer electronics industry by providing such access. - By embedding auxiliary data within the video data, the auxiliary data survives compression/decompression and is able to arrive intact at
television 122. In other words, the present invention advantageously embeds auxiliary data within the video signal (encoded within the video image, not as a separate ancillary data stream). The present invention therefore successfully overcomes the architectural roadblock discussed above. The implementation and utilization of theFIG. 1A electronic system 110 is further discussed below in conjunction withFIGS. 1B-8B . - Referring now to
FIG. 1B , a diagram of adisplay 138 from thetelevision 122 ofFIG. 1A is shown, in accordance with one embodiment of the present invention. TheFIG. 1B embodiment is presented for purposes of illustration, and in alternate embodiments,display 138 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with theFIG. 1B embodiment. - In the
FIG. 1B embodiment,display 138 includes a main screen region that typically displays video data provided by a content source 114 (FIG. 1A ). In theFIG. 1B embodiment,display 138 also includes awidget 144 that resides in a discrete area displayed ondisplay 138 to provide any desired type of additional information. In various different embodiments,widget 144 may be implemented in any desired shape or size, and may be displayed in any appropriate location. Furthermore, any desired number of different widgets are equally contemplated, including the possibility of multiple widgets on the display at any given time. - In the
FIG. 1B embodiment,display 138 supports synchronized widgets that function to provide information that is related (synchronized) to the video data that is currently being displayed ondisplay 138. For example,widget 144 may be utilized to display financial information of specific relevance to the viewer (e.g., his/her investment portfolio) during a television program regarding economic news or investment topics. In another example,widget 144 may be utilized during a televised automobile race to display relevant information or statistics regarding specific race car drivers, racecars, or automobile racing in general. Additional details regarding the implementation and utilization ofsynchronized widgets 144 is further discussed below in conjunction withFIGS. 2-8B . - Referring now to
FIG. 2 , a block diagram for one embodiment of theFIG. 1A content source 114 is shown, in accordance with the present invention. In theFIG. 2 embodiment,content source 114 may include, but is not limited to, a central processing unit (CPU) 212, asource memory 220, and input/output interfaces (I/O interfaces) 224. In alternate embodiments,content source 114 may be implemented using components and configurations in addition to, or instead of, those components and configurations discussed in conjunction with theFIG. 2 embodiment. In addition,content source 114 may alternately be implemented as any other desired type of electronic device or entity. - In the
FIG. 2 embodiment,CPU 212 may be implemented to include any appropriate and compatible microprocessor device(s) that preferably execute software instructions to thereby control and manage the operation ofcontent source 114. In theFIG. 2 embodiment,source memory 220 may be implemented to include any combination of desired storage devices, including, but not limited to, read-only memory (ROM), random-access memory (RAM), and various types of non-volatile memory, such as floppy disks or hard disks. The contents and functionality ofsource memory 220 are further discussed below in conjunction withFIGS. 3 and 4 . - In the
FIG. 2 embodiment, I/O interfaces 224 may include one or more input and/or output interfaces to receive and/or transmit any required types of information forcontent source 114. For example, in theFIG. 2 embodiment,content source 114 may utilize I/O interfaces 224 to communicate with other entities in electronic system 110 (FIG. 1A ). Furthermore, a system user may utilize I/O interfaces 224 to communicate withcontent source 114 by utilizing any appropriate and effective techniques. Additional details regardingcontent source 114 are further discussed below in conjunction withFIGS. 3-4 . - Referring now to
FIG. 3 , a block diagram for one embodiment of theFIG. 2 source memory 220 is shown, in accordance with the present invention. In theFIG. 3 embodiment,source memory 220 includes, but is not limited to, one ormore source applications 312,video data 316,audio data 318, anencoder 320, auxiliary (aux)data 322, an auxiliary (aux)data manager 324, andmiscellaneous information 328. In alternate embodiments,source memory 220 may include components in addition to, or instead of, those components discussed in conjunction with theFIG. 3 embodiment. - In the
FIG. 3 embodiment, source application(s) 312 may include program instructions that are preferably executed by CPU 212 (FIG. 2 ) to perform various functions and operations forcontent source 114. The particular nature and functionality of source application(s) 312 preferably varies depending upon factors such as the specific type and particular functionality of the correspondingcontent source 114. In theFIG. 3 embodiment,video data 316 may include any appropriate information or data for display on, or for processing within, television 122 (FIG. 1A ). Similarly,audio data 318 may include any appropriate information or data for reproduction by television 122 (FIG. 1A ). - In the
FIG. 3 embodiment,encoder 320 may include any appropriate means for convertingvideo data 316 andaudio data 318 into a compressed distribution multiplex for distribution totelevision 122. In theFIG. 3 embodiment, auxdata manager 324 coordinates and manages various functions for creatingaux data 322, and embeddingaux data 322 as an integral part ofvideo data 316, in accordance with the present invention. Miscellaneous 328 may include any additional information for utilization bycontent source 114. - In the
FIG. 3 embodiment, the present invention is disclosed and discussed as being implemented primarily as software. However, in alternate embodiments, some or all of the functions of the present invention may be performed by appropriate electronic hardware circuits that are configured for performing various functions that are equivalent to those functions of the software modules discussed herein. Additional details regarding the functionality ofaux data manager 324 andaux data 322 are further discussed below in conjunction withFIGS. 4, 7, and 8 . - Referring now to
FIG. 4 , a block diagram of theFIG. 3 aux data 322 is shown, in accordance with one embodiment of the present invention. In theFIG. 4 embodiment,aux data 322 may include, but is not limited to, triggerdata 412,widget content 416, synchronization (sync)data 418, content identification (ID)data 420,pointer data 422, andmiscellaneous information 424. In alternate embodiments,aux data 322 may be implemented using various components and functionalities in addition to, or instead of, those components and functionalities discussed in conjunction with theFIG. 4 embodiment. - In the
FIG. 4 embodiment, triggerdata 412 may include any type of information that defines various characteristics of a widget 144 (FIG. 1B ). For example, triggerdata 412 may include, but is not limited to, data that defines awidget 144 with respect to widget visual appearance and behavior, information presented by a widget (such as readout values), widget graphical states (such as colors, levels, or settings), and optimal widget location, shape, size, and display times. - In the
FIG. 4 embodiment,widget content 416 may include any content data for display in awidget 144. In certain embodiments,widget content 416 may alternately be obtained from sources or entities other thanaux data 322. In theFIG. 4 embodiment, synchronization (sync)data 418 may include any appropriate means for allowingtelevision 122 to detectaux data 322 while it is embedded invideo data 316. For example, in certain embodiments,sync data 418 may include a pre-defined identification pattern that indicates the specific location ofaux data 322 withinvideo data 316. - In the
FIG. 4 embodiment,content ID data 420 may include any appropriate information for identifying the specific content of a given corresponding program. For example, in certain embodiments,content ID data 420 may include an International Standard Audio-Visual Number (ISAN) number as an identifier. In theFIG. 4 embodiment,pointer data 422 may include any type of required information thattelevision 122 utilizes to locate and obtain additional information (such as widget content or trigger data) for using in producingsynchronized widgets 144. - For example,
pointer data 422 may include, but is not limited to, a URL that identifies an Internet location where more information pertaining to the currently-displayedvideo data 316 may be found. The URL could represent a website on server 130 (FIG. 1A ) or elsewhere providing more information about a product being advertised, a URL of a home page of an episode or series, a website where a viewer could sign up for a service or vote on a program, etc. In theFIG. 4 embodiment, miscellaneous 424 may include any additional information for utilization bytelevision 122. For example, in certain embodiments, miscellaneous 424 may include one or more scripts or executable programs. Additional details regarding the creation, distribution, and utilization ofaux data 322 are further discussed below in conjunction withFIGS. 7 and 8 . - Referring now to
FIG. 5 , a block diagram for one embodiment of theFIG. 1A television (TV) 122 is shown, in accordance with the present invention. In theFIG. 2 embodiment,TV 122 may include, but is not limited to, a central processing unit (CPU) 512, adisplay 138, aTV memory 520, and input/output interfaces (I/O interfaces) 524. In alternate embodiments,TV 122 may be implemented using components and configurations in addition to, or instead of, those components and configurations discussed in conjunction with theFIG. 5 embodiment. In addition,TV 122 may alternately be implemented as any other desired type of electronic device or entity. - In the
FIG. 5 embodiment,CPU 512 may be implemented to include any appropriate and compatible microprocessor device(s) that preferably execute software instructions to thereby control and manage the operation ofTV 122. TheFIG. 5 display 138 may include any effective type of display technology including a cathode-ray-tube monitor or a liquid-crystal display device with an appropriate screen for displaying various information to a device user. In theFIG. 5 embodiment,TV memory 520 may be implemented to include any combination of desired storage devices, including, but not limited to, read-only memory (ROM), random-access memory (RAM), and various types of non-volatile memory, such as floppy disks or hard disks. The contents and functionality ofTV memory 520 are further discussed below in conjunction withFIG. 6 . - In the
FIG. 5 embodiment, I/O interfaces 524 may include one or more input and/or output interfaces to receive and/or transmit any required types of information forTV 122. For example, in theFIG. 5 embodiment,TV 122 may utilize I/O interfaces 524 to communicate with other entities in electronic system 110 (FIG. 1A ). Furthermore, a system user may utilize I/O interfaces 524 to communicate withTV 122 by utilizing any appropriate and effective techniques. Additionaldetails regarding TV 122 are further discussed below in conjunction withFIGS. 6-8B . - Referring now to
FIG. 6 , a block diagram for one embodiment of theFIG. 5 TV memory 520 is shown, in accordance with the present invention. In theFIG. 6 embodiment,TV memory 520 includes, but is not limited to, one ormore TV applications 612,video data 316,audio data 318, adetection module 620, andextraction module 622, an auxiliary (aux)data module 624, auxiliary (aux)data 322, andmiscellaneous information 628. In alternate embodiments,TV memory 520 may include components in addition to, or instead of, those components discussed in conjunction with theFIG. 6 embodiment. - In the
FIG. 6 embodiment, TV application(s) 312 may include program instructions that are preferably executed by CPU 512 (FIG. 5 ) to perform various functions and operations forTV 122. The particular nature and functionality of TV application(s) 612 preferably varies depending upon factors such as the specific type and particular functionality of thecorresponding TV 122. In theFIG. 6 embodiment,video data 316 may include any appropriate information or data for display on television 122 (FIG. 1A ). Similarly,audio data 318 may include any appropriate information or data for reproduction by television 122 (FIG. 1A ). - In the
FIG. 6 embodiment,detection module 620 may be utilized byTV 122 to detect and locateaux data 322 that has been embedded invideo data 316, as discussed above. In theFIG. 6 embodiment,extraction module 620 may be utilized byTV 122 to remove the detectedaux data 322 from thevideo data 316. In theFIG. 3 embodiment, auxdata module 624 coordinates and manages various functions for processing the extractedaux data 322 to effectively support synchronized widgets 144 (FIG. 1B ) or other TV applications, in accordance with the present invention. Miscellaneous 628 may include any additional information for utilization byTV 122. - In the
FIG. 6 embodiment, the present invention is disclosed and discussed as being implemented primarily as software. However, in alternate embodiments, some or all of the functions of the present invention may be performed by appropriate electronic hardware circuits that are configured for performing various functions that are equivalent to those functions of the software modules discussed herein. Additional details regarding the functionality ofaux data module 324 andaux data 322 are further discussed below in conjunction withFIGS. 7 and 8 . - Referring now to
FIGS. 7A and 7B , diagrams of auxiliary (aux)data 322 embedded invideo data 316 are shown, in accordance with two different embodiments of the present invention.FIGS. 7A and 7B present a frontal view of adisplay 138 from TV 122 (FIG. 1A ). The embodiments ofFIGS. 7A and 7B are presented for purposes of illustration, and in alternate embodiments,aux data 322 may be embedded using techniques and configurations in addition to, or instead of, certain of those techniques and configurations discussed in conjunction with the embodiments ofFIGS. 7A and 7B . - In the
FIG. 7A embodiment,display 138 includes a main screen region that typically displaysvideo data 316 provided by a content source 114 (FIG. 1A ). In theFIG. 7A embodiment, the displayedvideo data 316 ondisplay 138 also includes embeddedaux data 322 that is preferably located in an unobtrusive area ofdisplay 138. In various, different embodiments,aux data 322 may be implemented in any desired shape or size, and may be displayed in any appropriate location(s) ondisplay 138. For purposes of illustration, theaux data 322 inFIG. 7A is depicted as a small cross-hatched rectangle. However, any effective configuration or appearance is equally contemplated for implementingaux data 322. - In the
FIG. 7A embodiment,aux data 322 may be encoded to represent any required information (seeFIG. 4 ) in any effective manner. For example, in certain embodiments,aux data 322 may be formatted by utilizing conventional or enhanced bar code technologies. In other words,aux data 322 could be effectively formatted as a video two-dimensional bar code that is embedded in a corner or at the edge of the displayedvideo data 316. In addition, the bar code or other formatting ofaux data 322 could be displayed as a part of a small graphical logo icon known as a “bug.” Furthermore, in various other embodiments,aux data 322 may encoded or displayed by utilizing any other effective techniques. For example,aux data 322 could be implemented by modulating various attributes of luminance or chrominance information. In addition,aux data 322 may be implemented by utilizing multiple display lines. - Such an encoding of
aux data 322 could represent a substantial amount of information, and could be quite small and dense, asaux data 322 would be read by theTV 122processing video data 316 in video memory. Where printed barcodes are optimized for readout by laser scanners, the type of video barcode used foraux data 322 is embedded in a digital video signal, which is processed directly by the TV 122 (as pixel luminance or chrominance samples). - In certain embodiments, quantization errors in the video compression could possibly obliterate a video barcode (so a bar code occurring within a fast-moving, hard-to-compress video sequence might not survive). However, if the bar code is left on-screen for some amount of time (a few seconds), that concern is mitigated. The resulting barcode image may not need to be shown with high contrast (black lines on white background), since
TV 122 will be able to extract the information via a filtering mechanism. The bar code could thus be encoded with various shades of gray (as long as there is enough contrast for reliable extraction). - As discussed above,
aux data 322 could be displayed in conjunction with a graphical logo icon (“bug”), as a caption or border, or it could be placed at one more of the extreme edges of the image (because these are usually cropped before display, and are less obtrusive in any case). The bits ofaux data 322 could be spread out spatially over the area of the video frame if the pattern of their location was known to theTV 122 beforehand. Even a small amount ofaux data 322, such as thecontent ID data 420 or thepointer data 422 ofFIG. 4 , can be of great help in enhancing the user experience, as this information can be expanded via an interaction with a web server 130 (seeFIG. 1A ) to obtain additional required information including, but not limited to,aux data 322 or content data. - In the
FIG. 7B embodiment,display 138 includes a main screen region that typically displaysvideo data 316 provided by a content source 114 (FIG. 1A ). In theFIG. 7B embodiment, the displayedvideo data 316 ondisplay 138 also includes embeddedaux data 322 that is preferably located in an unobtrusive area ofdisplay 138. In various different embodiments,aux data 322 may be implemented in any desired shape or size, and may be displayed in any appropriate location(s) ondisplay 138. For purposes of illustration, theaux data 322 inFIG. 7B is depicted as a thin cross-hatched line. However, any effective configuration or appearance is equally contemplated for implementingaux data 322. - In the
FIG. 7B embodiment,aux data 322 may be encoded to represent the required information (seeFIG. 4 ) in any effective manner. For example, in certain embodiments,aux data 322 may be formatted as one or more horizontal lines of digital video information positioned in or near the region of the video signal's vertical blanking interval (VBI). Because digital television is often encoded with 1280 to 1920 horizontal pixels per scan line, theFIG. 7B VBI configuration foraux data 322 may provide a substantial amount of digital information toTV 122. - The present invention thus supports a method of camouflaging
aux data 322 asvideo data 316 so that a portion of active video (potentially visible to the viewer) is used to convey theaux data 322. In addition, the present invention includes standardizing an encoding format for video auxdata 322 to survive video compression and decompression. The present invention further supports embeddingaux data 322 in the video image so that theaux data 322 can be recovered (detected, extracted, and processed by TV 122) in a standardized way, without excessive CPU overhead. The implementation and utilization ofaux data 322 are further discussed below in conjunction withFIGS. 8A-8C . - Referring now to
FIGS. 8A-8C , a flowchart of method steps for distributingauxiliary data 322 embedded invideo data 316 is shown, in accordance with one embodiment of the present invention. TheFIG. 8 example is presented for purposes of illustration, and in alternate embodiments, the present invention may utilize steps and sequences other than certain of those steps and sequences discussed in conjunction with theFIG. 8 embodiment. - In the
FIG. 8A embodiment, instep 812, acontent source 114 or other appropriate entity initially produces A/V content data that typically includesvideo data 316 andaudio data 318. Instep 814, thecontent source 114 or other appropriate entity then creates auxiliary (aux)data 322 to support various advanced interactive features on atelevision device 122, such as displaying one or moresynchronized widgets 144. Instep 816, thecontent source 114 or other appropriate entity inserts theaux data 322 into thevideo data 316. - In
step 818, thecontent source 114 or other appropriate entity compresses theaudio data 318 and the video data 316 (including the embedded aux data 322) to create a compressed distribution multiplex. TheFIG. 8A process then advances to step 822 ofFIG. 8B through connecting letter “A.” - In
step 822 ofFIG. 8B , a set-top box 118 or other appropriate entity receives and demultiplexes the distribution multiplex distributed bycontent source 114 to produce compressed audio data and video data. Instep 824, a decoder device of set-top box 118 or other appropriate entity then uncompresses the compressed audio data and video data to produceuncompressed audio data 318 and uncompressed video data 316 (including the embedded aux data 322). Instep 826, the set-top box 118 or other appropriate entity formats theaudio data 318 and thevideo data 316 for delivery to adisplay 138 of thetelevision 122. TheFIG. 8B process then advances to step 828 ofFIG. 8C through connecting letter “B.” - In
step 828 ofFIG. 8C ,television 122 or other appropriate entity receives theuncompressed audio data 318 and uncompressed video data 316 (including the embedded aux data 322). Instep 830, adetection module 620 of thetelevision 122 scans thevideo data 316 to detect the embeddedaux data 322 by utilizing any effective techniques. Instep 832, anextraction module 622 oftelevision 122 extracts the locatedaux data 322 from thevideo data 316. Finally, instep 834, anaux data module 624 processes the extractedaux data 322 to successfully support appropriate advanced interactive features, such as displaying one or moresynchronized widgets 144 on adisplay 138 oftelevision 122. TheFIG. 8C process may then terminate. - In certain alternate embodiments,
aux data 322 may similarly be created and inserted into thevideo data 316 by any other appropriate entity at any point along the distribution path. In certain of these alternate embodiments,aux data 322 may be inserted without completely decompressingvideo data 316. For example, individual macro-blocks of compressed video data 316 (without any aux data 322) could be replaced by corresponding compressed macro-blocks that contain theaux data 322 already embedded. For all of the foregoing reasons, the present invention thus provides an improved system and method for distributing auxiliary data embedded in video data. - The invention has been explained above with reference to certain embodiments. Other embodiments will be apparent to those skilled in the art in light of this disclosure. For example, the present invention may readily be implemented using configurations and techniques other than those described in the embodiments above. Additionally, the present invention may effectively be used in conjunction with systems other than those described above. Therefore, these and other variations upon the discussed embodiments are intended to be covered by the present invention, which is limited only by the appended claims.
Claims (21)
1. (canceled)
2. A method for receiving video data, the method comprising:
decoding a distribution multiplex including the video data embedded with auxiliary data,
the auxiliary data including content identification data that identifies specific content; and
displaying the video data and the embedded auxiliary data, wherein the auxiliary data is displayed by modulating pixel luminance or chrominance.
3. The method according to claim 2 , wherein the method further comprises:
detecting and extracting the auxiliary data from the distribution multiplex; and
executing an application related to the video data based on the detected and extracted auxiliary data.
4. The method according to claim 3 , wherein the detecting and extracting the auxiliary data includes detecting a predefined pattern in the decoded distribution multiplex, the predefined pattern corresponding to a location of the auxiliary data embedded in the video data.
5. The method according to claim 2 , wherein the displayed auxiliary data has rectangular shape.
6. The method according to claim 2 , wherein same auxiliary data is continued to be displayed during display of plural frames of the video data.
7. A receiving device comprising:
circuitry configured to
decode a distribution multiplex including the video data embedded with auxiliary data, the auxiliary data including content identification data that identifies specific content; and
display the video data and the embedded auxiliary data, wherein the auxiliary data is displayed by modulating pixel luminance or chrominance.
8. The receiving device according to claim 7 , wherein the circuitry is further configured to:
detect and extract the auxiliary data from the distribution multiplex; and
execute an application related to the video data based on the detected and extracted auxiliary data.
9. The receiving device according to claim 8 , wherein the detecting and extracting the auxiliary data includes detecting a predefined pattern in the decoded distribution multiplex, the predefined pattern corresponding to a location of the auxiliary data embedded in the video data.
10. The receiving device according to claim 7 , wherein the displayed auxiliary data has rectangular shape.
11. The receiving device according to claim 7 , wherein same auxiliary data is continued to be displayed during display of plural frames of the video data.
12. A method for distributing video data, the method comprising:
embedding auxiliary data into video data;
encoding the video data embedded with the auxiliary data to create a distribution multiplex, the auxiliary data includes content identification data that identifies specific content and is encoded with pixel luminance or chrominance;
distributing the distribution multiplex to at least one receiving device through a distribution network.
13. The method according to claim 12 , wherein the auxiliary data is used to execute an application related to the video data by the at least one receiving device.
14. The method according to claim 12 , wherein the auxiliary data includes a predefined pattern, the predefined pattern corresponding to a location of the auxiliary data embedded in the video data.
15. The method according to claim 12 , wherein the auxiliary data is embedded in the video data such that the auxiliary data displayed by the at least one receiving device has rectangular shape.
16. The method according to claim 12 , wherein the same auxiliary data is embedded into plural frames of the video data.
17. A distribution device comprising:
circuitry configured to
embed auxiliary data into video data;
encode the video data embedded with the auxiliary data to create a distribution multiplex, the auxiliary data includes content identification data that identifies specific content and is encoded with pixel luminance or chrominance;
distribute the distribution multiplex to at least one receiving device through a distribution network.
18. The distribution device according to claim 17 , wherein the auxiliary data is used to execute an application related to the video data by the at least one receiving device.
19. The distribution device according to claim 17 , wherein the auxiliary data includes a predefined pattern, the predefined pattern corresponding to a location of the auxiliary data embedded in the video data.
20. The distribution device according to claim 17 , wherein the auxiliary data is embedded in the video data such that the auxiliary data displayed by the at least one receiving device has rectangular shape.
21. The distribution device according to claim 17 , wherein the same auxiliary data is embedded into plural frames of the video data.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/001,514 US20160134909A1 (en) | 2009-10-13 | 2016-01-20 | System and method for distributing auxiliary data embedded in video data |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US25109909P | 2009-10-13 | 2009-10-13 | |
US12/798,118 US9277183B2 (en) | 2009-10-13 | 2010-03-30 | System and method for distributing auxiliary data embedded in video data |
US15/001,514 US20160134909A1 (en) | 2009-10-13 | 2016-01-20 | System and method for distributing auxiliary data embedded in video data |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/798,118 Continuation US9277183B2 (en) | 2009-10-13 | 2010-03-30 | System and method for distributing auxiliary data embedded in video data |
Publications (1)
Publication Number | Publication Date |
---|---|
US20160134909A1 true US20160134909A1 (en) | 2016-05-12 |
Family
ID=43855868
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/798,118 Active 2030-12-18 US9277183B2 (en) | 2009-10-13 | 2010-03-30 | System and method for distributing auxiliary data embedded in video data |
US15/001,514 Abandoned US20160134909A1 (en) | 2009-10-13 | 2016-01-20 | System and method for distributing auxiliary data embedded in video data |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/798,118 Active 2030-12-18 US9277183B2 (en) | 2009-10-13 | 2010-03-30 | System and method for distributing auxiliary data embedded in video data |
Country Status (7)
Country | Link |
---|---|
US (2) | US9277183B2 (en) |
EP (1) | EP2489181A4 (en) |
KR (1) | KR101367459B1 (en) |
CN (2) | CN105263031B (en) |
CA (1) | CA2775263C (en) |
MX (1) | MX2012004085A (en) |
WO (1) | WO2011046590A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180152752A1 (en) * | 2016-11-30 | 2018-05-31 | Samsung Electronics Co., Ltd. | Electronic apparatus, controlling method thereof and display system comprising electronic apparatus and a plurality of display apparatuses |
Families Citing this family (101)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9961388B2 (en) | 2008-11-26 | 2018-05-01 | David Harrison | Exposure of public internet protocol addresses in an advertising exchange server to improve relevancy of advertisements |
US9986279B2 (en) | 2008-11-26 | 2018-05-29 | Free Stream Media Corp. | Discovery, access control, and communication with networked services |
US10567823B2 (en) | 2008-11-26 | 2020-02-18 | Free Stream Media Corp. | Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device |
US10977693B2 (en) | 2008-11-26 | 2021-04-13 | Free Stream Media Corp. | Association of content identifier of audio-visual data with additional data through capture infrastructure |
US8180891B1 (en) | 2008-11-26 | 2012-05-15 | Free Stream Media Corp. | Discovery, access control, and communication with networked services from within a security sandbox |
US9026668B2 (en) | 2012-05-26 | 2015-05-05 | Free Stream Media Corp. | Real-time and retargeted advertising on multiple screens of a user watching television |
US10419541B2 (en) | 2008-11-26 | 2019-09-17 | Free Stream Media Corp. | Remotely control devices over a network without authentication or registration |
US9154942B2 (en) | 2008-11-26 | 2015-10-06 | Free Stream Media Corp. | Zero configuration communication between a browser and a networked media device |
US9519772B2 (en) | 2008-11-26 | 2016-12-13 | Free Stream Media Corp. | Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device |
US10334324B2 (en) | 2008-11-26 | 2019-06-25 | Free Stream Media Corp. | Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device |
US10631068B2 (en) | 2008-11-26 | 2020-04-21 | Free Stream Media Corp. | Content exposure attribution based on renderings of related content across multiple devices |
US10880340B2 (en) | 2008-11-26 | 2020-12-29 | Free Stream Media Corp. | Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device |
US9386356B2 (en) | 2008-11-26 | 2016-07-05 | Free Stream Media Corp. | Targeting with television audience data across multiple screens |
US8705933B2 (en) | 2009-09-25 | 2014-04-22 | Sony Corporation | Video bookmarking |
US9066154B2 (en) * | 2009-11-13 | 2015-06-23 | Triveni Digital, Inc. | System and method for enhanced television and delivery of enhanced television content |
US8730301B2 (en) | 2010-03-12 | 2014-05-20 | Sony Corporation | Service linkage to caption disparity data transport |
US8941779B2 (en) | 2012-03-21 | 2015-01-27 | Sony Corporation | Non-closed caption data transport in standard caption service |
US8760498B2 (en) | 2011-03-14 | 2014-06-24 | Sony Corporation | Disparity data transport in standard caption service |
US8839338B2 (en) | 2012-03-28 | 2014-09-16 | Sony Corporation | Service usage reporting data transport |
US20110247028A1 (en) | 2010-04-01 | 2011-10-06 | Mark Eyer | Receiver and System Using an Electronic Questionnaire for Advanced Broadcast Services |
US9723360B2 (en) | 2010-04-01 | 2017-08-01 | Saturn Licensing Llc | Interests and demographics profile for advanced broadcast services |
US8863171B2 (en) | 2010-06-14 | 2014-10-14 | Sony Corporation | Announcement of program synchronized triggered declarative objects |
US8898723B2 (en) | 2010-08-20 | 2014-11-25 | Sony Corporation | Virtual channel declarative script binding |
US8893210B2 (en) | 2010-08-20 | 2014-11-18 | Sony Corporation | Server load balancing for interactive television |
US8909694B2 (en) | 2010-08-27 | 2014-12-09 | Sony Corporation | Transmission apparatus and method, reception apparatus and method, and transmission and reception system |
US8875169B2 (en) | 2010-08-27 | 2014-10-28 | Sony Corporation | Transmission and reception apparatus, methods, and systems for filtering content |
US8892636B2 (en) | 2010-08-30 | 2014-11-18 | Sony Corporation | Transmission apparatus and method, reception apparatus and method, and transmission and reception system |
US8595783B2 (en) | 2010-08-30 | 2013-11-26 | Sony Corporation | Receiving device, receiving method, program, and broadcasting system |
US9179188B2 (en) | 2010-08-30 | 2015-11-03 | Sony Corporation | Transmission apparatus and method, reception apparatus and method, and transmission and reception system |
US8918801B2 (en) | 2010-08-30 | 2014-12-23 | Sony Corporation | Transmission apparatus, transmission method, reception apparatus, reception method, program, and broadcasting system |
US8872888B2 (en) | 2010-10-01 | 2014-10-28 | Sony Corporation | Content transmission apparatus, content transmission method, content reproduction apparatus, content reproduction method, program and content delivery system |
US9179198B2 (en) | 2010-10-01 | 2015-11-03 | Sony Corporation | Receiving apparatus, receiving method, and program |
US8908103B2 (en) | 2010-10-01 | 2014-12-09 | Sony Corporation | Content supplying apparatus, content supplying method, content reproduction apparatus, content reproduction method, program and content viewing system |
CA2809311C (en) | 2010-10-01 | 2019-01-08 | Sony Corporation | Information processing device, information processing method, and program |
US9078031B2 (en) | 2010-10-01 | 2015-07-07 | Sony Corporation | Reception apparatus, reception method, and program |
KR20120035986A (en) * | 2010-10-07 | 2012-04-17 | 삼성전자주식회사 | Method and apparatus for displaying using image code |
JP5783402B2 (en) | 2011-01-25 | 2015-09-24 | ソニー株式会社 | Reception device, reception method, supply device, supply method, program, and broadcasting system |
JP5668512B2 (en) | 2011-02-15 | 2015-02-12 | ソニー株式会社 | Information processing apparatus and information processing method |
TWI574565B (en) | 2011-03-31 | 2017-03-11 | Sony Corp | A transmitting apparatus, a receiving method, a receiving apparatus, a receiving method, a computer-readable medium, and a broadcasting system |
TWI545955B (en) | 2011-04-28 | 2016-08-11 | Sony Corp | Signal receiving apparatus and method, a signal transmission apparatus and method, and program |
US20120304224A1 (en) * | 2011-05-25 | 2012-11-29 | Steven Keith Hines | Mechanism for Embedding Metadata in Video and Broadcast Television |
US9554175B2 (en) | 2011-07-20 | 2017-01-24 | Sony Corporation | Method, computer program, reception apparatus, and information providing apparatus for trigger compaction |
US8917358B2 (en) | 2011-07-27 | 2014-12-23 | Sony Corporation | Reception apparatus, terminal apparatus, control method, program, and communication system |
US10491966B2 (en) | 2011-08-04 | 2019-11-26 | Saturn Licensing Llc | Reception apparatus, method, computer program, and information providing apparatus for providing an alert service |
WO2013021824A1 (en) | 2011-08-05 | 2013-02-14 | ソニー株式会社 | Reception device, reception method, program, and information processing system |
CN103748895B (en) | 2011-08-24 | 2018-11-13 | 索尼公司 | Reception device, method of reseptance and information processing system |
US10097869B2 (en) * | 2011-08-29 | 2018-10-09 | Tata Consultancy Services Limited | Method and system for embedding metadata in multiplexed analog videos broadcasted through digital broadcasting medium |
TWI528749B (en) | 2011-09-06 | 2016-04-01 | Sony Corp | A signal receiving device, a signal receiving method, an information processing program and an information processing system |
US8730328B2 (en) | 2011-10-06 | 2014-05-20 | Qualcomm Incorporated | Frame buffer format detection |
US9374620B2 (en) | 2011-10-21 | 2016-06-21 | Sony Corporation | Terminal apparatus, server apparatus, information processing method, program, and interlocked application feed system |
US9015785B2 (en) | 2011-11-29 | 2015-04-21 | Sony Corporation | Terminal apparatus, server apparatus, information processing method, program, and linking application supply system |
US9113230B2 (en) | 2011-12-21 | 2015-08-18 | Sony Corporation | Method, computer program, and reception apparatus for delivery of supplemental content |
US8930988B2 (en) | 2011-12-21 | 2015-01-06 | Sony Corporation | Reception apparatus, reception method, program, and information processing system |
US9131281B2 (en) | 2011-12-29 | 2015-09-08 | Tata Consultancy Services Limited | Method for embedding and multiplexing audio metadata in a broadcasted analog video stream |
JP6082946B2 (en) | 2012-01-24 | 2017-02-22 | サターン ライセンシング エルエルシーSaturn Licensing LLC | Receiving apparatus, receiving method, program, and information processing system |
CA2859115C (en) | 2012-02-07 | 2020-01-21 | Sony Corporation | Receiving apparatus, receiving method, and program |
KR20140125766A (en) * | 2012-02-10 | 2014-10-29 | 엘지전자 주식회사 | Image display apparatus and method for operating same |
US9936231B2 (en) | 2012-03-21 | 2018-04-03 | Saturn Licensing Llc | Trigger compaction |
WO2013191976A1 (en) | 2012-06-19 | 2013-12-27 | Sony Corporation | Extensions to trigger parameters table for interactive television |
US9456245B2 (en) | 2012-07-05 | 2016-09-27 | Sony Corporation | Receiving device, receiving method, transmitting device, and transmitting method for controlling applications |
WO2014011355A1 (en) | 2012-07-12 | 2014-01-16 | Sony Corporation | Parameterized services descriptor for advanced television services |
GB2504270A (en) | 2012-07-23 | 2014-01-29 | Wirewax Ltd | Synchronising hyperlink and associated object positions using frame numbers encoded as pixel data |
US9432744B2 (en) | 2012-07-31 | 2016-08-30 | Sony Corporation | Receiving device, receiving method, transmitting device, and transmitting method |
US9154840B2 (en) | 2012-07-31 | 2015-10-06 | Sony Corporation | Reception apparatus, reception method, transmission apparatus, and transmission method |
US9883247B2 (en) | 2012-08-13 | 2018-01-30 | Saturn Licensing Llc | Reception apparatus, reception method, transmission apparatus, and transmission method |
US9264648B2 (en) | 2012-10-09 | 2016-02-16 | Sony Corporation | Receiving device, receiving method, transmitting device, and transmitting method |
KR20150073987A (en) * | 2012-10-18 | 2015-07-01 | 엘지전자 주식회사 | Apparatus and method for processing an interactive service |
US10002206B2 (en) | 2012-10-26 | 2018-06-19 | Saturn Licensing Llc | Information processing device and information processing method |
US9253518B2 (en) | 2012-11-09 | 2016-02-02 | Sony Corporation | On-demand access to scheduled content |
US9769503B2 (en) | 2012-11-14 | 2017-09-19 | Saturn Licensing Llc | Information processor, information processing method and program |
US11558672B1 (en) * | 2012-11-19 | 2023-01-17 | Cox Communications, Inc. | System for providing new content related to content currently being accessed |
US9936256B2 (en) | 2012-11-28 | 2018-04-03 | Saturn Licensing Llc | Receiver, reception method, transmitter and transmission method |
US9148676B2 (en) | 2012-12-04 | 2015-09-29 | Sony Corporation | Broadcast transition channel |
US8914832B2 (en) | 2012-12-14 | 2014-12-16 | Sony Corporation | Information processing apparatus, information processing method, and program |
US10257564B2 (en) | 2013-01-24 | 2019-04-09 | Saturn Licensing Llc | Distributed non-real-time content |
US9942601B2 (en) | 2013-01-24 | 2018-04-10 | Saturn Licensing Llc | Storing non-real time content |
US20140337905A1 (en) * | 2013-05-09 | 2014-11-13 | Telefonaktiebolaget L M Ericsson (Publ) | System and method for delivering extended media content |
US9300896B2 (en) | 2013-06-03 | 2016-03-29 | Sony Corporation | Method, computer-readable storage medium, reception apparatus, and information providing apparatus for identifying available channels and/or services |
US9980014B2 (en) | 2013-06-28 | 2018-05-22 | Saturn Licensing Llc | Methods, information providing system, and reception apparatus for protecting content |
US9137566B2 (en) | 2013-06-28 | 2015-09-15 | Sony Corporation | Distribution system, reception apparatus, and methods for transitioning to new services |
US9038095B2 (en) | 2013-07-03 | 2015-05-19 | Sony Corporation | Methods, information providing system, and reception apparatus for distribution of at least one content version |
US8884800B1 (en) | 2013-07-31 | 2014-11-11 | Sony Corporation | Method and apparatus for performing analog-to-digital conversion on multiple input signals |
CN104581341B (en) * | 2013-10-24 | 2018-05-29 | 华为终端有限公司 | A kind of caption presentation method and subtitle display |
US9596521B2 (en) | 2014-03-13 | 2017-03-14 | Verance Corporation | Interactive content acquisition using embedded codes |
US10504200B2 (en) | 2014-03-13 | 2019-12-10 | Verance Corporation | Metadata acquisition using embedded watermarks |
WO2016028934A1 (en) | 2014-08-20 | 2016-02-25 | Verance Corporation | Content management based on dither-like watermark embedding |
US9769543B2 (en) | 2014-11-25 | 2017-09-19 | Verance Corporation | Enhanced metadata and content delivery using watermarks |
US9942602B2 (en) | 2014-11-25 | 2018-04-10 | Verance Corporation | Watermark detection and metadata delivery associated with a primary content |
US9326295B1 (en) | 2014-12-10 | 2016-04-26 | Sony Corporation | Method and apparatus for transmitting a-priori information in a communication system |
WO2016100916A1 (en) | 2014-12-18 | 2016-06-23 | Verance Corporation | Service signaling recovery for multimedia content using embedded watermarks |
US20160204804A1 (en) | 2015-01-13 | 2016-07-14 | Sony Corporation | Data processing apparatus and method |
US9762354B2 (en) | 2015-01-27 | 2017-09-12 | Sony Corporation | Signaling methods and apparatus |
US9912986B2 (en) | 2015-03-19 | 2018-03-06 | Sony Corporation | System for distributing metadata embedded in video |
US9756401B2 (en) | 2015-04-07 | 2017-09-05 | Sony Corporation | Processing and providing an image in which a plurality of symbols are encoded |
US20180131995A1 (en) * | 2015-05-08 | 2018-05-10 | Nagravision S.A. | Method for rendering audio-video content, decoder for implementing this method and rendering device for rendering this audio-video content |
US9590782B2 (en) | 2015-06-22 | 2017-03-07 | Sony Corporation | Signaling methods and apparatus |
US10205991B2 (en) | 2015-07-27 | 2019-02-12 | Sony Corporation | Hidden replaceable media slots |
CN106997770B (en) | 2016-01-22 | 2023-01-03 | 鼎卓创意科技股份有限公司 | Audio-video synchronous control method, audio-video synchronous control system and related electronic device |
US20190208236A1 (en) * | 2018-01-02 | 2019-07-04 | Source Digital, Inc. | Coordinates as ancillary data |
RU2732198C1 (en) * | 2020-06-15 | 2020-09-14 | Максим Анатольевич Красавин | System for providing interactive media content to user's computing device using static graphic marker |
US11722741B2 (en) | 2021-02-08 | 2023-08-08 | Verance Corporation | System and method for tracking content timeline in the presence of playback rate changes |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6160587A (en) * | 1997-01-16 | 2000-12-12 | Motorola, Inc. | Waveform generator for insertion of data into digital television signals |
US6411725B1 (en) * | 1995-07-27 | 2002-06-25 | Digimarc Corporation | Watermark enabled video objects |
US20040022444A1 (en) * | 1993-11-18 | 2004-02-05 | Rhoads Geoffrey B. | Authentication using a digital watermark |
US20040117856A1 (en) * | 2002-10-01 | 2004-06-17 | Barsoum Yousri H. | Method and apparatus for modulating a video signal with data |
US20060093140A1 (en) * | 2004-10-28 | 2006-05-04 | Macrovision Corporation | Content management for high definition television |
US20080276325A1 (en) * | 2007-05-02 | 2008-11-06 | Macrovision Corporation | Method and apparatus for providing content control via detection of modifications to a signal |
US20090265738A1 (en) * | 2008-04-22 | 2009-10-22 | Kung-Lieh Liao | Method for dynamically displaying auxiliary display data |
Family Cites Families (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5659368A (en) * | 1992-04-28 | 1997-08-19 | Thomson Consumer Electronics, Inc. | Auxiliary video information system including extended data services |
US6614914B1 (en) * | 1995-05-08 | 2003-09-02 | Digimarc Corporation | Watermark embedder and reader |
US6590996B1 (en) * | 2000-02-14 | 2003-07-08 | Digimarc Corporation | Color adaptive watermarking |
US6055023A (en) * | 1996-12-19 | 2000-04-25 | Thomson Consumer Electronics | Television apparatus for simultaneous decoding of auxiliary data included in multiple television signals |
JP2002077572A (en) | 2000-08-29 | 2002-03-15 | Nec Corp | Digital contents generating and reproducing apparatus and advertisement information delivery system |
US7676822B2 (en) * | 2001-01-11 | 2010-03-09 | Thomson Licensing | Automatic on-screen display of auxiliary information |
US20030014748A1 (en) * | 2001-07-16 | 2003-01-16 | Gal Ben-David | Methods for data transmission |
US20030110511A1 (en) * | 2001-12-11 | 2003-06-12 | Schutte Mark E. | Controlling personal video recording functions from interactive television |
KR100764303B1 (en) * | 2004-07-22 | 2007-10-05 | 마츠시타 덴끼 산교 가부시키가이샤 | Multicolored two-dimensional barcode, image display apparatus thereof, information terminal apparatus, display method, decoding method, information communication system, and information communication method |
JP4980913B2 (en) | 2004-09-23 | 2012-07-18 | トムソン ライセンシング | Insert trick play metadata into the video transport stream |
KR100901142B1 (en) * | 2005-08-04 | 2009-06-04 | 니폰 덴신 덴와 가부시끼가이샤 | Digital watermark detecting method, digital watermark detection device, and program |
US20090262238A1 (en) | 2005-12-16 | 2009-10-22 | Stepframe Media, Inc. | Generation And Delivery of Stepped-Frame Content Via MPEG Transport Streams |
CN101558647A (en) | 2006-08-29 | 2009-10-14 | 希尔克瑞斯特实验室公司 | Television control, playlist generation and DVR systems and methods |
KR20080023891A (en) * | 2006-09-12 | 2008-03-17 | 엘지전자 주식회사 | Method for widget type user interface and digital tv thereof |
US9179086B2 (en) | 2007-05-07 | 2015-11-03 | Yahoo! Inc. | System and method for providing dynamically updating applications in a television display environment |
KR100946824B1 (en) | 2007-10-31 | 2010-03-09 | (주)피엑스디 | Digital broadcast widget system and method of displying widget |
US8776117B2 (en) * | 2007-12-28 | 2014-07-08 | Verizon Patent And Licensing Inc. | Method and apparatus for providing expanded displayable applications |
US8886745B2 (en) * | 2008-04-07 | 2014-11-11 | Qualcomm Incorporated | Methods and apparatus for delivering auxiliary data to device |
US20100086283A1 (en) * | 2008-09-15 | 2010-04-08 | Kumar Ramachandran | Systems and methods for updating video content with linked tagging information |
US20100131981A1 (en) * | 2008-11-26 | 2010-05-27 | Samsung Electronics Co., Ltd. | Method for displaying a widget and a broadcast receiving apparatus thereof |
US9191726B2 (en) * | 2009-09-30 | 2015-11-17 | Disney Enterprises, Inc. | System and method for providing media content enhancement |
-
2010
- 2010-03-30 US US12/798,118 patent/US9277183B2/en active Active
- 2010-10-05 MX MX2012004085A patent/MX2012004085A/en active IP Right Grant
- 2010-10-05 WO PCT/US2010/002677 patent/WO2011046590A2/en active Application Filing
- 2010-10-05 KR KR1020127012157A patent/KR101367459B1/en active IP Right Grant
- 2010-10-05 EP EP10823724.9A patent/EP2489181A4/en not_active Ceased
- 2010-10-05 CN CN201510681173.9A patent/CN105263031B/en active Active
- 2010-10-05 CN CN201080046101.2A patent/CN102577366B/en active Active
- 2010-10-05 CA CA2775263A patent/CA2775263C/en active Active
-
2016
- 2016-01-20 US US15/001,514 patent/US20160134909A1/en not_active Abandoned
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040022444A1 (en) * | 1993-11-18 | 2004-02-05 | Rhoads Geoffrey B. | Authentication using a digital watermark |
US6411725B1 (en) * | 1995-07-27 | 2002-06-25 | Digimarc Corporation | Watermark enabled video objects |
US6160587A (en) * | 1997-01-16 | 2000-12-12 | Motorola, Inc. | Waveform generator for insertion of data into digital television signals |
US20040117856A1 (en) * | 2002-10-01 | 2004-06-17 | Barsoum Yousri H. | Method and apparatus for modulating a video signal with data |
US20060093140A1 (en) * | 2004-10-28 | 2006-05-04 | Macrovision Corporation | Content management for high definition television |
US20080276325A1 (en) * | 2007-05-02 | 2008-11-06 | Macrovision Corporation | Method and apparatus for providing content control via detection of modifications to a signal |
US20090265738A1 (en) * | 2008-04-22 | 2009-10-22 | Kung-Lieh Liao | Method for dynamically displaying auxiliary display data |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180152752A1 (en) * | 2016-11-30 | 2018-05-31 | Samsung Electronics Co., Ltd. | Electronic apparatus, controlling method thereof and display system comprising electronic apparatus and a plurality of display apparatuses |
US10123071B2 (en) * | 2016-11-30 | 2018-11-06 | Samsung Electronics Co., Ltd. | Electronic apparatus, controlling method thereof and display system comprising electronic apparatus and a plurality of display apparatuses |
Also Published As
Publication number | Publication date |
---|---|
CN105263031B (en) | 2019-07-16 |
US20110088075A1 (en) | 2011-04-14 |
WO2011046590A2 (en) | 2011-04-21 |
WO2011046590A3 (en) | 2011-07-21 |
CA2775263A1 (en) | 2011-04-21 |
EP2489181A2 (en) | 2012-08-22 |
CA2775263C (en) | 2016-05-03 |
CN105263031A (en) | 2016-01-20 |
EP2489181A4 (en) | 2014-05-21 |
CN102577366B (en) | 2015-10-21 |
CN102577366A (en) | 2012-07-11 |
KR20120067370A (en) | 2012-06-25 |
MX2012004085A (en) | 2012-04-20 |
US9277183B2 (en) | 2016-03-01 |
KR101367459B1 (en) | 2014-02-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9277183B2 (en) | System and method for distributing auxiliary data embedded in video data | |
US11218765B2 (en) | System for distributing metadata embedded in video | |
US11146864B2 (en) | Processing and providing an image in which a plurality of symbols are encoded | |
US9451207B2 (en) | Automatic subtitle resizing | |
KR20140125766A (en) | Image display apparatus and method for operating same | |
KR20150120963A (en) | Video display apparatus and operating method thereof | |
US20120155552A1 (en) | Concealed metadata transmission system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |